CVE-2025-66491: Traefik's "Verify=On" Turned TLS Off

A semantic mismatch in Traefik's experimental ingress-nginx provider inverted TLS verification for five months.

Traefik is one of the most popular cloud-native reverse proxies and ingress controllers in the world. With over 60,000 GitHub stars and more than 3 billion downloads, it routes traffic for countless production systems: financial services, healthcare platforms, SaaS products, and internal control planes across enterprises of every size. When you configure Traefik, you're often configuring the front door to your entire infrastructure.

Our autonomous analyzer, operated by Pavel Kohout of Aisle Research, discovered that Traefik's experimental ingress-nginx provider inverted the semantics of the nginx.ingress.kubernetes.io/proxy-ssl-verify annotation. Setting it to "on" (the NGINX convention for enabling verification) actually disabled TLS certificate verification, exposing HTTPS backends to man-in-the-middle attacks. In Kubernetes, annotations function as policy at scale: a single line can govern thousands of routes across an entire organization. This one word, intended to harden security, did the opposite.

This issue is tracked as CVE-2025-66491 (CWE-295: Improper Certificate Validation). It affected Traefik versions v3.5.0 through v3.6.2 and was fixed in v3.6.3. The bug lived in the codebase for approximately five months, from the provider's introduction on June 23, 2025 until the fix on December 3, 2025.

The scale of Traefik

Traefik sits at the critical path of modern infrastructure. It handles:

Ingress traffic for Kubernetes clusters running in AWS, GCP, Azure, and on-premises data centers
Service mesh routing as part of Traefik Mesh deployments
API gateway functions for microservices architectures
Load balancing across containers, VMs, and serverless functions

The project is backed by Traefik Labs and used by organizations ranging from startups to Fortune 500 companies. Any vulnerability in Traefik's TLS handling has the potential to affect a vast number of systems. The ingress-nginx provider specifically targets teams migrating from NGINX Ingress Controller, meaning it serves security-conscious operators who explicitly configured backend certificate verification.

What we found

The ingress-nginx provider translates NGINX-style annotations into Traefik's internal configuration. The annotation nginx.ingress.kubernetes.io/proxy-ssl-verify tells the ingress controller whether to verify TLS certificates when proxying to HTTPS backends. In NGINX, "on" means "verify the backend certificate" and "off" means "skip verification."

Go's crypto/tls package uses a field called InsecureSkipVerify. When this field is true, verification is skipped. When it is false, verification is performed. The names point in opposite directions: NGINX's verify is a positive assertion, while Go's InsecureSkipVerify is a negative one.

The provider's code at pkg/provider/kubernetes/ingress-nginx/kubernetes.go line 512 contained this logic:

JavaScript
1nst := &namedServersTransport{
2    Name: provider.Normalize(namespace + "-" + name),
3    ServersTransport: &dynamic.ServersTransport{
4        ServerName:         ptr.Deref(cfg.ProxySSLName, ptr.Deref(cfg.ProxySSLServerName, "")),
5        InsecureSkipVerify: strings.ToLower(ptr.Deref(cfg.ProxySSLVerify, "off")) == "on",
6    },
7}

The expression == "on" evaluates to true when the annotation is set to "on". That sets InsecureSkipVerify: true, which disables verification. The mapping was exactly inverted:

The unit test suite unintentionally encoded this inverted behavior as the expected outcome. The test fixture at pkg/provider/kubernetes/ingress-nginx/fixtures/ingresses/07-ingress-with-proxy-ssl.yml set proxy-ssl-verify: "on", and the test at pkg/provider/kubernetes/ingress-nginx/kubernetes_test.go expected InsecureSkipVerify: true:

JavaScript
1ServersTransports: map[string]*dynamic.ServersTransport{
2    "default-ingress-with-proxy-ssl": {
3        ServerName:         "whoami.localhost",
4        InsecureSkipVerify: true,  // expected by the test when annotation is "on"
5        RootCAs:            []types.FileOrContent{"-----BEGIN CERTIFICATE-----"},
6    },
7},

This meant the bug passed CI. A test that looked like it was validating correct behavior was actually validating incorrect behavior. The test became a shield protecting the very flaw it was meant to prevent.

How trivial the bug was

The root cause was a single misplaced comparison operator. The entire vulnerability came down to this expression:

JavaScript
1InsecureSkipVerify: strings.ToLower(ptr.Deref(cfg.ProxySSLVerify, "off")) == "on"

That == "on" should have been == "off". One word. Three characters. That's the difference between "verify certificates" and "accept anything."

The fix (commit 14a1aedf5) was correspondingly simple. A single string changed:

JavaScript
1- InsecureSkipVerify: strings.ToLower(ptr.Deref(cfg.ProxySSLVerify, "off")) == "on",
2+ InsecureSkipVerify: strings.ToLower(ptr.Deref(cfg.ProxySSLVerify, "off")) == "off",

Now the logic correctly maps the annotation: when proxy-ssl-verify is "on", the expression == "off" evaluates to false, so InsecureSkipVerify is false, and verification is enabled. The test expectation was also updated from InsecureSkipVerify: true to InsecureSkipVerify: false.

Why this matters

The Kubernetes policy model at stake

Kubernetes standardizes control through declarative manifests. Annotations extend this model, letting operators express policy intent across clusters and namespaces with a single line of configuration. The ingress-nginx provider exists specifically to let teams migrate from NGINX Ingress Controller to Traefik while preserving their existing annotation-based policies.

When the annotation semantics are inverted, operators who set proxy-ssl-verify: "on" believing they are hardening their security are actually weakening it. They have no warning, no error, and no indication that their intent has been subverted.

Upstream TLS as the last line of defense

TLS verification between the ingress and backend services is often the last cryptographic checkpoint for east-west traffic within a cluster. If an attacker gains a network position between the ingress pod and a backend (through ARP spoofing, a compromised sidecar, or a misconfigured network policy), presenting a self-signed or invalid certificate should fail the connection. With InsecureSkipVerify: true, the attacker's certificate is accepted without question.

Once in this position, the attacker establishes an encrypted connection with Traefik while impersonating your backend. They can read every request in plaintext: user credentials, session tokens, API keys, and sensitive data. They can also modify responses, inject malicious content, or silently reroute traffic to a completely different service. The connection still appears encrypted to operators monitoring the cluster, making detection difficult.

This is particularly dangerous in multi-tenant Kubernetes environments, service meshes with optional mTLS, and any architecture where not all internal traffic is encrypted by the network layer.

The Boolean Mirage pattern

This bug exemplifies what we call the Boolean Mirage: a category of vulnerabilities where security intent expressed in one system gets inverted when translated to another system's primitives. The NGINX world says verify: on. The Go TLS world says InsecureSkipVerify: false. These two positives must meet, but without an explicit inversion step, they collide and produce the opposite of what was intended.

Why traditional scanners miss this

Traditional static analysis tools excel at finding certain patterns: buffer overflows, SQL injection, use-after-free, hardcoded credentials. They look for code that is syntactically or structurally dangerous. But this bug has none of those signatures. The code is clean:

No memory corruption
No injection points
No unsafe functions
No obviously wrong types
Compiles without warnings
Passes all tests

A conventional scanner sees a string comparison setting a boolean field. That's normal. That's fine. There's nothing to flag. The bug is invisible to tools that analyze code in isolation because the code is correct in isolation. The error exists only in the relationship between two systems: what NGINX annotations mean versus what Go TLS fields mean.

Our analyzer approaches this differently. Instead of pattern-matching against known vulnerability signatures, it builds a stack model of how security-relevant values flow through the system. It understands that proxy-ssl-verify is a policy input, that InsecureSkipVerify is a security-critical output, and that the mapping between them must preserve intent. When the semantics drift, when "on" becomes "skip," the analyzer flags the invariant violation. This is the difference between scanning for known bugs and reasoning about whether the code does what it should.

Affected versions and the fix

The ingress-nginx provider was introduced in v3.5.0-rc1 (June 23, 2025). The bug was present from the very first commit.

Release notes: Traefik v3.6.3

Severity assessment

CVSS 3.1 Score: 5.9 (Medium)

Vector: CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N

In plain terms: An attacker who can position themselves in the network path between Traefik and a backend service can intercept HTTPS traffic that operators believed was protected by certificate verification. This requires network access (not just internet access) and typically means the attacker has already compromised some part of the infrastructure. The rating of "High" for Attack Complexity reflects this prerequisite. However, once that position is achieved, interception is straightforward.

Mitigating factors:

Only affects the experimental ingress-nginx provider
Requires explicit operator configuration (the annotation must be set)
The default value ("off") accidentally resulted in verification being enabled (secure by accident, insecure by intent)
Exploitation requires network positioning, not just remote access

Reproduction guide

Deploy Traefik v3.5.0 through v3.6.2 with the experimental ingress-nginx provider enabled
Create an Ingress resource with the following annotations:

JavaScript
1annotations:
2  nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
3  nginx.ingress.kubernetes.io/proxy-ssl-verify: "on"
4  nginx.ingress.kubernetes.io/proxy-ssl-secret: "<namespace>/<secret>"

Configure a backend that presents an invalid or self-signed certificate (one that should fail verification)
Send a request through Traefik to the backend
Observe that the request succeeds despite the invalid certificate, because verification was disabled

Mitigation for operators

Upgrade to v3.6.3 as soon as possible. This is the only complete fix.

If you cannot upgrade immediately: The counterintuitive workaround is to set proxy-ssl-verify: "off" if you want verification enabled, or remove the annotation entirely. The default behavior in vulnerable versions accidentally enables verification when no annotation is present.

This inversion is exactly the bug. We document it here only for operators who cannot immediately upgrade. The fixed release restores intuitive behavior.

Timeline and disclosure

We appreciate the Traefik team's responsiveness and professionalism throughout the disclosure process. Their quick turnaround from report to fix demonstrates the kind of security-conscious development that makes open source infrastructure trustworthy.

The broader pattern

This vulnerability is a case study in cross-system semantic mapping. Provider bridges that translate configuration from one system (NGINX annotations) to another (Go TLS configuration) are security-critical glue code. They must handle:

Naming conventions that differ: "verify" vs. "skip verify"
Boolean polarities that oppose: positive assertion vs. negative assertion
Default values that may not align: what happens when the annotation is absent?
Test cases that may encode the wrong invariant: passing tests don't mean correct behavior

The Traefik maintainers have an excellent track record, and this provider was marked as experimental. The bug slipped through because the logic was locally reasonable (compare a string, set a boolean) but globally incorrect (the comparison inverted the intended semantics). This is the kind of glue-point error that our analyzer is designed to find: not broken code, but broken mappings between systems.

What makes this discovery notable is the combination of factors: a trivial bug (three characters), in critical infrastructure (3+ billion downloads), in a new and actively developed feature (not legacy code), that completely inverted security intent (verify became skip). The ingress-nginx provider was introduced just five months before we found this. It wasn't buried in ancient code that nobody looked at. It was fresh, reviewed, tested, and still wrong.

Acknowledgements

Thank you to the Traefik maintainers for their quick response, their commitment to security, and their continued work on an excellent project. The ingress-nginx provider makes migration easier for many teams, and this fix ensures it does so safely.

Learn more

Interested in Aisle's autonomous security platform? Visit aisle.com or reach out to [email protected].

Technical references: