Use instead of ingress-nginx
🧪 Private preview
The Kubernetes ingress controller is in private preview. Its options can change before general availability.
🚧 In development
The controller image and the install manifests are not published yet, so the steps below can't be followed end to end today.
The Cruma controller reads the same networking.k8s.io/v1 Ingress resources as ingress-nginx, so you can move Ingresses over one at a time. This chapter covers what to pick, how to switch, and which nginx.ingress.kubernetes.io/* annotations carry over. Install and the basics are in Kubernetes ingress.
Pick how traffic arrives
| Mode | Traffic reaches your cluster | Use it when |
|---|---|---|
tunnel (default) | Through your Cruma tunnel. Nothing needs a public IP, a load balancer or an open port | The cluster is behind NAT or a firewall, or you want no public entry point |
direct | On the controller's own ports, like ingress-nginx behind a LoadBalancer Service. No tunnel credentials needed | You already have a public IP or load balancer and want the controller to terminate TLS there |
both | Both of the above | You move over gradually, or want the tunnel as a second way in |
Set the mode for the whole controller, or per Ingress with cruma.io/exposure. See Serve on the cluster's own ports.
Switch one Ingress at a time
The Cruma controller serves only Ingresses of its own class, so ingress-nginx keeps serving everything else while you move.
- Install the controller next to ingress-nginx. It creates the IngressClass
cruma. - Pick one Ingress and change
spec.ingressClassNamefromnginxtocruma. Addcruma.io/exposureif it should differ from the controller's mode. - Check it:
kubectl describe ingress <name>shows anAcceptedEvent, and oneUnsupportedValueWarning for every annotation Cruma doesn't carry over (see below). - Move the traffic: point the hostname's DNS at the new entry (a CNAME to your tunnel, or the controller's address), then repeat with the next Ingress.
To go back, set ingressClassName: nginx again.
An Ingress that has no nginx.ingress.kubernetes.io/* annotation behaves as it did on any other class; the controller adds nothing to it.
Annotations
Annotations apply to every rule of the Ingress that carries them. Two Ingresses that send the same host and path to the same backend must carry the same annotations; the younger one is left out otherwise.
- Honoured: Cruma applies the annotation with the feature named in the table.
- Gateway API: Cruma has no annotation for this; an
HTTPRoutefilter is the way. See Expose a service with an HTTPRoute. - Not supported: Cruma ignores the annotation and serves the Ingress without it.
An annotation under nginx.ingress.kubernetes.io/ that is not in the table is treated as not supported. In both cases the Ingress is still served, and the controller publishes one Warning Event per annotation, with reason UnsupportedValue, that names the annotation and what to use instead. Check kubectl describe ingress for them before you move traffic, because a dropped auth-url or auth-type changes who can reach the service.
A supported annotation with a value Cruma can't use is different: it would silently weaken the Ingress, so the Ingress is not served and gets the Warning UnsupportedValue. That covers, for example, backend-protocol: HTTPS, a malformed CIDR, or permanent-redirect together with temporal-redirect.
All names below are prefixed with nginx.ingress.kubernetes.io/.
| Annotation | Status | What it does or what to use |
|---|---|---|
ssl-redirect | Honoured | turns the HTTPS redirect of the Ingress' direct hosts off with false |
force-ssl-redirect | Honoured | keeps the HTTPS redirect on with true, over ssl-redirect |
permanent-redirect | Honoured | Redirect middleware, 301 unless permanent-redirect-code |
permanent-redirect-code | Honoured | the status of permanent-redirect: 301, 302, 307 or 308 |
temporal-redirect | Honoured | Redirect middleware with status 302 |
proxy-body-size | Honoured | route request limit max_request_body_bytes; 0 sets none |
proxy-read-timeout | Honoured | backend backend_timeout_seconds, the wait for the response |
backend-protocol | Honoured | upstream protocol: HTTP is HTTP/1.1, GRPC is cleartext HTTP/2 |
allowlist-source-range | Honoured | IpFilter middleware allow |
whitelist-source-range | Honoured | IpFilter middleware allow, the old name of allowlist-source-range |
denylist-source-range | Honoured | IpFilter middleware deny |
limit-rps | Honoured | RateLimit middleware by client IP, per second |
limit-rpm | Honoured | RateLimit middleware by client IP, per minute |
enable-cors | Honoured | AllowCors middleware, with the cors-* annotations below |
cors-allow-origin | Honoured | AllowCors origins: *, a list, or * inside an origin as a wildcard |
cors-allow-methods | Honoured | AllowCors methods |
cors-allow-headers | Honoured | AllowCors request headers |
cors-expose-headers | Honoured | AllowCors exposed response headers |
cors-allow-credentials | Honoured | AllowCors credentials, on unless false |
cors-max-age | Honoured | AllowCors preflight cache seconds |
rewrite-target | Gateway API | Use an HTTPRoute URLRewrite filter |
use-regex | Gateway API | Use an HTTPRoute RegularExpression path match |
app-root | Gateway API | Use an HTTPRoute RequestRedirect filter on / |
canary | Gateway API | Use an HTTPRoute with weighted backendRefs |
canary-weight | Gateway API | Use an HTTPRoute with weighted backendRefs |
proxy-connect-timeout | Not supported | The agent has one timeout per route, set by proxy-read-timeout |
proxy-send-timeout | Not supported | The agent has one timeout per route, set by proxy-read-timeout |
limit-connections | Not supported | The agent limits connections per listener, not per route |
auth-type | Not supported | The controller does not read credentials from Secrets |
auth-secret | Not supported | The controller does not read credentials from Secrets |
auth-realm | Not supported | The controller does not read credentials from Secrets |
auth-url | Not supported | The agent has no external authentication request |
affinity | Not supported | The agent has no sticky sessions |
configuration-snippet | Not supported | Nginx configuration snippets are never supported |
server-snippet | Not supported | Nginx configuration snippets are never supported |
Details that differ from ingress-nginx:
- The IP allow and deny lists look at the address that connects to the controller. Behind another proxy or load balancer that hides the visitor's address, they see the proxy.
limit-rpsandlimit-rpmcount per client IP. With both set, the stricter applies.limit-burst-multiplierisn't read.proxy-body-sizeacceptsk,mandg;0removes the limit.proxy-read-timeoutis the time to wait for the backend's response.- The
cors-*annotations only take effect withenable-cors: "true"; their defaults are ingress-nginx's. - HTTPS to the backend isn't available per Ingress, so
backend-protocol: HTTPSis refused rather than served as plain HTTP.
Certificates
On the controller's own ports, Cruma gets a Let's Encrypt certificate for each hostname and redirects HTTP to HTTPS, except under /.well-known/acme-challenge/, so a cert-manager HTTP-01 solver Ingress keeps working. ssl-redirect: "false" turns the redirect off for an Ingress; force-ssl-redirect: "true" keeps it on.
An Ingress whose spec.tls[].secretName names a cert-manager Secret is served with that certificate, so an existing cert-manager setup moves over unchanged. See TLS certificates and cert-manager.
Without a tls entry, through a tunnel Cruma serves its own certificate for the hostname. See TLS for assigned hostnames.
What kubectl get ingress shows
The ADDRESS column shows where the Ingress is reachable: your tunnel's hostname for tunnel, the controller's addresses for direct, and both for both.
$ kubectl get ingress shop
NAME CLASS HOSTS ADDRESS PORTS AGE
shop cruma shop.example.com 203.0.113.7,<tunnel-id>.tun.cruma-cust1.com 80, 443 1m
How the address is chosen is described in Where an object is reachable.