Cruma
DashboardDownload

Use instead of ingress-nginx

🧪 Private preview

The Kubernetes ingress controller is in private preview. Its options can change before general availability.

🚧 In development

The controller image and the install manifests are not published yet, so the steps below can't be followed end to end today.

The Cruma controller reads the same networking.k8s.io/v1 Ingress resources as ingress-nginx, so you can move Ingresses over one at a time. This chapter covers what to pick, how to switch, and which nginx.ingress.kubernetes.io/* annotations carry over. Install and the basics are in Kubernetes ingress.

Pick how traffic arrives

ModeTraffic reaches your clusterUse it when
tunnel (default)Through your Cruma tunnel. Nothing needs a public IP, a load balancer or an open portThe cluster is behind NAT or a firewall, or you want no public entry point
directOn the controller's own ports, like ingress-nginx behind a LoadBalancer Service. No tunnel credentials neededYou already have a public IP or load balancer and want the controller to terminate TLS there
bothBoth of the aboveYou move over gradually, or want the tunnel as a second way in

Set the mode for the whole controller, or per Ingress with cruma.io/exposure. See Serve on the cluster's own ports.

Switch one Ingress at a time

The Cruma controller serves only Ingresses of its own class, so ingress-nginx keeps serving everything else while you move.

  1. Install the controller next to ingress-nginx. It creates the IngressClass cruma.
  2. Pick one Ingress and change spec.ingressClassName from nginx to cruma. Add cruma.io/exposure if it should differ from the controller's mode.
  3. Check it: kubectl describe ingress <name> shows an Accepted Event, and one UnsupportedValue Warning for every annotation Cruma doesn't carry over (see below).
  4. Move the traffic: point the hostname's DNS at the new entry (a CNAME to your tunnel, or the controller's address), then repeat with the next Ingress.

To go back, set ingressClassName: nginx again.

An Ingress that has no nginx.ingress.kubernetes.io/* annotation behaves as it did on any other class; the controller adds nothing to it.

Annotations

Annotations apply to every rule of the Ingress that carries them. Two Ingresses that send the same host and path to the same backend must carry the same annotations; the younger one is left out otherwise.

  • Honoured: Cruma applies the annotation with the feature named in the table.
  • Gateway API: Cruma has no annotation for this; an HTTPRoute filter is the way. See Expose a service with an HTTPRoute.
  • Not supported: Cruma ignores the annotation and serves the Ingress without it.

An annotation under nginx.ingress.kubernetes.io/ that is not in the table is treated as not supported. In both cases the Ingress is still served, and the controller publishes one Warning Event per annotation, with reason UnsupportedValue, that names the annotation and what to use instead. Check kubectl describe ingress for them before you move traffic, because a dropped auth-url or auth-type changes who can reach the service.

A supported annotation with a value Cruma can't use is different: it would silently weaken the Ingress, so the Ingress is not served and gets the Warning UnsupportedValue. That covers, for example, backend-protocol: HTTPS, a malformed CIDR, or permanent-redirect together with temporal-redirect.

All names below are prefixed with nginx.ingress.kubernetes.io/.

AnnotationStatusWhat it does or what to use
ssl-redirectHonouredturns the HTTPS redirect of the Ingress' direct hosts off with false
force-ssl-redirectHonouredkeeps the HTTPS redirect on with true, over ssl-redirect
permanent-redirectHonouredRedirect middleware, 301 unless permanent-redirect-code
permanent-redirect-codeHonouredthe status of permanent-redirect: 301, 302, 307 or 308
temporal-redirectHonouredRedirect middleware with status 302
proxy-body-sizeHonouredroute request limit max_request_body_bytes; 0 sets none
proxy-read-timeoutHonouredbackend backend_timeout_seconds, the wait for the response
backend-protocolHonouredupstream protocol: HTTP is HTTP/1.1, GRPC is cleartext HTTP/2
allowlist-source-rangeHonouredIpFilter middleware allow
whitelist-source-rangeHonouredIpFilter middleware allow, the old name of allowlist-source-range
denylist-source-rangeHonouredIpFilter middleware deny
limit-rpsHonouredRateLimit middleware by client IP, per second
limit-rpmHonouredRateLimit middleware by client IP, per minute
enable-corsHonouredAllowCors middleware, with the cors-* annotations below
cors-allow-originHonouredAllowCors origins: *, a list, or * inside an origin as a wildcard
cors-allow-methodsHonouredAllowCors methods
cors-allow-headersHonouredAllowCors request headers
cors-expose-headersHonouredAllowCors exposed response headers
cors-allow-credentialsHonouredAllowCors credentials, on unless false
cors-max-ageHonouredAllowCors preflight cache seconds
rewrite-targetGateway APIUse an HTTPRoute URLRewrite filter
use-regexGateway APIUse an HTTPRoute RegularExpression path match
app-rootGateway APIUse an HTTPRoute RequestRedirect filter on /
canaryGateway APIUse an HTTPRoute with weighted backendRefs
canary-weightGateway APIUse an HTTPRoute with weighted backendRefs
proxy-connect-timeoutNot supportedThe agent has one timeout per route, set by proxy-read-timeout
proxy-send-timeoutNot supportedThe agent has one timeout per route, set by proxy-read-timeout
limit-connectionsNot supportedThe agent limits connections per listener, not per route
auth-typeNot supportedThe controller does not read credentials from Secrets
auth-secretNot supportedThe controller does not read credentials from Secrets
auth-realmNot supportedThe controller does not read credentials from Secrets
auth-urlNot supportedThe agent has no external authentication request
affinityNot supportedThe agent has no sticky sessions
configuration-snippetNot supportedNginx configuration snippets are never supported
server-snippetNot supportedNginx configuration snippets are never supported

Details that differ from ingress-nginx:

  • The IP allow and deny lists look at the address that connects to the controller. Behind another proxy or load balancer that hides the visitor's address, they see the proxy.
  • limit-rps and limit-rpm count per client IP. With both set, the stricter applies. limit-burst-multiplier isn't read.
  • proxy-body-size accepts k, m and g; 0 removes the limit.
  • proxy-read-timeout is the time to wait for the backend's response.
  • The cors-* annotations only take effect with enable-cors: "true"; their defaults are ingress-nginx's.
  • HTTPS to the backend isn't available per Ingress, so backend-protocol: HTTPS is refused rather than served as plain HTTP.

Certificates

On the controller's own ports, Cruma gets a Let's Encrypt certificate for each hostname and redirects HTTP to HTTPS, except under /.well-known/acme-challenge/, so a cert-manager HTTP-01 solver Ingress keeps working. ssl-redirect: "false" turns the redirect off for an Ingress; force-ssl-redirect: "true" keeps it on.

An Ingress whose spec.tls[].secretName names a cert-manager Secret is served with that certificate, so an existing cert-manager setup moves over unchanged. See TLS certificates and cert-manager.

Without a tls entry, through a tunnel Cruma serves its own certificate for the hostname. See TLS for assigned hostnames.

What kubectl get ingress shows

The ADDRESS column shows where the Ingress is reachable: your tunnel's hostname for tunnel, the controller's addresses for direct, and both for both.

$ kubectl get ingress shop
NAME   CLASS   HOSTS              ADDRESS                                       PORTS     AGE
shop   cruma   shop.example.com   203.0.113.7,<tunnel-id>.tun.cruma-cust1.com   80, 443   1m

How the address is chosen is described in Where an object is reachable.