We’re proud to announce the release of version 1.8 of the HAProxy Kubernetes Ingress Controller!

In this release, we added support for full rootless mode, Prometheus metrics for the controller itself, and examples that are synchronized with our Helm chart. In this blog post, you will learn more about the changes in this version.

Register for our webinar to learn more about this release.

Running unprivileged

Prior to this version, the ingress controller process ran with elevated privileges (root) inside its Docker container. The reason why we had to default to root privileges was to be able to bind to privileged ports (those below 1024), such as the standard HTTP and HTTPS ports 80 and 443. However, containers that run with elevated privileges pose a risk, since it becomes easier to escape the container sandbox and access the host system. In fact, some hosted Kubernetes providers such as Google Kubernetes Engine (GKE) Autopilot even disallow running privileged containers altogether.

Workarounds exist, including having a sidecar container running sysctl net.ipv4.ip_unprivileged_port_start=0 next to the ingress controller container. However, GKE Autopilot would not allow that either since it requires a privileged container to run sysctl.

Now, we no longer run a privileged container by default. Instead, we drop privilege to UID 1000 and GID 1000 for all processes in the container, including the S6, ingress controller, and HAProxy processes. We grant the HAProxy binary the CAP_NET_BIND_SERVICE capability via the setcap command when building the ingress controller Docker image so that it can bind to the desired ports:

This is possibly a breaking change for older host systems such as Debian Stretch, Ubuntu Willy, and Ubuntu Xenial, because their kernels disable some needed features for advanced multi-layered unification filesystem (AUFS), which had been the preferred storage driver for Docker 18.06 and older. Most Docker / Kubernetes installations have already moved to the OverlayFS or OverlayFS2 storage driver, making this problem mostly a legacy systems issue.

HAProxy Kubernetes Ingress Controller 1.8 permits running unprivileged, and the latest Helm chart uses that mode by default.

Diagnostic pprof data

The ingress controller exposes an endpoint for viewing pprof diagnostic data.

Diagnostic pprof data

Diagnostic pprof data

You enable it with the controller argument --pprof. With version 1.8, an additional argument, --controller-port, lets you change the endpoint’s port. By default, if pprof is enabled, it listens at port 6060, which is compatible with version 1.7. This port is also shared with the new Prometheus metrics endpoint, which is described in the next section.

To enable this feature when using Helm, pass the controller.extraArgs parameter:

Access the pprof diagnostics page at the path /debug/pprof.

Prometheus metrics

This version adds a new endpoint for collecting Prometheus metrics that are specific to the inner workings of the controller, such as the amount of memory allocated and CPU time spent.

Prometheus metrics for the controller

Prometheus metrics for the controller

Metrics for the load balancing portion of the controller have been available since version 1.0, which you can access on port 1024 at the URL path /metrics, but this version includes metrics for the controller itself. Access the new metrics on port 6060 at the URL path /metrics.

In order to enable them, two new controller arguments were introduced:

  • --controller-port – port where this will be exposed (defaults to 6060)
  • --prometheus – to enable prometheus

Here is an example, using Helm:

Default backend

The ingress controller routes traffic to a configured set of pods by evaluating routing rules defined in Ingress objects. However, if no ingress rule matches, the request will go to a default backend, which displays a 404 error message. Until now, the --default-backend-service controller argument pointed to the pod that ran the default backend application. However, that approach required running yet another service in your cluster.

With this release, another option is available. If you omit --default-backend-service then a default service controller will create one that’s part of the controller itself. If you run the ingress controller in external mode, you can set --default-backend-port to define a port where the default backend service runs. If you don’t set this, port 6061 will be used.

Examples

The project’s GitHub repository now contains a collection of examples. The examples are synced and produce the same setup as when you use Helm (same namespaces, service names, etc.), which is especially useful for learning how to create everything by hand with kubectl and then comparing it with Helm’s setup. A shortcut for initializing the examples is to invoke the command make example.

Additional improvements

Several other improvements join this release:

  • You can restrict access to your applications by setting the blacklist or whitelist annotations. These now accept pattern files that store the IP addresses, which makes it easier to support long lists of IP addresses.
  • Support for the new annotation client-strict-sni has been added. It returns a TLS error if no certificate is found for the client SNI.
  • Support for a default ingress class has been added.

Contributions

We’d like to thank the code contributors who helped make this version possible!

Moemen MHEDHBI TEST OPTIM REORG BUILD FEATURE BUG DOC
Ivan Matmati REORG TEST BUG FEATURE
Dinko Korunic FEATURE BUILD
Zlatko Bratkovic BUG CLEANUP FEATURE DOC BUILD
Nico Braun FEATURE
Davor Kapsa DOC
Petr Studeny BUG
Jonas Weber FEATURE
Frank Villaro-Dixon BUILD
Daniel Lenar BUG
Jakub Granieczny BUG
LarsBingBong BUG
SHARE THIS ARTICLE