Release notes

The release notes give you an overview of the what’s new, updated, and removed in this version of HAProxy Enterprise Kubernetes Ingress Controller.

Version 3.0 Jump to heading #

Key changes in the HAProxy Kubernetes Ingress Controller 3.0 release include:

Upgraded HAProxy Enterprise

• This version upgrades to HAProxy Enterprise 3.0 as the underlying load balancer engine. From this time onward, the version of the ingress controller will match the HAProxy Enterprise version.

TCP Custom Resource

• A new TCP custom resource allows you to set new listening ports for TCP services and map them to backend services. This simplifies TCP load balancing, which had before required you to create a ConfigMap to define the mapping. You can customize the generated frontend with this custom resource. And by using a Backend custom resource, you can customize the generated backend too.

Backend Custom Resource

• The Backend custom resource definition now has options to add ACLs and http-request options to the backend.

TCP ConfigMap default backend

• Now, configuration options you set in the ingress controller’s ConfigMap will also apply to backends generated for TCP services added through --configmap-tcp-services. Note: This could be a breaking change depending on your configuration.

Custom Resource Definitions v1alpha1 removed

• Support for apiVersion: "core.haproxy.org/v1alpha1" has been removed in this version. Note: This could be a breaking change depending on your configuration.

Prometheus

• Two new Prometheus metrics were added:

  • haproxy_reloads_total
  • haproxy_runtime_socket_connections_total

Logging

• The logs will now show additional messages to indicate when map files change.
• The number of repeating messages has been reduced in certain scenarios. For example, there had been repeated messages when the same service was used for multiple paths in the same ingress.

Performance improvements

• Server slots in the backend are now calculated in a way that on scaling, the ingress controller always has at least scale-server-slots number of empty servers. This is a slightly different approach, but it will produce fewer reloads of HAProxy Enterprise.
• To further reduce the number of reloads, you can add a new annotation named haproxy.com/deployment on your service to link a Deployment resource to the service. This will connect the service to a single deployment so that the ingress controller can get the number of desired replicas and use it as the required number of servers slots.
• Backend updates are now more efficient by using fewer connections to the HAProxy Enterprise Runtime API.
• With this version, the ingress controller will parallelize writing to maps files. To reduce I/O latency, no more than 10 maps will be written in parallel.
• Starting from this version, the ingress controller uses the Runtime API to update TLS certificates. Previously for this action, a HAProxy Enterprise reload was needed. With the new approach, we remove the need for a reload.

Support thread pinning on http/https/healthz/stats

• This version adds controller arguments that allow you to pin threads for better performance of bound listeners:

  • http-bind-thread
  • https-bind-thread
  • healthz-bind-thread
  • stats-bind-thread

Version 1.11 Jump to heading #

Key changes in the HAProxy Enterprise Kubernetes Ingress Controller 1.11 release include:

Upgraded HAProxy Enterprise

• This version upgrades to HAProxy Enterprise 2.8r1 as the underlying load balancer engine.

Custom Resource Definitions graduated to v1

• The backend, defaults, and global custom resource definitions that were introduced in version 1.7 provide a number of benefits compared to using annotations in an Ingress resource. These CRDs allow you to customize the ingress controller in a more expressive way, since CRDs support booleans, strings, integers, enums, arrays, and objects. They also ensure better validation, readability, and versioning of your configuration settings. In version 1.11, the CRDs leave the alpha designation and graduate to v1. They now support all HAProxy configuration options.
• In this version, the API group also changed. Instead of using apiVersion: "core.haproxy.org/v1alpha2" in your custom resources you will use apiVersion: "ingress.v1.haproxy.com/v1", although the old group will still be supported. When migrating to v1, be sure to update your RBAC rules to accommodate this new group.
• An invalid CRD resource will now display an error message when you try to apply it with kubectl, helping you to catch errors earlier before they can go to production.
• Version 1.11 deprecates the alpha1 and alpha2 CRD versions. Support for them may be removed in a future version of the ingress controller.
• A new startup argument --job-check-crd runs a one-time job that updates your CRDs. When you run Helm to install the ingress controller, this argument is automatic so that the job runs once during installation.

QUIC support

• Version 1.11 adds limited support for the QUIC protocol. It will be enabled automatically if you use TLS via the --default-ssl-certificate argument. To disable it, set --disable-quic. You can also change the ports QUIC uses by setting --quic-bind-port and --quic-announce-port. Also, the --quic-alt-svc-max-age lets you change the Alt-Svc HTTP header.

Upgraded s6-overlay

• This version upgrades s6-overlay, which is the init system we use in the container, to version v3.

Port changes

• The HTTP and HTTPS ports on which the ingress controller listens have changed from 80 and 443 to 8080 and 8443, respectively. Mostly, this will affect those who run the ingress controller in external mode. You can change the ports with the --http-bind-port and --https-bind-port startup arguments.

Logging

• All log messages now contain a unique transaction ID to make it easier to trace back and troubleshoot errors.
• A new ConfigMap option named log-format-tcp lets you set the format for TCP traffic logs. This applies to the TCP ConfigMap defined by the --configmap-tcp-services startup argument.

Config snippets

• You can now disable config snippets with the --disable-config-snippets startup argument. It accepts a comma-separated list of any of the following values: backend, frontend, global, and all. For example, a value of --disable-config-snippets=all would disable the following annotations:

  • backend-config-snippet
  • frontend-config-snippet
  • global-config-snippet

Standalone backends

• You can now instruct the ingress controller to create a separate backend for an Ingress resource by setting the standalone-backend annotation. Although this will likely be uncommon, you could use this to allow more customization per ingress.

Security context

• The securityContext section of the ingress controller’s Deployment resource has changed. It now sets runAsNonRoot, allowPrivilegeEscalation, and seccompProfile as follows:

  yaml
  securityContext:
    runAsNonRoot: true
    allowPrivilegeEscalation: false
    runAsUser:  1000
    runAsGroup: 1000
    capabilities:
      drop:
        - ALL
      add:
        - NET_BIND_SERVICE
    seccompProfile:
      type: RuntimeDefault

  yaml
  securityContext:
    runAsNonRoot: true
    allowPrivilegeEscalation: false
    runAsUser:  1000
    runAsGroup: 1000
    capabilities:
      drop:
        - ALL
      add:
        - NET_BIND_SERVICE
    seccompProfile:
      type: RuntimeDefault

Deprecated features

• In this version, the whitelist and blacklist annotations have been deprecated and may be removed in a future version. Please migrate to the allow-list and deny-list annotations.

Version 1.10 Jump to heading #

Key changes in the HAProxy Enterprise Kubernetes Ingress Controller 1.10 release include:

Upgraded HAProxy

• This version upgrades to HAProxy Enterprise 2.7r1 as the underlying load balancer engine.

New tech dump feature

• A new feature for getting a tech dump will assist in troubleshooting.

Dropped support for Beta versions of Ingress

• We no longer support the deprecated Kubernetes APIs extensions/v1beta1 and networking.k8s.io/v1beta1 on Ingress resources. Instead, use networking.k8s.io/v1.

Matching of CPU quotas

• The ingress controller now sets its GOMAXPROCS variable to match the match the Linux container’s CPU quota, providing better performance.

Gateway API (TCPRoute)

• Added support for the TCPRoute resource in Gateway API. This provides a new way of routing TCP traffic, as an alternative to the --configmap-tcp-services startup argument.

Version 1.9 Jump to heading #

Key changes in the HAProxy Enterprise Kubernetes Ingress Controller 1.9 release include:

Upgraded HAProxy Enterprise

• This version upgrades to HAProxy Enterprise 2.6 as the underlying load balancer engine.

Prometheus metrics for config snippets

• The Prometheus endpoint, which provides network traffic metrics, now includes metrics that will help when troubleshooting config snippets. Errors in config snippet files can prevent them from loading. The metric haproxy_active_backend_config_snippets shows successfully loaded, active config snippets; haproxy_inactive_backend_config_snippets shows inactive snippets that didn’t load correctly; haproxy_unable_to_sync_configuration shows a 1 if any config snippet are inactive or a 0 otherwise.

RedHat Docker image

• You can now deploy the ingress controller’s RedHat Enterprise Linux Docker image, hapee-ingress-ubi.

External mode configuration file

• You can store startup arguments in a YAML file when running in external mode. Use the startup argument --config-file to set the YAML file to load.

Other improvements

• The Global, Defaults, and Backend custom resources received minor updates.
• We added trace logs for endpoints and servers. Set the startup argument --log=trace to see trace-level logs.

Version 1.8 Jump to heading #

Key changes in the HAProxy Enterprise Kubernetes Ingress Controller 1.8 release include:

New configuration file

• Startup arguments that previously had to be set as command-line arguments can now be saved in a YAML file by using the --config-file argument, promoting reusability.

Default backend

• When no ingress rules match, the ingress controller displays a default Not Found web page. To enable this in prior versions, you had to set the startup argument --default-backend-service. This is no longer necessary. The ingress controller will self host a web page for this purpose, reducing the number of services deployed to your Kubernetes cluster.

Upgraded HAProxy Enterprise

• This version upgrades to HAProxy Enterprise 2.5 as the underlying load balancer engine.

Security enhancements

• The ingress controller process now runs as a non-root user inside its Docker container while keeping the ability to bind to privileged ports 80 and 443.
• The annotations blacklist and whitelist, which you can use to allow or deny ranges of IP addresses, now accept a file that stores the IP addresses.
• The new annotation client-strict-sni returns an error if no certificate is found for a client SNI.

Debugging

• Since version 1.7, if you include the --pprof startup argument the ingress controller exposes pprof debug data at port 6060 at the URL path /debug/pprof. Now you can change the port by specifying the --controller-port argument.
• If you set the startup argument --prometheus, new Prometheus metrics are exposed at port 6060 at the URL path /metrics. They include diagnostic information such as the amount of memory allocated to the process and CPU time spent.