An ingress controller is a Kubernetes resource that routes traffic from outside your cluster to services within the cluster. You can learn more about Ingress on the kubernetes.io website.

This section gives instructions for the installation and use of the HAProxy Ingress Controller.

HAProxy Ingress Controller features

The ingress controller gives you the ability to:

  • Use only one IP address and port and direct requests to the correct pod based on the Host header and request path

  • Secure communication with built-in SSL termination

  • Apply rate limits for clients while optionally whitelisting IP addresses

  • Select from among any of HAProxy's load-balancing algorithms

  • Get superior Layer 7 observability with the HAProxy Stats page and Prometheus metrics

  • Set maximum connection limits to backend servers to prevent overloading services

Installing the HAProxy Ingress Controller

Supported HAProxy Enterprise versions

  • 1.8r2 or newer

Other prerequisites

  • Kubernetes v1.1

  • Working knowledge of Kubernetes

Pre-installation checklist

  • The kubectl command-line tool

  • A running Kubernetes cluster

Install the HAProxy Ingress Controller

Note

You can also download the YAML file used for installation and modify it for your needs before you apply it (such as changing the namespace in which the controller runs and/or exposed ports).

  • Launch an instance of the HAProxy ingress controller into a Kubernetes cluster with the command kubectl apply:

$ kubectl apply -f https://raw.githubusercontent.com/haproxytech/kubernetes-ingress/master/deploy/haproxy-ingress.yaml

Check your installation

  • Verify that the controller is correctly installed into your Kubernetes cluster with the command kubectl get pods -A:

kubectl get pods -A

NAMESPACE            NAME                                       READY   STATUS    RESTARTS   AGE
haproxy-controller   ingress-default-backend-5579b8796c-dkk4v   1/1     Running   4          45h
haproxy-controller   haproxy-ingress-7dd4cc4b-x5fkv             1/1     Running   0          27m

Customizing the HAProxy Ingress Controller

You can customize the ingress controller Deployment resource in the file haproxy-ingress.yaml in the repository https://github.com/haproxytech/ by adding any of the following arguments under the section spec.template.spec.containers.args:

Argument

Description

--default-backend-service

Default service that returns a 404 page.

--default-ssl-certificate

Name of secret for SSL certificate. Default: default/tls-secret

--configmap

ConfigMap designated for HAProxy. Default: default/haproxy-configmap

--kubeconfig

Location of kubernetes configuration file.

--namespace-whitelist

The controller watches all namespaces, but you can specify a specific namespace to watch. You can specify this setting multiple times.

--namespace-blacklist

The controller watches all namespaces, but you can blacklist a namespace that you do not want to watch for changes. You can specify this setting multiple times.

Configuring your HAProxy Ingress Controller

You can customize the HAProxy ingress controller using a ConfigMap resource. See Configuration Examples.

The table below gives the customizable options:

Option

Value

Description

check

enabled, disabled (Default): enabled

Enables active health checking (TCP connection attempted) on all backend servers.

forwarded-for

enabled, disabled (Default: enabled)

Adds the X-Forwarded-For HTTP header to requests to capture and relay the client's source IP address to backend servers.

load-balance

roundrobin, static-rr, leastconn, first, source, uri, url_param, hdr(<name>), random, rdp-cookie(<name>) (Default: roundrobin)

Sets the load-balancing algorithm to use for backend servers.

maxconn

<int> (Default: "")

Sets the maximum number of concurrent connections on a frontend server.

nbthread

<int> (Default: "")

Sets the number of worker threads that the HAProxy process must start.

pod-maxconn

<int> (Default: "")

Sets the maximum number of concurrent connections to send to the server.

rate-limit

ON, OFF (Default: OFF)

Adds a limit on the average rate of HTTP requests incoming from a connection's source IP address.

rate-limit-expire

<time> (Default: 30m)

Sets the maximum duration for a client's rate limit, unless that client continues to make requests. This value takes a time suffix, e.g. 30m for 30 minutes.

rate-limit-interval

<time> (Default: 10s)

Sets the time period over which to calculate an average for the client's request rate. This value takes a time suffix, e.g. 10s for the last 10 seconds.

rate-limit-size

<size> (Default: 100k)

Sets the maximum number of IP addresses to track for rate limiting, as this uses more memory. This value takes a size suffix: "k" = 2^20, "m" = 2^20, "g = 2^30.

servers-increment

<int> (Default: 42)

Sets the number of disabled servers to add to the backend in order for the controller to insert new pods dynamically without a reload. When the ingress controller creates new pods and there are not enough disabled servers standing by, it adds a new batch of servers to the number specified here.

servers-increment-max-disabled

<int> (Default: 66)

Sets the maximum number of disabled servers in a backend. Newly created pods activate and run on disabled servers (groups of disabled servers added in the quantity specified in servers-increment). As pods get deleted, the disabled servers remain on the backend; this can result in having too many disabled servers. With this option, when the number of disabled servers exceeds servers-increment-max-disabled, the controller removes the number of disabled servers specified in servers-increment from the backend.

ssl-certificate

<namespace>/<secret> (Default: "")

Sets the name of the Kubernetes secret that contains both the TLS key and certificate.

ssl-numproc

<int> (Default:"")

Sets the number of processes to use for SSL termination.

ssl-redirect

ON, OFF (Default: ON)

Sets whether to redirect traffic from HTTP to HTTPS. By default, this is activated when ssl-certificate is set.

ssl-redirect-code

301, 302, 303 (Default: 302)

Sets the HTTP status code to use when redirecting a client to HTTPS.

timeout-http-request

<time> (Default: 5s)

Sets the maximum time to wait for a complete HTTP request to be sent by the client. This is a form of protection against Slowloris DoS attacks.

timeout-connect

<time> (Default: 5s)

Sets the maximum time to wait for a connection attempt to a server to succeed.

timeout-client

<time> (Default: 50s)

Sets the maximum inactivity time on the client side.

timeout-queue

<time> (Default: 5s)

Sets the maximum time to wait in the queue for a connection slot to a server to be free.

timeout-server

<time> (Default: 50s)

Sets the maximum inactivity time on the server side.

timeout-tunnel

<time> (Default: 1h)

Sets the maximum inactivity time on both the client and server for tunnels (e.g. WebSocket).

timeout-http-keep-alive

<time> (Default: 1m)

Sets the maximum allowed time to wait for a new HTTP request to appear when Keep-Alive is enabled.

whitelist

<comma-separated list of IP addresses and/or CIDRs> (Default: "")

Sets a list of IP addresses or CIDRs to exclude from deny rules, such as rate limiting.

whitelist-with-rate-limit

ON, OFF (Default: OFF)

Specifies whether to exclude the addresses in the whitelist from rate limiting.

Configuration examples

Basic ingress resource

The following ingress resource routes traffic to pods that match the following:

  • service name: web

  • client's Host header: webdemo.com

  • path begins with /

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: web-ingress
  namespace: default
spec:
  rules:
  - host: webdemo.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

Ingress with annotations

The ingress controller runs an instance of HAProxy. You can customize the underlying HAProxy configuration by adding annotations to the ingress resource. Below is an example ingress that shows the available options:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: web-ingress
  namespace: default
  annotations:
    haproxy.org/forwarded-for: "enabled"
    haproxy.org/load-balance: "roundrobin"
    haproxy.org/whitelist: "127.0.0.1,192.168.50.1/24"
    haproxy.org/whiteist-with-rate-limit: "ON"
spec:
  rules:
  - host: webdemo.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

Service with annotations

You can also customize the underlying HAProxy configuration by adding annotations to the Service resource. Below is an example Service with the available options:

apiVersion: v1
kind: Service
metadata:
  labels:
    run: web
  name: web
  annotations:
    haproxy.org/check: "disabled"
    haproxy.org/forwarded-for: "disabled"
    haproxy.org/load-balance: "leastconn"
    haproxy.org/pod-maxconn: "50"
    haproxy.org/whitelist: "127.0.0.1,192.168.50.1/24"
    haproxy.org/whitelist-with-rate-limit: "ON"
spec:
  selector:
    run: web
  ports:
  - name: port-1
    port: 80
    protocol: TCP
    targetPort: 80

ConfigMap resource

You can also customize the underlying HAProxy configuration by adding a ConfigMap resource that matches the value for the --configmap argument passed to the controller, which defaults to default/haproxy-configmap. Below is an example ConfigMap resource with each possible option:

apiVersion: v1
kind: ConfigMap
metadata:
  name: haproxy-configmap
  namespace: default
data:
  check: "enabled"
  forwarded-for: "enabled"
  load-balance: "roundrobin"
  maxconn: "2000"
  nbthread: "1"
  rate-limit: "OFF"
  rate-limit-expire: "30m"
  rate-limit-interval: "10s"
  rate-limit-size: "100k"
  servers-increment: "42"
  servers-increment-max-disabled: "66"
  ssl-certificate: "default/tls-secret"
  ssl-numproc: "1"
  ssl-redirect: "ON"
  ssl-redirect-code: "302"
  timeout-http-request: "5s"
  timeout-connect: "5s"
  timeout-client: "50s"
  timeout-queue: "5s"
  timeout-server: "50s"
  timeout-tunnel: "1h"
  timeout-http-keep-alive: "1m"
  whitelist: "127.0.0.1, 192.168.50.1/24"
  whitelist-with-rate-limit: "ON"

Viewing Metrics

The ingress controller exposes the HAProxy Stats page and Prometheus metrics on container port 1024. To see the metrics, you need to know the NodePort to which Kubernetes mapped this container port.

Display the NodePort

  • Run the kubectl get svc -A command:

    kubectl get svc -A
    
    NAMESPACE            NAME              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                                     AGE
    haproxy-controller   haproxy-ingress   NodePort   10.96.15.205    <none>        80:30279/TCP,443:30775/TCP,1024:31912/TCP   84s
  • Port 1024 mapped to NodePort 31912.

  • The HAProxy Stats page listens at the / URL

  • Prometheus counters are exposed at the /metrics URL