Load balance TCP services

The HAProxy Kubernetes Ingress Controller can load balance TCP services.

Load balancing TCP services is different from load balancing HTTP services. With HTTP, the ingress controller listens on ports 80 and 443, receiving traffic for all backend services and then routing requests based on the requested DNS hostname or URL path. With TCP, however, you must expose entirely new ports and map them to backend services.

Choose one of the following methods, but use only one. Using both could cause port conflicts:

• Define TCP services with a custom resource.
• Define TCP services with a ConfigMap.

Define TCP services with a custom resource Jump to heading #

To define the mapping of TCP ports to backend services, you will create a YAML file that contains a TCP custom resource. We will name the file tcp-customresource.yaml.

1. Install the TCP custom resource definition.

   Skip if using Helm

   If you installed the ingress controller with Helm, this CRD is installed and updated automatically. Once installed, to perform updates on this CRD with Helm see: Update CRDs.

   nix
   kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml

   nix
   kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml

2. Define the listening TCP port and the Kubernetes service to which it should be mapped:

   tcp-customresource.yaml
   yaml
   apiVersion: ingress.v1.haproxy.org/v1
   kind: TCP
   metadata:
     name: example-service1-tcp
   spec:
     - name: example-tcp
       frontend:
         name: example-frontend
         tcplog: true
         binds:
           - name: bind1
             port: 2000
       service:
         name: example-service1
         port: 3000

   tcp-customresource.yaml
   yaml
   apiVersion: ingress.v1.haproxy.org/v1
   kind: TCP
   metadata:
     name: example-service1-tcp
   spec:
     - name: example-tcp
       frontend:
         name: example-frontend
         tcplog: true
         binds:
           - name: bind1
             port: 2000
       service:
         name: example-service1
         port: 3000

   In this example:

   • The frontend section configure a new frontend in the ingress controller. You can add other frontend arguments. See the CRD reference page (/kubernetes-ingress/community/configuration-reference/tcps-crd/) for a list of supported keywords.
   • The binds section sets TCP port on which to listen for incoming connections.
   • The service section sets the Kubernetes service to relay requests to.

3. Apply the custom resource:

   nix
   kubectl apply -f tcp-customresource.yaml

   nix
   kubectl apply -f tcp-customresource.yaml

4. Define your backend service:

   example-service.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service1
     namespace: default
   spec:
     selector:
       app: example-service1
     ports:
     - protocol: TCP
       port: 3000
       targetPort: 3000

   example-service.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service1
     namespace: default
   spec:
     selector:
       app: example-service1
     ports:
     - protocol: TCP
       port: 3000
       targetPort: 3000

   In this example:

   • The service’s name should match the mapped service in the custom resource. For example, example-service1.
   • The protocol should be TCP.
   • The port should match the service’s port in the custom resource. For example, 3000.
   • The targetPort is your pod’s listening port.

5. Apply the service:

   nix
   kubectl apply -f example-service.yaml

   nix
   kubectl apply -f example-service.yaml

   output
   service/example-service1 created

   output
   service/example-service1 created

6. If you deployed the ingress controller as a NodePort service, which is the default, then in addition to publishing the listening ports on the ingress controller, you will also need to define NodePort port numbers. How you do this depends on whether you are using Helm.

   Helm Kubectl

   • Create or edit your Helm values file to set the service.tcpPorts section:

     myvals.yaml
     yaml
     controller:
       name: controller
       service: 
         tcpPorts:
         - name: example-service1
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP

     myvals.yaml
     yaml
     controller:
       name: controller
       service: 
         tcpPorts:
         - name: example-service1
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP

     In this example, for each TCP service:

     • Provide a name for the port. The name of the port cannot exceed 11 characters.
     • port and targetPort are both the port at which the ingress controller is listening.
     • nodePort is the port to publish for external access. Valid NodePorts are in the 30000-32767 range.
     • Set protocol to TCP.

   • Execute the helm upgrade command, providing the name of the YAML values file with -f:

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress
     
     [...]
     
     Service ports mapped are:
       - name: http
         containerPort: 8080
         protocol: TCP
       - name: https
         containerPort: 8443
         protocol: TCP
       - name: stat
         containerPort: 1024
         protocol: TCP
       - name: quic
         containerPort: 8443
         protocol: UDP
       - name: service-1-tcp
         containerPort: 2000
         protocol: TCP

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress
     
     [...]
     
     Service ports mapped are:
       - name: http
         containerPort: 8080
         protocol: TCP
       - name: https
         containerPort: 8443
         protocol: TCP
       - name: stat
         containerPort: 1024
         protocol: TCP
       - name: quic
         containerPort: 8443
         protocol: UDP
       - name: service-1-tcp
         containerPort: 2000
         protocol: TCP

     The tcpPorts we specified in the YAML myvals.yaml file are present in the output above. Note that to the ingress controller, the names of our ports have changed and have -tcp appended to the end of their names. This does not affect operation.

   • To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor.

     You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

     nix
     kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

     nix
     kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

     Add entries to the ports section for each of your TCP services.

     yaml
     spec:
     [...]
       ports:
     [...]
       - name: example-service1
         nodePort: 30000
         port: 2000
         protocol: TCP
         targetPort: 2000

     yaml
     spec:
     [...]
       ports:
     [...]
       - name: example-service1
         nodePort: 30000
         port: 2000
         protocol: TCP
         targetPort: 2000

     In this example, for each TCP service:

     • Name the port. For example, example-service1.
     • Specify a nodePort. Valid NodePorts are in the 30000-32767 range.
     • Set the port and targetPort to the port the ingress controller service will listen on. For example, 2000.
     • Set the protocol to TCP.

   • Save the changes and close the file. The display shows that the file was edited:

     output
     service/haproxy-kubernetes-ingress edited

     output
     service/haproxy-kubernetes-ingress edited

   You can connect to your TCP service through the load balancer on the port you specify as the nodePort.

See the extended example for more details, including how to customize the generated backend.

Define TCP services with a ConfigMap Jump to heading #

To define the mapping of TCP ports to backend services, you will create a YAML file that contains a ConfigMap definition. We will name the file tcp-configmap.yaml.

1. Define the listening TCP ports and the Kubernetes services to which they should be mapped:

   tcp-configmap.yaml
   yaml
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     2000:
       default/example-service1:3000
     2001:
       mynamespace/example-service2:3001

   tcp-configmap.yaml
   yaml
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     2000:
       default/example-service1:3000
     2001:
       mynamespace/example-service2:3001

   In this example:

   • We set the ConfigMap’s name to tcp-configmap, but you can use any name. Later, we will reference this name.
   • In the data section, add the ports that you will open on the ingress controller for receiving traffic, mapped to their associated backend services. Here, the ingress controller will listen on ports 2000 and 2001.

2. Apply the ConfigMap:

   nix
   kubectl apply -f tcp-configmap.yaml

   nix
   kubectl apply -f tcp-configmap.yaml

   output
   configmap/tcp-configmap created

   output
   configmap/tcp-configmap created

3. Edit the ingress controller to use the --configmap-tcp-services startup argument to reference the ConfigMap. How you do this depends on whether you are using Helm.

   Helm Kubectl

   • Create or edit your Helm values file to set the --configmap-tcp-services argument under the extraArgs key:

     myvals.yaml
     yaml
     controller:
       extraArgs:
       - --configmap-tcp-services=haproxy-controller/tcp-configmap

     myvals.yaml
     yaml
     controller:
       extraArgs:
       - --configmap-tcp-services=haproxy-controller/tcp-configmap

   • Execute the helm upgrade command, providing the name of the YAML values file with -f:

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress

   • To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor.

     You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

     nix
     kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

     nix
     kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

   • Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap in this example).

     yaml
     spec:
       containers:
       - args:
         - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
         - --configmap=haproxy-controller/kubernetes-ingress
         - --http-bind-port=8080
         - --https-bind-port=8443
         - --ingress.class=haproxy
         - --publish-service=haproxy-controller/kubernetes-ingress
         - --log=info
         - --configmap-tcp-services=haproxy-controller/tcp-configmap
     [...]

     yaml
     spec:
       containers:
       - args:
         - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
         - --configmap=haproxy-controller/kubernetes-ingress
         - --http-bind-port=8080
         - --https-bind-port=8443
         - --ingress.class=haproxy
         - --publish-service=haproxy-controller/kubernetes-ingress
         - --log=info
         - --configmap-tcp-services=haproxy-controller/tcp-configmap
     [...]

   • Save the changes and close the file. The display shows that the file was edited:

     output
     deployment.apps/haproxy-kubernetes-ingress edited

     output
     deployment.apps/haproxy-kubernetes-ingress edited

4. Define your backend services. The example below defines two services:

   example-services.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service1
     namespace: default
   spec:
     selector:
       app: example-service1
     ports:
     - protocol: TCP
       port: 3000
       targetPort: 3000
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service2
     namespace: mynamespace
   spec:
     selector:
       app: example-service2
     ports:
     - protocol: TCP
       port: 3001
       targetPort: 3001

   example-services.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service1
     namespace: default
   spec:
     selector:
       app: example-service1
     ports:
     - protocol: TCP
       port: 3000
       targetPort: 3000
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: example-service2
     namespace: mynamespace
   spec:
     selector:
       app: example-service2
     ports:
     - protocol: TCP
       port: 3001
       targetPort: 3001

   In this example:

   • The service’s name should match the mapped entry in the ConfigMap. For example, example-service1.
   • The protocol should be TCP.
   • The port should match the service’s port in the ConfigMap. For example, 3000.
   • The targetPort is your pod’s listening port.

5. Apply the services:

   nix
   kubectl apply -f example-services.yaml

   nix
   kubectl apply -f example-services.yaml

   output
   service/example-service1 created
   service/example-service2 created

   output
   service/example-service1 created
   service/example-service2 created

6. If you deployed the ingress controller as a NodePort service, which is the default, then in addition to publishing the listening ports on the ingress controller, you will also need to define NodePort port numbers. How you do this depends on whether you are using Helm.

   Helm Kubectl

   • Create or edit your Helm values file to set the service.tcpPorts section. Also, if you previously set the --configmap-tcp-services argument, keep that too.

     myvals.yaml
     yaml
     controller:
       name: controller
       service: 
         tcpPorts:
         - name: example-service1
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP
         - name: example-service2
           port: 2001
           targetPort: 2001
           nodePort: 30001
           protocol: TCP
       extraArgs:
       - --configmap-tcp-services=haproxy-controller/tcp-configmap

     myvals.yaml
     yaml
     controller:
       name: controller
       service: 
         tcpPorts:
         - name: example-service1
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP
         - name: example-service2
           port: 2001
           targetPort: 2001
           nodePort: 30001
           protocol: TCP
       extraArgs:
       - --configmap-tcp-services=haproxy-controller/tcp-configmap

     In this example, for each TCP service:

     • Provide a name for the port. The name of the port cannot exceed 11 characters.
     • port and targetPort are both the port at which the ingress controller is listening.
     • nodePort is the port to publish for external access. Valid NodePorts are in the 30000-32767 range.
     • Set protocol to TCP.

   • Execute the helm upgrade command, providing the name of the YAML values file with -f:

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     nix
     helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress
     
     [...]
     
     Service ports mapped are:
       - name: http
         containerPort: 8080
         protocol: TCP
       - name: https
         containerPort: 8443
         protocol: TCP
       - name: stat
         containerPort: 1024
         protocol: TCP
       - name: quic
         containerPort: 8443
         protocol: UDP
       - name: service-1-tcp
         containerPort: 2000
         protocol: TCP
       - name: service-2-tcp
         containerPort: 2001
         protocol: TCP

     output
     Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
     NAME: haproxy-kubernetes-ingress
     
     [...]
     
     Service ports mapped are:
       - name: http
         containerPort: 8080
         protocol: TCP
       - name: https
         containerPort: 8443
         protocol: TCP
       - name: stat
         containerPort: 1024
         protocol: TCP
       - name: quic
         containerPort: 8443
         protocol: UDP
       - name: service-1-tcp
         containerPort: 2000
         protocol: TCP
       - name: service-2-tcp
         containerPort: 2001
         protocol: TCP

     The tcpPorts we specified in the YAML myvals.yaml file are present in the output above. Note that to the ingress controller, the names of our ports have changed and have -tcp appended to the end of their names. This does not affect operation.

   • To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor.

     You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

     nix
     kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

     nix
     kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

     Add entries to the ports section for each of your TCP services.

     yaml
     spec:
     [...]
       ports:
     [...]
       - name: example-service1
         nodePort: 30000
         port: 2000
         protocol: TCP
         targetPort: 2000
       - name: example-service2
         nodePort: 30001
         port: 2001
         protocol: TCP
         targetPort: 2001

     yaml
     spec:
     [...]
       ports:
     [...]
       - name: example-service1
         nodePort: 30000
         port: 2000
         protocol: TCP
         targetPort: 2000
       - name: example-service2
         nodePort: 30001
         port: 2001
         protocol: TCP
         targetPort: 2001

     In this example, for each TCP service:

     • Name the port. For example, example-service1.
     • Specify a nodePort. Valid NodePorts are in the 30000-32767 range.
     • Set the port and targetPort to the port the ingress controller service will listen on. For example, 2000.
     • Set the protocol to TCP.

   • Save the changes and close the file. The display shows that the file was edited:

     output
     service/haproxy-kubernetes-ingress edited

     output
     service/haproxy-kubernetes-ingress edited

   You can connect to your TCP service through the load balancer on the port you specify as the nodePort.

Examples Jump to heading #

In this section, we’ll show more examples of configuring TCP services in the ingress controller.

Load balance a TCP service using a ConfigMap Jump to heading #

In the following example, we will deploy a Pod running BusyBox and we will configure the ingress controller for load balancing traffic to it over TCP. BusyBox provides several Linux utilities and is useful for troubleshooting and testing. We will use an instance of BusyBox to run netcat (nc) which will listen for the incoming traffic.

1. To deploy an instance of BusyBox in your Kubernetes cluster, copy the following YAML and save it to a file named busybox.yaml:

   busybox.yaml
   yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: busybox-deployment
     labels:
       app: busybox
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: busybox
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: busybox
           command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
           ports:
           - containerPort: 5570
             protocol: TCP
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: busybox-service
   spec:
     selector:
       app: busybox
     ports:
     - protocol: TCP
       port: 5570
       targetPort: 5570
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     2000:
       default/busybox-service:5570

   busybox.yaml
   yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: busybox-deployment
     labels:
       app: busybox
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: busybox
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: busybox
           command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
           ports:
           - containerPort: 5570
             protocol: TCP
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: busybox-service
   spec:
     selector:
       app: busybox
     ports:
     - protocol: TCP
       port: 5570
       targetPort: 5570
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     2000:
       default/busybox-service:5570

   This YAML contains the definitions for:

   • A Deployment for BusyBox named busybox-deployment. We specify that we want to use the busybox image.
     • We specify a containerPort of 5570. We will run netcat on this port and connect to it through the load balancer.
     • The deployment includes a command that will run netcat (nc) listening on TCP port 5570.
   • A Service named busybox-service that will expose port 5570. Note that in the next steps, we will configure the ingress controller to connect to this port.
   • A ConfigMap named tcp-configmap. This provides the ingress controller with connection information. Note that this ConfigMap belongs to the haproxy-controller namespace (the namespace created when you installed the ingress controller), whereas the other BusyBox components belong to the default namespace. This ConfigMap specifies that the ingress controller will make connection to the service named busybox-service on port 5570. We will map port 2000 to a NodePort in our haproxy-kubernetes-ingress service.

2. Apply the changes to create the resources:

   nix
   kubectl apply -f busybox.yaml

   nix
   kubectl apply -f busybox.yaml

   output
   deployment.apps/busybox-deployment created
   service/busybox-service created
   configmap/tcp-configmap created

   output
   deployment.apps/busybox-deployment created
   service/busybox-service created
   configmap/tcp-configmap created

3. Edit the haproxy-kubernetes-ingress deployment and the haproxy-kubernetes-ingress service to configure the ingress controller to make connection to the Service named busybox-service on port 5570. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

   Helm Kubectl

   We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress deployment and service. We will provide a values file to the command using the -f option. This values file will specify an additional argument for the ingress controller deployment and it will configure our ports.

   1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: busybox
            port: 2000
            targetPort: 2000
            nodePort: 30000
            protocol: TCP
        extraArgs:
        - --configmap-tcp-services=haproxy-controller/tcp-configmap

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: busybox
            port: 2000
            targetPort: 2000
            nodePort: 30000
            protocol: TCP
        extraArgs:
        - --configmap-tcp-services=haproxy-controller/tcp-configmap

   2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
        --namespace haproxy-controller

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
        --namespace haproxy-controller

      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
      NAME: haproxy-kubernetes-ingress
      
      [...]
      
      Service ports mapped are:
        - name: http
          containerPort: 8080
          protocol: TCP
        - name: https
          containerPort: 8443
          protocol: TCP
        - name: stat
          containerPort: 1024
          protocol: TCP
        - name: quic
          containerPort: 8443
          protocol: UDP
        - name: busybox-tcp
          containerPort: 2000
          protocol: TCP

      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
      NAME: haproxy-kubernetes-ingress
      
      [...]
      
      Service ports mapped are:
        - name: http
          containerPort: 8080
          protocol: TCP
        - name: https
          containerPort: 8443
          protocol: TCP
        - name: stat
          containerPort: 1024
          protocol: TCP
        - name: quic
          containerPort: 8443
          protocol: UDP
        - name: busybox-tcp
          containerPort: 2000
          protocol: TCP

   1. To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor:

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

      1. Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap).

         yaml
         spec:
           containers:
           - args:
             - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
             - --configmap=haproxy-controller/kubernetes-ingress
             - --http-bind-port=8080
             - --https-bind-port=8443
             - --ingress.class=haproxy
             - --publish-service=haproxy-controller/kubernetes-ingress
             - --log=info
             - --configmap-tcp-services=haproxy-controller/tcp-configmap
         [...]

         yaml
         spec:
           containers:
           - args:
             - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
             - --configmap=haproxy-controller/kubernetes-ingress
             - --http-bind-port=8080
             - --https-bind-port=8443
             - --ingress.class=haproxy
             - --publish-service=haproxy-controller/kubernetes-ingress
             - --log=info
             - --configmap-tcp-services=haproxy-controller/tcp-configmap
         [...]

      2. Save the changes and close the file. The display shows that the file was edited:

         output
         deployment.apps/haproxy-kubernetes-ingress edited

         output
         deployment.apps/haproxy-kubernetes-ingress edited

   2. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      1. Add an entry in the ports section with the following:

         yaml
         - name: busybox
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP

         yaml
         - name: busybox
           port: 2000
           targetPort: 2000
           nodePort: 30000
           protocol: TCP

      2. Save the changes and close the file. The display shows that the file was edited:

         output
         service/haproxy-kubernetes-ingress edited

         output
         service/haproxy-kubernetes-ingress edited

   You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

Load balance an external endpoint using a ConfigMap Jump to heading #

In the following example, we will load balance traffic to an external endpoint at ifconfig.info. This service displays your IP address.

We will create a Service and then map an Endpoint to it at port 80.

1. Copy the following YAML and save it to a file named ifconfig.yaml:

   ifconfig.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: ifcservice
   spec:
     ports:
     - port: 80
       protocol: TCP
       name: http
   ---
   apiVersion: v1
   kind: Endpoints
   metadata:
     name: ifcservice
   subsets:
   - addresses:
     - ip: 104.21.4.246
     ports:
     - port: 80
       protocol: TCP
       name: http
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     1981:
       default/ifcservice:80

   ifconfig.yaml
   yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: ifcservice
   spec:
     ports:
     - port: 80
       protocol: TCP
       name: http
   ---
   apiVersion: v1
   kind: Endpoints
   metadata:
     name: ifcservice
   subsets:
   - addresses:
     - ip: 104.21.4.246
     ports:
     - port: 80
       protocol: TCP
       name: http
   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: tcp-configmap
     namespace: haproxy-controller
   data:
     1981:
       default/ifcservice:80

   This YAML contains the definitions for:

   • A Service named ifcservice that will expose port 80. Note that in the next steps, we will configure the ingress controller to connect to this port.
   • An Endpoint mapped to ifcservice. Note that this Endpoint must have the same name as the Service.
   • A ConfigMap named tcp-configmap. This provides the ingress controller with connection information. Note that this ConfigMap belongs to the haproxy-controller namespace (the namespace created when you installed the ingress controller), whereas the other components belong to the default namespace. This ConfigMap specifies that the ingress controller will make connection to the service named ifcservice on port 80. We will map port 1981 to a NodePort in our haproxy-kubernetes-ingress service.

2. Apply the changes to create the resources:

   nix
   kubectl apply -f busybox.yaml

   nix
   kubectl apply -f busybox.yaml

   output
   service/ifcservice created
   endpoints/ifcservice created
   configmap/tcp-configmap created

   output
   service/ifcservice created
   endpoints/ifcservice created
   configmap/tcp-configmap created

3. Edit the haproxy-kubernetes-ingress deployment and the haproxy-kubernetes-ingress service to configure the ingress controller to make connection to the Service named ifcservice on port 80. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

   Helm Kubectl

   We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress deployment and service. We will provide a values file to the command using the -f option. This values file will specify an additional argument for the ingress controller deployment and it will configure our ports.

   1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: ifconn
            port: 1981
            targetPort: 1981
            nodePort: 30681
            protocol: TCP
        extraArgs:
        - --configmap-tcp-services=haproxy-controller/tcp-configmap

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: ifconn
            port: 1981
            targetPort: 1981
            nodePort: 30681
            protocol: TCP
        extraArgs:
        - --configmap-tcp-services=haproxy-controller/tcp-configmap

   2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
        --namespace haproxy-controller

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
        --namespace haproxy-controller

      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
      NAME: haproxy-kubernetes-ingress
      
      [...]
      
      Service ports mapped are:
        - name: http
          containerPort: 8080
          protocol: TCP
        - name: https
          containerPort: 8443
          protocol: TCP
        - name: stat
          containerPort: 1024
          protocol: TCP
        - name: quic
          containerPort: 8443
          protocol: UDP
        - name: ifconn-tcp
          containerPort: 1981
          protocol: TCP

      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming!
      NAME: haproxy-kubernetes-ingress
      
      [...]
      
      Service ports mapped are:
        - name: http
          containerPort: 8080
          protocol: TCP
        - name: https
          containerPort: 8443
          protocol: TCP
        - name: stat
          containerPort: 1024
          protocol: TCP
        - name: quic
          containerPort: 8443
          protocol: UDP
        - name: ifconn-tcp
          containerPort: 1981
          protocol: TCP

   1. To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor:

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress

      1. Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap).

         yaml
         spec:
           containers:
           - args:
             - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
             - --configmap=haproxy-controller/kubernetes-ingress
             - --http-bind-port=8080
             - --https-bind-port=8443
             - --ingress.class=haproxy
             - --publish-service=haproxy-controller/kubernetes-ingress
             - --log=info
             - --configmap-tcp-services=haproxy-controller/tcp-configmap
         [...]

         yaml
         spec:
           containers:
           - args:
             - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
             - --configmap=haproxy-controller/kubernetes-ingress
             - --http-bind-port=8080
             - --https-bind-port=8443
             - --ingress.class=haproxy
             - --publish-service=haproxy-controller/kubernetes-ingress
             - --log=info
             - --configmap-tcp-services=haproxy-controller/tcp-configmap
         [...]

      2. Save the changes and close the file. The display shows that the file was edited:

         output
         deployment.apps/haproxy-kubernetes-ingress edited

         output
         deployment.apps/haproxy-kubernetes-ingress edited

   2. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      1. Add an entry in the ports section with the following:

         yaml
         - name: busybox
           port: 1981
           targetPort: 1981
           nodePort: 30681
           protocol: TCP

         yaml
         - name: busybox
           port: 1981
           targetPort: 1981
           nodePort: 30681
           protocol: TCP

      2. Save the changes and close the file. The display shows that the file was edited:

         output
         service/haproxy-kubernetes-ingress edited

         output
         service/haproxy-kubernetes-ingress edited

   You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

Load balance a TCP service using a custom resource Jump to heading #

In the following example, we will deploy a Pod running BusyBox and we will configure the ingress controller for load balancing traffic to it over TCP. BusyBox provides several Linux utilities and is useful for troubleshooting and testing. We will use an instance of BusyBox to run netcat (nc) which will listen for the incoming traffic.

1. To deploy an instance of BusyBox in your Kubernetes cluster, copy the following YAML and save it to a file named busybox.yaml:

   busybox.yaml
   yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: busybox-deployment
     labels:
       app: busybox
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: busybox
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: busybox
           command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
           ports:
           - containerPort: 5570
             protocol: TCP
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: busybox-service
   spec:
     selector:
       app: busybox
     ports:
     - protocol: TCP
       port: 5570
       targetPort: 5570

   busybox.yaml
   yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: busybox-deployment
     labels:
       app: busybox
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: busybox
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: busybox
           command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
           ports:
           - containerPort: 5570
             protocol: TCP
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: busybox-service
   spec:
     selector:
       app: busybox
     ports:
     - protocol: TCP
       port: 5570
       targetPort: 5570

   This YAML contains the definitions for:

   • A Deployment for BusyBox named busybox-deployment. We specify that we want to use the busybox image.
     • We specify a containerPort of 5570. We will run netcat on this port and connect to it through the load balancer.
     • The deployment includes a command that will run netcat (nc) listening on TCP port 5570.
   • A Service named busybox-service that will expose port 5570. Note that in the next steps, we will configure the ingress controller to connect to this port.

2. Apply the changes to create the resources:

   nix
   kubectl apply -f busybox.yaml

   nix
   kubectl apply -f busybox.yaml

   output
   deployment.apps/busybox-deployment created
   service/busybox-service created

   output
   deployment.apps/busybox-deployment created
   service/busybox-service created

3. Install the TCP custom resource definition.

   nix
   kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml

   nix
   kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml

4. Define the listening TCP port and the Kubernetes service to which it should be mapped:

   tcp-customresource.yaml
   yaml
   apiVersion: ingress.v1.haproxy.org/v1
   kind: TCP
   metadata:
     name: busybox-service-tcp
   spec:
     - name: busybox-tcp
       frontend:
         name: busybox-frontend
         tcplog: true
         binds:
           - name: bind1
             port: 2002
       service:
         name: busybox-service
         port: 5570

   tcp-customresource.yaml
   yaml
   apiVersion: ingress.v1.haproxy.org/v1
   kind: TCP
   metadata:
     name: busybox-service-tcp
   spec:
     - name: busybox-tcp
       frontend:
         name: busybox-frontend
         tcplog: true
         binds:
           - name: bind1
             port: 2002
       service:
         name: busybox-service
         port: 5570

5. Apply the custom resource:

   nix
   kubectl apply -f tcp-customresource.yaml

   nix
   kubectl apply -f tcp-customresource.yaml

6. Edit the haproxy-kubernetes-ingress service to map the listening port 2002 to NodePort 30000. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

   Helm Kubectl

   We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress service. We will provide a values file to the command using the -f option. This values file will configure our ports.

   1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: busybox
            port: 2002
            targetPort: 2002
            nodePort: 30000
            protocol: TCP

      myvals.yaml
      yaml
      controller:
        name: controller
        service: 
          tcpPorts:
          - name: busybox
            port: 2002
            targetPort: 2002
            nodePort: 30000
            protocol: TCP

   2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller

   1. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      1. Add an entry in the ports section with the following:

         yaml
         - name: busybox
           port: 2002
           targetPort: 2002
           nodePort: 30000
           protocol: TCP

         yaml
         - name: busybox
           port: 2002
           targetPort: 2002
           nodePort: 30000
           protocol: TCP

      2. Save the changes and close the file. The display shows that the file was edited:

         output
         service/haproxy-kubernetes-ingress edited

         output
         service/haproxy-kubernetes-ingress edited

7. Customize the backend service by defining a Backend custom resource. For example, below we change the load balancing algorithm to be leastconn.

   • Create the custom resource. Below we set the load balancing algorithm to leastconn:

     backend-crd.yaml
     yaml
     apiVersion: "ingress.v1.haproxy.org/v1"
     kind: Backend
     metadata:
       name: example-backend
       namespace: default
     spec:
       config:
         name: example-config
         balance:
           algorithm: "leastconn"

     backend-crd.yaml
     yaml
     apiVersion: "ingress.v1.haproxy.org/v1"
     kind: Backend
     metadata:
       name: example-backend
       namespace: default
     spec:
       config:
         name: example-config
         balance:
           algorithm: "leastconn"

   • Apply the change:

     nix
     kubectl apply -f backend-crd.yaml

     nix
     kubectl apply -f backend-crd.yaml

     output
     backend.ingress.v1.haproxy.org/example-backend created

     output
     backend.ingress.v1.haproxy.org/example-backend created

   • Update your Service definition to use this custom resource by adding the haproxy.org/cr-backend annotation:

     busybox.yaml
     yaml
     apiVersion: v1
     kind: Service
     metadata:
       name: busybox-service
       namespace: default
       annotations:
         haproxy.org/cr-backend: default/example-backend
     spec:
       selector:
         app: busybox
       ports:
       - protocol: TCP
         port: 5570
         targetPort: 5570

     busybox.yaml
     yaml
     apiVersion: v1
     kind: Service
     metadata:
       name: busybox-service
       namespace: default
       annotations:
         haproxy.org/cr-backend: default/example-backend
     spec:
       selector:
         app: busybox
       ports:
       - protocol: TCP
         port: 5570
         targetPort: 5570

   • Apply the service:

     nix
     kubectl apply -f busybox.yaml

     nix
     kubectl apply -f busybox.yaml

     output
     service/busybox-service created

     output
     service/busybox-service created

   In the end, the generated load balancer configuration inside the ingress controller pod will look like this:

   haproxy
   frontend tcpcr_default_busybox-frontend
     mode tcp
     bind :2002 name bind1
     option tcplog
     default_backend default_busybox-service_3000
   backend default_busybox-service_3000
     mode tcp
     balance leastconn
     server SRV_1 10.244.0.2:3000 enabled

   haproxy
   frontend tcpcr_default_busybox-frontend
     mode tcp
     bind :2002 name bind1
     option tcplog
     default_backend default_busybox-service_3000
   backend default_busybox-service_3000
     mode tcp
     balance leastconn
     server SRV_1 10.244.0.2:3000 enabled