Ingress tutorials

Load balance TCP services

The HAProxy Kubernetes Ingress Controller can load balance TCP services.

Load balancing TCP services is different from load balancing HTTP services. With HTTP, the ingress controller listens on ports 80 and 443, receiving traffic for all backend services and then routing requests based on the requested DNS hostname or URL path. With TCP, however, you must expose entirely new ports and map them to backend services.

Choose one of the following methods, but use only one. Using both could cause port conflicts:

Define TCP services with a custom resource Jump to heading

Available since

To define the mapping of TCP ports to backend services, you will create a YAML file that contains a TCP custom resource. We will name the file tcp-customresource.yaml.

  1. Install the TCP custom resource definition.

    Skip if using Helm

    If you installed the ingress controller with Helm, this CRD is installed and updated automatically. Once installed, to perform updates on this CRD with Helm see: Update CRDs.

    nix
    kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml
    nix
    kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml
  2. Define the listening TCP port and the Kubernetes service to which it should be mapped:

    tcp-customresource.yaml
    yaml
    apiVersion: ingress.v1.haproxy.org/v1
    kind: TCP
    metadata:
    name: example-service1-tcp
    spec:
    - name: example-tcp
    frontend:
    name: example-frontend
    tcplog: true
    binds:
    - name: bind1
    port: 2000
    service:
    name: example-service1
    port: 3000
    tcp-customresource.yaml
    yaml
    apiVersion: ingress.v1.haproxy.org/v1
    kind: TCP
    metadata:
    name: example-service1-tcp
    spec:
    - name: example-tcp
    frontend:
    name: example-frontend
    tcplog: true
    binds:
    - name: bind1
    port: 2000
    service:
    name: example-service1
    port: 3000

    In this example:

    • The frontend section configure a new frontend in the ingress controller. You can add other frontend arguments. See the CRD reference page (/kubernetes-ingress/community/configuration-reference/tcps-crd/) for a list of supported keywords.
    • The binds section sets TCP port on which to listen for incoming connections.
    • The service section sets the Kubernetes service to relay requests to.
  3. Apply the custom resource:

    nix
    kubectl apply -f tcp-customresource.yaml
    nix
    kubectl apply -f tcp-customresource.yaml
  4. Define your backend service:

    example-service.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service1
    namespace: default
    spec:
    selector:
    app: example-service1
    ports:
    - protocol: TCP
    port: 3000
    targetPort: 3000
    example-service.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service1
    namespace: default
    spec:
    selector:
    app: example-service1
    ports:
    - protocol: TCP
    port: 3000
    targetPort: 3000

    In this example:

    • The service’s name should match the mapped service in the custom resource. For example, example-service1.
    • The protocol should be TCP.
    • The port should match the service’s port in the custom resource. For example, 3000.
    • The targetPort is your pod’s listening port.
  5. Apply the service:

    nix
    kubectl apply -f example-service.yaml
    nix
    kubectl apply -f example-service.yaml
    output
    service/example-service1 created
    output
    service/example-service1 created
  6. If you deployed the ingress controller as a NodePort service, which is the default, then in addition to publishing the listening ports on the ingress controller, you will also need to define NodePort port numbers. How you do this depends on whether you are using Helm.

    • Create or edit your Helm values file to set the service.tcpPorts section:

      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: example-service1
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP
      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: example-service1
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP

      In this example, for each TCP service:

      • Provide a name for the port. The name of the port cannot exceed 11 characters.
      • port and targetPort are both the port at which the ingress controller is listening.
      • nodePort is the port to publish for external access. Valid NodePorts are in the 30000-32767 range.
      • Set protocol to TCP.
    • Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: service-1-tcp containerPort: 2000 protocol: TCP
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: service-1-tcp containerPort: 2000 protocol: TCP

      The tcpPorts we specified in the YAML myvals.yaml file are present in the output above. Note that to the ingress controller, the names of our ports have changed and have -tcp appended to the end of their names. This does not affect operation.

    • To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor.

      You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      Add entries to the ports section for each of your TCP services.

      yaml
      spec:
      [...]
      ports:
      [...]
      - name: example-service1
      nodePort: 30000
      port: 2000
      protocol: TCP
      targetPort: 2000
      yaml
      spec:
      [...]
      ports:
      [...]
      - name: example-service1
      nodePort: 30000
      port: 2000
      protocol: TCP
      targetPort: 2000

      In this example, for each TCP service:

      • Name the port. For example, example-service1.
      • Specify a nodePort. Valid NodePorts are in the 30000-32767 range.
      • Set the port and targetPort to the port the ingress controller service will listen on. For example, 2000.
      • Set the protocol to TCP.
    • Save the changes and close the file. The display shows that the file was edited:

      output
      service/haproxy-kubernetes-ingress edited
      output
      service/haproxy-kubernetes-ingress edited

    You can connect to your TCP service through the load balancer on the port you specify as the nodePort.

See the extended example for more details, including how to customize the generated backend.

Define TCP services with a ConfigMap Jump to heading

Available since

  • HAProxy Kubernetes Ingress Controller 1.4
  • HAProxy Enterprise Kubernetes Ingress Controller 1.4

To define the mapping of TCP ports to backend services, you will create a YAML file that contains a ConfigMap definition. We will name the file tcp-configmap.yaml.

  1. Define the listening TCP ports and the Kubernetes services to which they should be mapped:

    tcp-configmap.yaml
    yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    2000:
    default/example-service1:3000
    2001:
    mynamespace/example-service2:3001
    tcp-configmap.yaml
    yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    2000:
    default/example-service1:3000
    2001:
    mynamespace/example-service2:3001

    In this example:

    • We set the ConfigMap’s name to tcp-configmap, but you can use any name. Later, we will reference this name.
    • In the data section, add the ports that you will open on the ingress controller for receiving traffic, mapped to their associated backend services. Here, the ingress controller will listen on ports 2000 and 2001.
  2. Apply the ConfigMap:

    nix
    kubectl apply -f tcp-configmap.yaml
    nix
    kubectl apply -f tcp-configmap.yaml
    output
    configmap/tcp-configmap created
    output
    configmap/tcp-configmap created
  3. Edit the ingress controller to use the --configmap-tcp-services startup argument to reference the ConfigMap. How you do this depends on whether you are using Helm.

    • Create or edit your Helm values file to set the --configmap-tcp-services argument under the extraArgs key:

      myvals.yaml
      yaml
      controller:
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      myvals.yaml
      yaml
      controller:
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
    • Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress
    • To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor.

      You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
    • Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap in this example).

      yaml
      spec:
      containers:
      - args:
      - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
      - --configmap=haproxy-controller/kubernetes-ingress
      - --http-bind-port=8080
      - --https-bind-port=8443
      - --ingress.class=haproxy
      - --publish-service=haproxy-controller/kubernetes-ingress
      - --log=info
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      [...]
      yaml
      spec:
      containers:
      - args:
      - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
      - --configmap=haproxy-controller/kubernetes-ingress
      - --http-bind-port=8080
      - --https-bind-port=8443
      - --ingress.class=haproxy
      - --publish-service=haproxy-controller/kubernetes-ingress
      - --log=info
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      [...]
    • Save the changes and close the file. The display shows that the file was edited:

      output
      deployment.apps/haproxy-kubernetes-ingress edited
      output
      deployment.apps/haproxy-kubernetes-ingress edited
  4. Define your backend services. The example below defines two services:

    example-services.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service1
    namespace: default
    spec:
    selector:
    app: example-service1
    ports:
    - protocol: TCP
    port: 3000
    targetPort: 3000
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service2
    namespace: mynamespace
    spec:
    selector:
    app: example-service2
    ports:
    - protocol: TCP
    port: 3001
    targetPort: 3001
    example-services.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service1
    namespace: default
    spec:
    selector:
    app: example-service1
    ports:
    - protocol: TCP
    port: 3000
    targetPort: 3000
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: example-service2
    namespace: mynamespace
    spec:
    selector:
    app: example-service2
    ports:
    - protocol: TCP
    port: 3001
    targetPort: 3001

    In this example:

    • The service’s name should match the mapped entry in the ConfigMap. For example, example-service1.
    • The protocol should be TCP.
    • The port should match the service’s port in the ConfigMap. For example, 3000.
    • The targetPort is your pod’s listening port.
  5. Apply the services:

    nix
    kubectl apply -f example-services.yaml
    nix
    kubectl apply -f example-services.yaml
    output
    service/example-service1 created service/example-service2 created
    output
    service/example-service1 created service/example-service2 created
  6. If you deployed the ingress controller as a NodePort service, which is the default, then in addition to publishing the listening ports on the ingress controller, you will also need to define NodePort port numbers. How you do this depends on whether you are using Helm.

    • Create or edit your Helm values file to set the service.tcpPorts section. Also, if you previously set the --configmap-tcp-services argument, keep that too.

      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: example-service1
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP
      - name: example-service2
      port: 2001
      targetPort: 2001
      nodePort: 30001
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: example-service1
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP
      - name: example-service2
      port: 2001
      targetPort: 2001
      nodePort: 30001
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap

      In this example, for each TCP service:

      • Provide a name for the port. The name of the port cannot exceed 11 characters.
      • port and targetPort are both the port at which the ingress controller is listening.
      • nodePort is the port to publish for external access. Valid NodePorts are in the 30000-32767 range.
      • Set protocol to TCP.
    • Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: service-1-tcp containerPort: 2000 protocol: TCP - name: service-2-tcp containerPort: 2001 protocol: TCP
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: service-1-tcp containerPort: 2000 protocol: TCP - name: service-2-tcp containerPort: 2001 protocol: TCP

      The tcpPorts we specified in the YAML myvals.yaml file are present in the output above. Note that to the ingress controller, the names of our ports have changed and have -tcp appended to the end of their names. This does not affect operation.

    • To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor.

      You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress

      Add entries to the ports section for each of your TCP services.

      yaml
      spec:
      [...]
      ports:
      [...]
      - name: example-service1
      nodePort: 30000
      port: 2000
      protocol: TCP
      targetPort: 2000
      - name: example-service2
      nodePort: 30001
      port: 2001
      protocol: TCP
      targetPort: 2001
      yaml
      spec:
      [...]
      ports:
      [...]
      - name: example-service1
      nodePort: 30000
      port: 2000
      protocol: TCP
      targetPort: 2000
      - name: example-service2
      nodePort: 30001
      port: 2001
      protocol: TCP
      targetPort: 2001

      In this example, for each TCP service:

      • Name the port. For example, example-service1.
      • Specify a nodePort. Valid NodePorts are in the 30000-32767 range.
      • Set the port and targetPort to the port the ingress controller service will listen on. For example, 2000.
      • Set the protocol to TCP.
    • Save the changes and close the file. The display shows that the file was edited:

      output
      service/haproxy-kubernetes-ingress edited
      output
      service/haproxy-kubernetes-ingress edited

    You can connect to your TCP service through the load balancer on the port you specify as the nodePort.

Examples Jump to heading

In this section, we’ll show more examples of configuring TCP services in the ingress controller.

Load balance a TCP service using a ConfigMap Jump to heading

In the following example, we will deploy a Pod running BusyBox and we will configure the ingress controller for load balancing traffic to it over TCP. BusyBox provides several Linux utilities and is useful for troubleshooting and testing. We will use an instance of BusyBox to run netcat (nc) which will listen for the incoming traffic.

  1. To deploy an instance of BusyBox in your Kubernetes cluster, copy the following YAML and save it to a file named busybox.yaml:

    busybox.yaml
    yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: busybox-deployment
    labels:
    app: busybox
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: busybox
    template:
    metadata:
    labels:
    app: busybox
    spec:
    containers:
    - name: busybox
    image: busybox
    command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
    ports:
    - containerPort: 5570
    protocol: TCP
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: busybox-service
    spec:
    selector:
    app: busybox
    ports:
    - protocol: TCP
    port: 5570
    targetPort: 5570
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    2000:
    default/busybox-service:5570
    busybox.yaml
    yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: busybox-deployment
    labels:
    app: busybox
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: busybox
    template:
    metadata:
    labels:
    app: busybox
    spec:
    containers:
    - name: busybox
    image: busybox
    command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
    ports:
    - containerPort: 5570
    protocol: TCP
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: busybox-service
    spec:
    selector:
    app: busybox
    ports:
    - protocol: TCP
    port: 5570
    targetPort: 5570
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    2000:
    default/busybox-service:5570

    This YAML contains the definitions for:

    • A Deployment for BusyBox named busybox-deployment. We specify that we want to use the busybox image.
      • We specify a containerPort of 5570. We will run netcat on this port and connect to it through the load balancer.
      • The deployment includes a command that will run netcat (nc) listening on TCP port 5570.
    • A Service named busybox-service that will expose port 5570. Note that in the next steps, we will configure the ingress controller to connect to this port.
    • A ConfigMap named tcp-configmap. This provides the ingress controller with connection information. Note that this ConfigMap belongs to the haproxy-controller namespace (the namespace created when you installed the ingress controller), whereas the other BusyBox components belong to the default namespace. This ConfigMap specifies that the ingress controller will make connection to the service named busybox-service on port 5570. We will map port 2000 to a NodePort in our haproxy-kubernetes-ingress service.
  2. Apply the changes to create the resources:

    nix
    kubectl apply -f busybox.yaml
    nix
    kubectl apply -f busybox.yaml
    output
    deployment.apps/busybox-deployment created service/busybox-service created configmap/tcp-configmap created
    output
    deployment.apps/busybox-deployment created service/busybox-service created configmap/tcp-configmap created
  3. Edit the haproxy-kubernetes-ingress deployment and the haproxy-kubernetes-ingress service to configure the ingress controller to make connection to the Service named busybox-service on port 5570. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

    We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress deployment and service. We will provide a values file to the command using the -f option. This values file will specify an additional argument for the ingress controller deployment and it will configure our ports.

    1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: busybox
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: busybox
      port: 2000
      targetPort: 2000
      nodePort: 30000
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
    2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
      --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
      --namespace haproxy-controller
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: busybox-tcp containerPort: 2000 protocol: TCP
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: busybox-tcp containerPort: 2000 protocol: TCP
    1. To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor:

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
      1. Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap).

        yaml
        spec:
        containers:
        - args:
        - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
        - --configmap=haproxy-controller/kubernetes-ingress
        - --http-bind-port=8080
        - --https-bind-port=8443
        - --ingress.class=haproxy
        - --publish-service=haproxy-controller/kubernetes-ingress
        - --log=info
        - --configmap-tcp-services=haproxy-controller/tcp-configmap
        [...]
        yaml
        spec:
        containers:
        - args:
        - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
        - --configmap=haproxy-controller/kubernetes-ingress
        - --http-bind-port=8080
        - --https-bind-port=8443
        - --ingress.class=haproxy
        - --publish-service=haproxy-controller/kubernetes-ingress
        - --log=info
        - --configmap-tcp-services=haproxy-controller/tcp-configmap
        [...]
      2. Save the changes and close the file. The display shows that the file was edited:

        output
        deployment.apps/haproxy-kubernetes-ingress edited
        output
        deployment.apps/haproxy-kubernetes-ingress edited
    2. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      1. Add an entry in the ports section with the following:

        yaml
        - name: busybox
        port: 2000
        targetPort: 2000
        nodePort: 30000
        protocol: TCP
        yaml
        - name: busybox
        port: 2000
        targetPort: 2000
        nodePort: 30000
        protocol: TCP
      2. Save the changes and close the file. The display shows that the file was edited:

        output
        service/haproxy-kubernetes-ingress edited
        output
        service/haproxy-kubernetes-ingress edited

    You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

Test the connection through the load balancer (click to expand)

To test the connection to the BusyBox instance running netcat through the load balancer:

  1. Get the name of the BusyBox pod by calling kubectl get pod:

    nix
    kubectl get pod
    nix
    kubectl get pod
    Example output
    NAME READY STATUS RESTARTS AGE busybox-deployment-6fbb645fd4-cfkwp 1/1 Running 0 12m
    Example output
    NAME READY STATUS RESTARTS AGE busybox-deployment-6fbb645fd4-cfkwp 1/1 Running 0 12m
  2. From a server that has connection to your cluster, such as the server from which you run kubectl, use netcat to connect to the port you specified as the NodePort for your TCP service when you configured the ingress controller. In this example, the NodePort we specified was 30000.

    nix
    nc 127.0.0.1 30000
    nix
    nc 127.0.0.1 30000
  3. Check the logs of the BusyBox pod to confirm that a connection was made:

    nix
    kubectl logs busybox-deployment-6fbb645fd4-cfkwp
    nix
    kubectl logs busybox-deployment-6fbb645fd4-cfkwp
    output
    connect to [::ffff:10.244.0.10]:5570 from 10-244-0-8.haproxy-kubernetes-ingress.haproxy-controller.svc.cluster.local:60598 ([::ffff:10.244.0.8]:60598)
    output
    connect to [::ffff:10.244.0.10]:5570 from 10-244-0-8.haproxy-kubernetes-ingress.haproxy-controller.svc.cluster.local:60598 ([::ffff:10.244.0.8]:60598)

Load balance an external endpoint using a ConfigMap Jump to heading

In the following example, we will load balance traffic to an external endpoint at ifconfig.info. This service displays your IP address.

We will create a Service and then map an Endpoint to it at port 80.

  1. Copy the following YAML and save it to a file named ifconfig.yaml:

    ifconfig.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: ifcservice
    spec:
    ports:
    - port: 80
    protocol: TCP
    name: http
    ---
    apiVersion: v1
    kind: Endpoints
    metadata:
    name: ifcservice
    subsets:
    - addresses:
    - ip: 104.21.4.246
    ports:
    - port: 80
    protocol: TCP
    name: http
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    1981:
    default/ifcservice:80
    ifconfig.yaml
    yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: ifcservice
    spec:
    ports:
    - port: 80
    protocol: TCP
    name: http
    ---
    apiVersion: v1
    kind: Endpoints
    metadata:
    name: ifcservice
    subsets:
    - addresses:
    - ip: 104.21.4.246
    ports:
    - port: 80
    protocol: TCP
    name: http
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: tcp-configmap
    namespace: haproxy-controller
    data:
    1981:
    default/ifcservice:80

    This YAML contains the definitions for:

    • A Service named ifcservice that will expose port 80. Note that in the next steps, we will configure the ingress controller to connect to this port.
    • An Endpoint mapped to ifcservice. Note that this Endpoint must have the same name as the Service.
    • A ConfigMap named tcp-configmap. This provides the ingress controller with connection information. Note that this ConfigMap belongs to the haproxy-controller namespace (the namespace created when you installed the ingress controller), whereas the other components belong to the default namespace. This ConfigMap specifies that the ingress controller will make connection to the service named ifcservice on port 80. We will map port 1981 to a NodePort in our haproxy-kubernetes-ingress service.
  2. Apply the changes to create the resources:

    nix
    kubectl apply -f busybox.yaml
    nix
    kubectl apply -f busybox.yaml
    output
    service/ifcservice created endpoints/ifcservice created configmap/tcp-configmap created
    output
    service/ifcservice created endpoints/ifcservice created configmap/tcp-configmap created
  3. Edit the haproxy-kubernetes-ingress deployment and the haproxy-kubernetes-ingress service to configure the ingress controller to make connection to the Service named ifcservice on port 80. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

    We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress deployment and service. We will provide a values file to the command using the -f option. This values file will specify an additional argument for the ingress controller deployment and it will configure our ports.

    1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: ifconn
      port: 1981
      targetPort: 1981
      nodePort: 30681
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: ifconn
      port: 1981
      targetPort: 1981
      nodePort: 30681
      protocol: TCP
      extraArgs:
      - --configmap-tcp-services=haproxy-controller/tcp-configmap
    2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
      --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress \
      --namespace haproxy-controller
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: ifconn-tcp containerPort: 1981 protocol: TCP
      output
      Release "haproxy-kubernetes-ingress" has been upgraded. Happy Helming! NAME: haproxy-kubernetes-ingress [...] Service ports mapped are: - name: http containerPort: 8080 protocol: TCP - name: https containerPort: 8443 protocol: TCP - name: stat containerPort: 1024 protocol: TCP - name: quic containerPort: 8443 protocol: UDP - name: ifconn-tcp containerPort: 1981 protocol: TCP
    1. To view/edit the haproxy-kubernetes-ingress deployment, call kubectl edit deployment. The command will open the deployment file in your configured editor:

      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit deployment haproxy-kubernetes-ingress
      1. Add the --configmap-tcp-services argument in args and set the value to the name of your ConfigMap (haproxy-controller/tcp-configmap).

        yaml
        spec:
        containers:
        - args:
        - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
        - --configmap=haproxy-controller/kubernetes-ingress
        - --http-bind-port=8080
        - --https-bind-port=8443
        - --ingress.class=haproxy
        - --publish-service=haproxy-controller/kubernetes-ingress
        - --log=info
        - --configmap-tcp-services=haproxy-controller/tcp-configmap
        [...]
        yaml
        spec:
        containers:
        - args:
        - --default-ssl-certificate=haproxy-controller/kubernetes-ingress-default-cert
        - --configmap=haproxy-controller/kubernetes-ingress
        - --http-bind-port=8080
        - --https-bind-port=8443
        - --ingress.class=haproxy
        - --publish-service=haproxy-controller/kubernetes-ingress
        - --log=info
        - --configmap-tcp-services=haproxy-controller/tcp-configmap
        [...]
      2. Save the changes and close the file. The display shows that the file was edited:

        output
        deployment.apps/haproxy-kubernetes-ingress edited
        output
        deployment.apps/haproxy-kubernetes-ingress edited
    2. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      1. Add an entry in the ports section with the following:

        yaml
        - name: busybox
        port: 1981
        targetPort: 1981
        nodePort: 30681
        protocol: TCP
        yaml
        - name: busybox
        port: 1981
        targetPort: 1981
        nodePort: 30681
        protocol: TCP
      2. Save the changes and close the file. The display shows that the file was edited:

        output
        service/haproxy-kubernetes-ingress edited
        output
        service/haproxy-kubernetes-ingress edited

    You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

Test the connection through the load balancer (click to expand)

To test the connection to the external Endpoint through the load balancer:

  1. From a server that has connection to your cluster, such as the server from which you run kubectl, use curl to connect to the port you specified as the NodePort for your TCP service when you configured the ingress controller. In this example, the NodePort we specified was 30681.

    nix
    curl -H "host: ifconfig.info" 127.0.0.1:30681
    nix
    curl -H "host: ifconfig.info" 127.0.0.1:30681

    Your IP address should display.

    output
    172.31.35.253
    output
    172.31.35.253

Load balance a TCP service using a custom resource Jump to heading

In the following example, we will deploy a Pod running BusyBox and we will configure the ingress controller for load balancing traffic to it over TCP. BusyBox provides several Linux utilities and is useful for troubleshooting and testing. We will use an instance of BusyBox to run netcat (nc) which will listen for the incoming traffic.

  1. To deploy an instance of BusyBox in your Kubernetes cluster, copy the following YAML and save it to a file named busybox.yaml:

    busybox.yaml
    yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: busybox-deployment
    labels:
    app: busybox
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: busybox
    template:
    metadata:
    labels:
    app: busybox
    spec:
    containers:
    - name: busybox
    image: busybox
    command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
    ports:
    - containerPort: 5570
    protocol: TCP
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: busybox-service
    spec:
    selector:
    app: busybox
    ports:
    - protocol: TCP
    port: 5570
    targetPort: 5570
    busybox.yaml
    yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: busybox-deployment
    labels:
    app: busybox
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: busybox
    template:
    metadata:
    labels:
    app: busybox
    spec:
    containers:
    - name: busybox
    image: busybox
    command: ["sh", "-c", "while true; do nc -v -lk -p 5570; done"]
    ports:
    - containerPort: 5570
    protocol: TCP
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: busybox-service
    spec:
    selector:
    app: busybox
    ports:
    - protocol: TCP
    port: 5570
    targetPort: 5570

    This YAML contains the definitions for:

    • A Deployment for BusyBox named busybox-deployment. We specify that we want to use the busybox image.
      • We specify a containerPort of 5570. We will run netcat on this port and connect to it through the load balancer.
      • The deployment includes a command that will run netcat (nc) listening on TCP port 5570.
    • A Service named busybox-service that will expose port 5570. Note that in the next steps, we will configure the ingress controller to connect to this port.
  2. Apply the changes to create the resources:

    nix
    kubectl apply -f busybox.yaml
    nix
    kubectl apply -f busybox.yaml
    output
    deployment.apps/busybox-deployment created service/busybox-service created
    output
    deployment.apps/busybox-deployment created service/busybox-service created
  3. Install the TCP custom resource definition.

    nix
    kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml
    nix
    kubectl apply -f https://www.haproxy.com/documentation/kubernetes-ingress/community/crd/v3-0/ingress.v1.haproxy.org_tcps.yaml
  4. Define the listening TCP port and the Kubernetes service to which it should be mapped:

    tcp-customresource.yaml
    yaml
    apiVersion: ingress.v1.haproxy.org/v1
    kind: TCP
    metadata:
    name: busybox-service-tcp
    spec:
    - name: busybox-tcp
    frontend:
    name: busybox-frontend
    tcplog: true
    binds:
    - name: bind1
    port: 2002
    service:
    name: busybox-service
    port: 5570
    tcp-customresource.yaml
    yaml
    apiVersion: ingress.v1.haproxy.org/v1
    kind: TCP
    metadata:
    name: busybox-service-tcp
    spec:
    - name: busybox-tcp
    frontend:
    name: busybox-frontend
    tcplog: true
    binds:
    - name: bind1
    port: 2002
    service:
    name: busybox-service
    port: 5570
  5. Apply the custom resource:

    nix
    kubectl apply -f tcp-customresource.yaml
    nix
    kubectl apply -f tcp-customresource.yaml
  6. Edit the haproxy-kubernetes-ingress service to map the listening port 2002 to NodePort 30000. Depending on whether you installed the ingress controller with Helm or Kubectl, you can edit these resources as follows:

    We will use the helm upgrade command to automatically update the haproxy-kubernetes-ingress service. We will provide a values file to the command using the -f option. This values file will configure our ports.

    1. Create a file named myvals.yaml and add the following:

      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: busybox
      port: 2002
      targetPort: 2002
      nodePort: 30000
      protocol: TCP
      myvals.yaml
      yaml
      controller:
      name: controller
      service:
      tcpPorts:
      - name: busybox
      port: 2002
      targetPort: 2002
      nodePort: 30000
      protocol: TCP
    2. Execute the helm upgrade command, providing the name of the YAML values file with -f:

      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
      nix
      helm upgrade haproxy-kubernetes-ingress -f myvals.yaml haproxytech/kubernetes-ingress --namespace haproxy-controller
    1. To view/edit the haproxy-kubernetes-ingress service, call kubectl edit service. The command will open the service file in your configured editor:

      You can define your default editor by using the KUBE_EDITOR or EDITOR environment variables or, if neither are defined, vi is used for Linux or notepad for Windows.

      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      nix
      kubectl -n haproxy-controller edit svc haproxy-kubernetes-ingress
      1. Add an entry in the ports section with the following:

        yaml
        - name: busybox
        port: 2002
        targetPort: 2002
        nodePort: 30000
        protocol: TCP
        yaml
        - name: busybox
        port: 2002
        targetPort: 2002
        nodePort: 30000
        protocol: TCP
      2. Save the changes and close the file. The display shows that the file was edited:

        output
        service/haproxy-kubernetes-ingress edited
        output
        service/haproxy-kubernetes-ingress edited
  7. Customize the backend service by defining a Backend custom resource. For example, below we change the load balancing algorithm to be leastconn.

    • Create the custom resource. Below we set the load balancing algorithm to leastconn:

      backend-crd.yaml
      yaml
      apiVersion: "ingress.v1.haproxy.org/v1"
      kind: Backend
      metadata:
      name: example-backend
      namespace: default
      spec:
      config:
      name: example-config
      balance:
      algorithm: "leastconn"
      backend-crd.yaml
      yaml
      apiVersion: "ingress.v1.haproxy.org/v1"
      kind: Backend
      metadata:
      name: example-backend
      namespace: default
      spec:
      config:
      name: example-config
      balance:
      algorithm: "leastconn"
    • Apply the change:

      nix
      kubectl apply -f backend-crd.yaml
      nix
      kubectl apply -f backend-crd.yaml
      output
      backend.ingress.v1.haproxy.org/example-backend created
      output
      backend.ingress.v1.haproxy.org/example-backend created
    • Update your Service definition to use this custom resource by adding the haproxy.org/cr-backend annotation:

      busybox.yaml
      yaml
      apiVersion: v1
      kind: Service
      metadata:
      name: busybox-service
      namespace: default
      annotations:
      haproxy.org/cr-backend: default/example-backend
      spec:
      selector:
      app: busybox
      ports:
      - protocol: TCP
      port: 5570
      targetPort: 5570
      busybox.yaml
      yaml
      apiVersion: v1
      kind: Service
      metadata:
      name: busybox-service
      namespace: default
      annotations:
      haproxy.org/cr-backend: default/example-backend
      spec:
      selector:
      app: busybox
      ports:
      - protocol: TCP
      port: 5570
      targetPort: 5570
    • Apply the service:

      nix
      kubectl apply -f busybox.yaml
      nix
      kubectl apply -f busybox.yaml
      output
      service/busybox-service created
      output
      service/busybox-service created

    In the end, the generated load balancer configuration inside the ingress controller pod will look like this:

    haproxy
    frontend tcpcr_default_busybox-frontend
    mode tcp
    bind :2002 name bind1
    option tcplog
    default_backend default_busybox-service_3000
    backend default_busybox-service_3000
    mode tcp
    balance leastconn
    server SRV_1 10.244.0.2:3000 enabled
    haproxy
    frontend tcpcr_default_busybox-frontend
    mode tcp
    bind :2002 name bind1
    option tcplog
    default_backend default_busybox-service_3000
    backend default_busybox-service_3000
    mode tcp
    balance leastconn
    server SRV_1 10.244.0.2:3000 enabled
Test the connection through the load balancer (click to expand)

To test the connection to the BusyBox instance running netcat through the load balancer:

  1. Get the name of the BusyBox pod by calling kubectl get pod:

    nix
    kubectl get pod
    nix
    kubectl get pod
    Example output
    NAME READY STATUS RESTARTS AGE busybox-deployment-6fbb645fd4-cfkwp 1/1 Running 0 12m
    Example output
    NAME READY STATUS RESTARTS AGE busybox-deployment-6fbb645fd4-cfkwp 1/1 Running 0 12m
  2. From a server that has connection to your cluster, such as the server from which you run kubectl, use netcat to connect to the port you specified as the NodePort for your TCP service when you configured the ingress controller. In this example, the NodePort we specified was 30000.

    nix
    nc 127.0.0.1 30000
    nix
    nc 127.0.0.1 30000
  3. Check the logs of the BusyBox pod to confirm that a connection was made:

    nix
    kubectl logs busybox-deployment-6fbb645fd4-cfkwp
    nix
    kubectl logs busybox-deployment-6fbb645fd4-cfkwp
    output
    connect to [::ffff:10.244.0.10]:5570 from 10-244-0-8.haproxy-kubernetes-ingress.haproxy-controller.svc.cluster.local:60598 ([::ffff:10.244.0.8]:60598)
    output
    connect to [::ffff:10.244.0.10]:5570 from 10-244-0-8.haproxy-kubernetes-ingress.haproxy-controller.svc.cluster.local:60598 ([::ffff:10.244.0.8]:60598)

Do you have any suggestions on how we can improve the content of this page?