High availability

Active/Active clustering

In an active-active cluster, two or more HAProxy Enterprise nodes receive traffic in a load-balanced rotation. This allows you to scale out your load-balancing capacity. There are several ways to achieve this setup, including:

  • Using DNS round-robin
  • Deploy two tiers of load balancers
  • Using Anycast with the HAProxy Enterprise Route Health Injection module (see Route Health Injection)

Option 1: DNS round-robin Jump to heading

GSLB for DNS round-robin

You can use an HAProxy Enterprise server for DNS round-robin. See GSLB.

To create an active-active cluster of load balancers, you can use DNS round-robin to send traffic to each load balancer in a rotation.

To remove unhealthy HAProxy Enterprise nodes from round-robin load-balancing rotation, you need a way to detect failure. If your DNS server supports health checking upstream servers, you can use that feature to remove unhealthy nodes. However, in this section, we assume that your DNS server does not support health checking. Instead, you need to configure the HAProxy Enterprise VRRP module to assign a virtual IP address to each load balancer. In the event that one of your load balancers fails, its virtual IP address will transfer to a healthy load balancer node.

A side effect of this failover is that your healthy node, which had been receiving its own share of traffic already, will temporarily take on more traffic until the other node recovers.

In this guide, we set up two load balancers: both actively receiving traffic.

Configure VRRP Jump to heading

  1. Install the VRRP module using your system’s package manager on both HAProxy Enterprise nodes that will participate in your load balancer cluster:

    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp
  2. On the first load balancer, edit the file /etc/hapee-extras/hapee-vrrp.cfg. By default, this file is configured for an active-standby cluster. Replace the vrrp_instance block with the following contents.

    • Replace the interface lines with the name of the network interface on which this server receives traffic.
    • Replace the IP addresses listed in the virtual_ipaddress_excluded blocks with addresses you’d like to use to receive traffic. These new addresses should fall within the interface’s IP subnet, but should not be assigned to any server already.
    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
    interface enp0s8 # Change network interface name
    state MASTER
    virtual_router_id 51
    priority 101
    virtual_ipaddress_excluded {
    192.168.50.10 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    vrrp_instance vrrp_2 {
    interface enp0s8 # Change network interface name
    state BACKUP
    virtual_router_id 52
    priority 100
    virtual_ipaddress_excluded {
    192.168.50.11 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
    interface enp0s8 # Change network interface name
    state MASTER
    virtual_router_id 51
    priority 101
    virtual_ipaddress_excluded {
    192.168.50.10 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    vrrp_instance vrrp_2 {
    interface enp0s8 # Change network interface name
    state BACKUP
    virtual_router_id 52
    priority 100
    virtual_ipaddress_excluded {
    192.168.50.11 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }

    This creates two virtual IP addresses: 192.168.50.10 and 192.168.50.11. The first has its state set to MASTER, while the second is set to BACKUP. This means that this load balancer owns the first IP address but will only bind to the second IP address as a backup, meaning when its primary owner fails.

  3. On the other load balancer, configure the VRRP module in a similar way, but swap the state values so that the first vrrp_instance block is set to BACKUP, while the second is set to MASTER. Also, swap the priority values. In this way, each load balancer is a backup for the other. If one fails, its virtual IP will transfer to the other node, which will at that point answer requests from both addresses until the failed node recovers.

    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
    interface enp0s8 # Change network interface name
    state BACKUP
    virtual_router_id 51
    priority 100
    virtual_ipaddress_excluded {
    192.168.50.10 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    vrrp_instance vrrp_2 {
    interface enp0s8 # Change network interface name
    state MASTER
    virtual_router_id 52
    priority 101
    virtual_ipaddress_excluded {
    192.168.50.11 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
    interface enp0s8 # Change network interface name
    state BACKUP
    virtual_router_id 51
    priority 100
    virtual_ipaddress_excluded {
    192.168.50.10 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
    vrrp_instance vrrp_2 {
    interface enp0s8 # Change network interface name
    state MASTER
    virtual_router_id 52
    priority 101
    virtual_ipaddress_excluded {
    192.168.50.11 # NEW IP address
    }
    track_interface {
    enp0s8 weight -2 # Change network interface name
    }
    track_script {
    chk_sshd
    chk_lb
    }
    }
  4. Start the hapee-extras-vrrp service on both servers:

    nix
    sudo systemctl start hapee-extras-vrrp
    sudo systemctl enable hapee-extras-vrrp
    nix
    sudo systemctl start hapee-extras-vrrp
    sudo systemctl enable hapee-extras-vrrp
  5. Edit your HAProxy Enterprise configurations to listen on the virtual IP addresses. Note that you must append the field transparent, indicating that the address will be bound even if it does not belong to the local machine, which is necessary since the virtual IP will float to the active load balancer.

    haproxy
    frontend myfrontend
    mode http
    bind 192.168.50.10:80 transparent
    bind 192.168.50.11:80 transparent
    default_backend web_servers
    backend web_servers
    server s1 192.168.50.20:80
    haproxy
    frontend myfrontend
    mode http
    bind 192.168.50.10:80 transparent
    bind 192.168.50.11:80 transparent
    default_backend web_servers
    backend web_servers
    server s1 192.168.50.20:80

    Alternatively, if you prefer to not add transparent to every bind line, you can set the kernel parameter ip_nonlocal_bind. Edit the file /etc/sysctl.conf and add the line net.ipv4.ip_nonlocal_bind=1:

    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1
    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1

    Then run sudo sysctl -p to reload the configuration.

    As a third option, you can listen on all IP addresses assigned to the server by omitting the IP address:

    haproxy
    frontend myfrontend
    mode http
    bind :80
    default_backend web_servers
    haproxy
    frontend myfrontend
    mode http
    bind :80
    default_backend web_servers

In essence, we have created an active-active cluster by creating two active-standby clusters, one in each direction. Each load balancer has an active IP address, but also serves as a standby server for the other IP address.

Configure DNS round-robin Jump to heading

DNS allows you to rotate which IP address is returned in response to a DNS query. When clients request your service by its domain name, they will receive the IP address of the next load balancer in the list. Clients tend to cache DNS results, so once a client receives a DNS answer, it will likely continue making requests to the same load balancer until the DNS answer expires.

Create an A record for each load balancer IP address. For example, consider the following DNS zone file:

text
$ORIGIN example.local.
@ 3600 IN SOA dns1.example.local. admin.example.local. (
2017042745 ; serial
1800 ; refresh (30 minutes)
900 ; retry (15 minutes)
1209600 ; expire (2 weeks)
60 ; minimum (1 minute)
)
3600 IN NS dns1.example.local.
@ 60 IN A 192.168.50.10
@ 60 IN A 192.168.50.11
www 60 IN CNAME @
text
$ORIGIN example.local.
@ 3600 IN SOA dns1.example.local. admin.example.local. (
2017042745 ; serial
1800 ; refresh (30 minutes)
900 ; retry (15 minutes)
1209600 ; expire (2 weeks)
60 ; minimum (1 minute)
)
3600 IN NS dns1.example.local.
@ 60 IN A 192.168.50.10
@ 60 IN A 192.168.50.11
www 60 IN CNAME @

Depending on your DNS server, you may need to enable load-balancing of the A records. It is also a good idea to set a shorter TTL for these records to avoid staying cached in intermediate nameservers for long.

Option 2: Two tiers of load balancers Jump to heading

To create an active-active cluster of load balancers, you can place another tier of HAProxy Enterprise load balancers in front. These load balancers operate at Layer 4, the Network layer, and load balance the load balancers. Because HAProxy Enterprise has the ability to check the health of the lower tier of load balancers, it can remove unhealthy nodes as needed.

Layer 4 load-balancing

Deploy Layer 7 load balancers Jump to heading

Create a tier of load balancers that will, themselves, be load-balanced:

  1. Deploy at least two servers to host HAProxy Enterprise. Add one or more static IP addresses to each server.

  2. Install HAProxy Enterprise on each server.

  3. Configure them to receive traffic on the assigned IP addresses and relay it to backend application servers. Other than the addresses, their configurations should match.

Deploy Layer 4 load balancers Jump to heading

Create a tier of load balancers that will load balance the layer 7 HAProxy Enterprise servers:

  1. Deploy one or more servers to host HAProxy Enterprise. These will load balance the other load balancers.

  2. If you deploy multiple load balancers in this tier, configure them in active-standby mode.

  3. Configure them to relay traffic to the layer 7 load balancers. In the following example, a listen section defines the pool of upstream load balancers:

    haproxy
    listen load_balancer_cluster
    mode tcp
    bind :80
    option tcplog
    balance roundrobin
    server lb1 192.168.1.25:80 check
    server lb2 192.168.1.26:80 check
    haproxy
    listen load_balancer_cluster
    mode tcp
    bind :80
    option tcplog
    balance roundrobin
    server lb1 192.168.1.25:80 check
    server lb2 192.168.1.26:80 check

Active/active setup on AWS Jump to heading

To create an active-active cluster of HAProxy Enterprise load balancers in AWS, you can place a Network Load Balancer in front. This load balancer operates at Layer 4, the Network layer, and load balances the HAProxy Enterprise load balancers. Because AWS Network Load Balancer can check the health of the lower-tier of load balancers, it can remove unhealthy load balancers as needed.

Layer 4 load-balancing in AWS

We will:

  • Create a virtual network dedicated to your AWS account (VPC).
  • Deploy several HAProxy Enterprise AMIs.
  • Create an AWS target group.
  • Create an AWS Network Load Balancer in a single Availability Zone.
  • Test the setup.

Create a VPC Jump to heading

You can launch HAProxy Enterprise nodes in a Virtual Private Cloud (VPC), which is a virtual network similar to a traditional network.

  1. Open the Amazon VPC console, then click Launch VPC Wizard.

  2. Create a VPC with a Single Public Subnet with Public subnet’s Availability Zone set to the availability zone of your choice, for example, eu-west-3a. In other fields, keep the default values or select other values that better suit your needs.

  3. In the Virtual Private Cloud section on the left, select Your VPCs, then write down the ID of the new VPC. For example, vpc-0deecc96935b9ef73.

Deploy HAProxy Enterprise AMIs Jump to heading

You can launch HAProxy Enterprise nodes directly from the AWS Marketplace. Create two or more HAProxy Enterprise nodes on AWS, with the following characteristics:

Field Value
Software version The version of HAProxy Enterprise.
Region Region to which your Availability Zone belongs.
VPC Settings ID of the VPC you previously created. For example, vpc-0deecc96935b9ef73.
Subnet Settings Public subnet that belongs to the VPC.
Security Group Settings Create a new security group based on seller settings for the first HAProxy Enterprise node. Then select the same security group for other nodes.
Key Pair Settings Create a new EC2 key pair, or select an existing one. Select the same key pair for all HAProxy Enterprise nodes.

In other fields, keep the default values or select other values that better suit your needs.

Create an AWS target group Jump to heading

A target group routes requests to one or more registered targets, such as HAProxy Enterprise nodes, using the TCP protocol and the port number that you specify.

  1. Open the Amazon EC2 console.

  2. In the Load Balancing section on the left, select Target Groups.

  3. Create a target group with the following characteristics, then click Next.

    Field Value
    Target type Instances
    Protocol TCP
    VPC The virtual private cloud (VPC) you created previously. For example, vpc-0deecc96935b9ef73.

    In the other fields, keep the default values or select other values that better suit your needs.

  4. On the Register targets page, select the HAProxy Enterprise nodes you created previously. Then click Include as pending below.

    AWS available instances

  5. Click Create target group.

Create an AWS Network Load Balancer Jump to heading

The AWS Network Load Balancer selects a target HAProxy Enterprise node using a flow hash algorithm based on the source and destination IP addresses and ports, the protocol, and the TCP sequence number.

  1. In the Load Balancing section on the left, select Load Balancers.

  2. Create a Network Load Balancer with the following characteristics:

    Field Value
    Scheme Internet-facing.
    VPC The virtual private cloud (VPC) you created previously. For example, vpc-0deecc96935b9ef73.
    Mappings Select your Availability Zone and the corresponding subnet.
    Listeners Default (a listener that accepts TCP traffic on port 80).
    Default action Select the target group you created previously.

    In the other fields, keep the default values or select other values that better suit your needs.

  3. Click Create load balancer.

Test your setup Jump to heading

Once you have created a Network Load Balancer, wait a few minutes and check that the HAProxy Enterprise nodes in your target group have passed the initial health check. You can then test that the layer 4 AWS Network Load Balancer sends traffic to your layer 7 HAProxy Enterprise load balancers.

  1. In the Load Balancing section on the left, select Target Groups.

  2. Select the newly created target group, and check that your HAProxy Enterprise nodes are ready.

    Nodes ready

  3. In the Load Balancing section on the left, select Load Balancers.

  4. Select the newly created load balancer.

  5. Copy the DNS name of the load balancer and paste it into the address field of a web browser.

    For example, strawberry-f9f565c7eb5b3cd3.elb.eu-west-3.amazonaws.com.

    The browser displays the statistics page of your HAProxy Enterprise node.

    If you launched nodes with different versions of HAProxy Enterprise, paste the DNS name of the Network Load Balancer into the address field of another web browser or in a private browser window. The browser should display the statistics page of one of your other HAProxy Enterprise nodes, which runs another version of HAProxy Enterprise. Since each TCP connection is routed to a single target as long as the connection is active, you may have to try connecting several times.

You can now:

  • configure your backend server pool.
  • edit the frontend section of your HAProxy configuration file (each AWS Network Load Balancer in the Availability Zone has a static IP).
  • edit the backend section of your HAProxy configuration file.
  • copy the configuration file to all HAProxy Enterprise nodes in the AWS target group.

See also Jump to heading

If this page was useful, please, Leave the feedback.