High availability

Active/Standby clustering

On this page

In an active-standby cluster, one HAProxy Enterprise node receives traffic while a second node is put on standby. In case the active node fails, the standby takes over. With HAProxy Enterprise, you configure an active-standby cluster by installing the VRRP module.

The HAProxy Enterprise VRRP module assigns a virtual, static IP address to your load balancer node. If the node fails, the IP address will instantly transfer to a backup node, avoiding any break in service. Your router can continue to send traffic to the same IP address for the load balancer, never knowing that there was a failover. The module utilizes a stable version of Keepalived, which implements the Virtual Router Redundancy Protocol (VRRP).

In this guide, we will set up two load balancers: one active and the other on standby.

Active/Standby with VRRP

Installation Jump to heading

Follow the steps on your active and backup load balancers.

Active load balancer Jump to heading

On the active load balancer, follow these steps:

  1. Install the VRRP module using your system’s package manager:

    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp

  2. Edit the file /etc/hapee-extras/hapee-vrrp.cfg.

    • In the vrrp_instance section, change the interface value from eth0 to the name of a network interface that receives traffic on the server. This is the same interface that you the bind HAProxy Enterprise’s frontend to. For example, if traffic reaches the load balancer using interface enp0s8. Set this line to interface enp0s8.

    • Use the same interface name in the track_interface block.

    • Change the IP address listed in the virtual_ipaddress_excluded block to the virtual IP address you’d like to assign to this interface. HAProxy Enterprise will bind to this address to receive traffic. If needed, add more IP addresses here, each on its own line. The new address(es) should fall within the interface’s IP subnet, but should not be assigned to any server already.

    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
      interface enp0s8       # Change network interface name
      state MASTER
      virtual_router_id 51
      priority 101
      virtual_ipaddress_excluded {
        192.168.50.10        # New IP address
      }
      track_interface {
        enp0s8 weight -2     # Change network interface name
      }
      track_script {
        chk_sshd
        chk_lb
      }
    }
    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
      interface enp0s8       # Change network interface name
      state MASTER
      virtual_router_id 51
      priority 101
      virtual_ipaddress_excluded {
        192.168.50.10        # New IP address
      }
      track_interface {
        enp0s8 weight -2     # Change network interface name
      }
      track_script {
        chk_sshd
        chk_lb
      }
    }

  3. Enable and start the hapee-extras-vrrp service:

    nix
    sudo systemctl enable hapee-extras-vrrp
    sudo systemctl start hapee-extras-vrrp
    nix
    sudo systemctl enable hapee-extras-vrrp
    sudo systemctl start hapee-extras-vrrp

  4. Allow the server to bind to the virtual IP address.

    Edit the file /etc/sysctl.conf and add the net.ipv4_ip_nonlocal_bind=1 directive, which allows the server to accept connections for IP addresses that are not bound to any of its interfaces, enabling the use of a floating, virtual IP.

    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1
    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1

  5. Configure your bind line in the HAProxy Enterprise configuration to use the virtual IP address.

    haproxy
    frontend myfrontend
      mode http
      bind 192.168.50.10:80
      default_backend web_servers
    haproxy
    frontend myfrontend
      mode http
      bind 192.168.50.10:80
      default_backend web_servers

  6. Reboot the server.

Backup load balancer Jump to heading

On the backup load balancer, follow these steps:

  1. Install the VRRP module using your system’s package manager:

    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo apt-get install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo yum install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo zypper install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp
    nix
    sudo pkg install hapee-extras-vrrp

  2. Edit the VRRP configuration /etc/hapee-extras/hapee-vrrp.cfg. It must be almost exactly the same as the file on the active load balancer, except:

    • change the priority field in the vrrp_instance block to have a lower number than the active node; The load balancer with the highest priority is promoted to be the active node. For example, if the priority on the active node is 101, then set the backup node’s priority to 100.

    • change state to BACKUP.

    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
      interface enp0s8
      state BACKUP
      virtual_router_id 51
      priority 100              # A lower priority value than the primary
      virtual_ipaddress_excluded {
        192.168.50.10           # Same IP address as on primary
      }
      track_interface {
        enp0s8 weight -2
      }
      track_script {
        chk_sshd
        chk_lb
      }
    }
    hapee-vrrp.cfg
    text
    vrrp_instance vrrp_1 {
      interface enp0s8
      state BACKUP
      virtual_router_id 51
      priority 100              # A lower priority value than the primary
      virtual_ipaddress_excluded {
        192.168.50.10           # Same IP address as on primary
      }
      track_interface {
        enp0s8 weight -2
      }
      track_script {
        chk_sshd
        chk_lb
      }
    }

  3. Enable and start the hapee-extras-vrrp service:

    nix
    sudo systemctl enable hapee-extras-vrrp
    sudo systemctl start hapee-extras-vrrp
    nix
    sudo systemctl enable hapee-extras-vrrp
    sudo systemctl start hapee-extras-vrrp

  4. Allow the server to bind to the virtual IP address.

    Edit the file /etc/sysctl.conf and add the net.ipv4_ip_nonlocal_bind=1 directive, which allows the server to accept connections for IP addresses that are not bound to any of its interfaces, enabling the use of a floating, virtual IP.

    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1
    sysctl.conf
    ini
    net.ipv4.ip_nonlocal_bind=1

  5. Configure your bind line in the HAProxy Enterprise configuration to use the virtual IP address.

    haproxy
    frontend myfrontend
      mode http
      bind 192.168.50.10:80
      default_backend web_servers
    haproxy
    frontend myfrontend
      mode http
      bind 192.168.50.10:80
      default_backend web_servers

  6. Reboot the server.

Verify the setup Jump to heading

To check that the VRRP service is running, use systemctl status:

nix
sudo systemctl status hapee-extras-vrrp --no-pager
nix
sudo systemctl status hapee-extras-vrrp --no-pager
output
text
● hapee-extras-vrrp.service - HAPEE VRRP : VRRP daemon (Keepalived)
     Loaded: loaded (/lib/systemd/system/hapee-extras-vrrp.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-10-18 18:01:59 UTC; 10min ago
   Main PID: 1191 (hapee-vrrp)
      Tasks: 2 (limit: 4555)
     Memory: 1.5M
        CPU: 4.632s
     CGroup: /system.slice/hapee-extras-vrrp.service
             ├─1191 /opt/hapee-extras/sbin/hapee-vrrp -D -f /etc/hapee-extras/hapee-vrrp.cfg -p /var/run/hapee-extras/hapee-vrrp.pid
             └─1192 /opt/hapee-extras/sbin/hapee-vrrp -D -f /etc/hapee-extras/hapee-vrrp.cfg -p /var/run/hapee-extras/hapee-vrrp.pid
Oct 18 18:02:01 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) forcing a new MASTER election
Oct 18 18:02:02 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Transition to MASTER STATE
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Entering MASTER STATE
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) setting protocol E-VIPs.
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Sending gratuitous ARPs on eth0 for 172.1.1.4
output
text
● hapee-extras-vrrp.service - HAPEE VRRP : VRRP daemon (Keepalived)
     Loaded: loaded (/lib/systemd/system/hapee-extras-vrrp.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-10-18 18:01:59 UTC; 10min ago
   Main PID: 1191 (hapee-vrrp)
      Tasks: 2 (limit: 4555)
     Memory: 1.5M
        CPU: 4.632s
     CGroup: /system.slice/hapee-extras-vrrp.service
             ├─1191 /opt/hapee-extras/sbin/hapee-vrrp -D -f /etc/hapee-extras/hapee-vrrp.cfg -p /var/run/hapee-extras/hapee-vrrp.pid
             └─1192 /opt/hapee-extras/sbin/hapee-vrrp -D -f /etc/hapee-extras/hapee-vrrp.cfg -p /var/run/hapee-extras/hapee-vrrp.pid
Oct 18 18:02:01 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) forcing a new MASTER election
Oct 18 18:02:02 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Transition to MASTER STATE
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Entering MASTER STATE
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) setting protocol E-VIPs.
Oct 18 18:02:03 hapee01 Keepalived_vrrp[1192]: VRRP_Instance(vrrp_1) Sending gratuitous ARPs on eth0 for 172.1.1.4

Manual failover Jump to heading

To activate the standby load balancer while simultaneously putting the active load balancer into standby mode:

  1. Edit the VRRP configuration /etc/hapee-extras/hapee-vrrp.cfg on the active load balancer.

  2. Lower the priority value to less than the standby load balancer’s priority. This will put the active load balancer into the standby state and simultaneously activate the standby load balancer. For example, change it from 101 to 90.

  3. Restart the hapee-extras-vrrp service.

    Reverse these actions to restore the original state.

Failover triggers Jump to heading

The following events can trigger a failover:

  • The active node lowers its weight below one of the backup nodes due to a failed health check.
  • A backup node is reconfigured with a weight larger than the current active node.
  • The active node stops emitting its heartbeat packet to the cluster.

VRRP health check scripts Jump to heading

In the VRRP configuration, the vrrp_script blocks define health checks that can trigger a failover. By default, our configuration checks whether the SSH daemon is running and whether HAProxy Enterprise is running. You may add more checks.

Consider this block, which checks the status of the SSH daemon every five seconds. If the service is not available, the VRRP instance’s weight (i.e. its priority) is reduced by 4 points:

hapee-vrrp.cfg
text
vrrp_script chk_sshd {
  script "pkill -0 sshd"  # pkill -0 is cheaper than pidof
  interval 5              # check every 5 seconds
  weight -4               # remove 4 points of prio if missing
  fall 2                  # check twice before setting down
  rise 1                  # check once before setting up
}
hapee-vrrp.cfg
text
vrrp_script chk_sshd {
  script "pkill -0 sshd"  # pkill -0 is cheaper than pidof
  interval 5              # check every 5 seconds
  weight -4               # remove 4 points of prio if missing
  fall 2                  # check twice before setting down
  rise 1                  # check once before setting up
}

The HAProxy Enterprise health check is similar, except that if the hapee-lb process is running, it adds 6 points. The reason is, in case one load balancer has SSH working but not hapee-lb, while the other has hapee-lb but not SSH, the one with hapee-lb will become the active node, even though it is not manageable remotely.

hapee-vrrp.cfg
text
vrrp_script chk_lb {
  script "pkill -0 hapee-lb"  # pkill -0 is cheaper than pidof
  interval 1                  # check every second
  weight 6                    # add 6 points of prio if present
  fall 2                      # check twice before setting down
  rise 1                      # check once before setting up
}
hapee-vrrp.cfg
text
vrrp_script chk_lb {
  script "pkill -0 hapee-lb"  # pkill -0 is cheaper than pidof
  interval 1                  # check every second
  weight 6                    # add 6 points of prio if present
  fall 2                      # check twice before setting down
  rise 1                      # check once before setting up
}

The following fields define a vrrp_script block:

Name Argument type Description
script string Command to run; Running pkill -0 checks if any process with the given name is running. If it returns 0, the check passes; If it returns 1, the check fails.
interval integer Interval (in seconds) between two checks.
weight integer Points to add or remove to instance weight.
fall integer Number of consecutive negative checks before considered as down.
rise integer Number of consecutive positive checks before considered as up.

VRRP instance reference Jump to heading

The following fields define a vrrp_instance block:

Name Argument type Description
vrrp_instance string Describes a new VRRP instance.
interface string Interface managed by the VRRP module.
state string State at startup until VRRP priority negotiation completes. Can be either MASTER or BACKUP.
virtual_router_id integer VRRP instance identifier which must be common to all nodes of the same cluster. Its value ranges from 0 to 255.
priority integer Weight of local node in this VRRP instance. NOTE This will not work on cloud provider networks that disable IP multicast.
virtual_ipaddress string List of virtual IP addresses to add or remove when the state changes to active or backup. All VRRP nodes in a cluster must own the same IP. NOTE This will not work on cloud provider networks that disable IP multicast.
virtual_ipaddress_excluded string Same as virtual_ipaddress; but IPs are not announced in the VRRP heartbeat packet, which reduces bandwidth usage when sending packets.
track_interface string Tracks interface status and updates priority accordingly.
track_script string Runs the health check scripts and updates priority accordingly.

Do you have any suggestions on how we can improve the content of this page?

Previous page Active/Active clustering Next page Logs
© 2024 HAProxy Technologies, LLC. All Rights Reserved
Manage Cookie Preferences