Route health injection

This page applies to:

HAProxy Enterprise 3.2r1 and newer

The Route Health Injection (RHI) module monitors your load balancer’s connectivity to backend servers and has the ability to remove the entire load balancer from duty if the load balancer suddenly can’t reach those servers. The idea is that if a load balancer can’t reach the servers, and you’re running an active/active load balancer pair, then you can deactive the problematic load balancer and route all traffic to the other, healthy load balancer.

The RHI module is meant to work with the IP routing protocol BGP, configured for Equal-cost multi-path (ECMP) routing. ECMP enables the network to route traffic to a destination over multiple paths, allowing you to relay IP packets to both of your load balancers in parallel. The ability to detect problems and route traffic away from unhealthy load balancers is important for making ECMP resilient.

Concepts Jump to heading

This section describes the concepts behind route health injection combined with ECMP.

How ECMP routing works Jump to heading

Your router attempts to send packets to their destination using the most efficient network path. When two network paths have identical costs, meaning they are equally good, then the router can load balance traffic across both paths given that it supports ECMP. With ECMP, you can configure the router to see both of your load balancers as being different, but equal, routes to the destination IP address. So the router can send traffic to both load balancers in parallel, achieving high availability.

A router on the 192.168.0.0/24 network sees:

Destination address	Route via
`192.168.1.10`	* Load balancer 1 at `192.168.0.101` or * Load balancer 2 at `192.168.0.102`

The RHI module shares routes with the router using the BGP protocol.

While to the router it looks as through the load balancers are simply the next hop towards the destination, actually the load balancers are the final hop. They own the destination IP address. Having both load balancers bound to the same IP address could cause conflicts on the network, though. Two ways to solve this problem that we’ll cover are to:

Add a second address to the load balancer’s loopback network interface and disable ARP so that this address isn’t advertised on the network. That way, only the load balancer sees this address and essentially sends the traffic to itself.
Intercept traffic destined for a non-locally bound address by configuring transparent proxying.

How Route Health Injection works Jump to heading

The RHI module uses the BGP protocol to announce to neighboring routers on the network that this load balancer is a route for the specified IP addresses. If either a frontend or a backend is down, then the RHI module stops announcing this load balancer as a route on the network, diverting the flow of traffic to the other load balancer in the active/active cluster. You can configure ECMP on your router to load balance traffic to both load balancers via the advertised routes.

Prerequisites Jump to heading

Ensure that you’ve met the following prerequisites:

Your router has enabled ECMP. Consult your router’s documentation for details.

Install and configure RHI Jump to heading

In this section, you’ll learn how to set up route health injection.

Install the RHI module Jump to heading

Install the RHI module according to your platform.

Install the RHI module

nix
sudo apt-get install hapee-3.2r1-lb-rhi-bgp

nix
sudo apt-get install hapee-3.2r1-lb-rhi-bgp

nix
sudo yum install hapee-3.2r1-lb-rhi-bgp

nix
sudo yum install hapee-3.2r1-lb-rhi-bgp

nix
sudo zypper install hapee-3.2r1-lb-rhi-bgp

nix
sudo zypper install hapee-3.2r1-lb-rhi-bgp

nix
sudo pkg install hapee-3.2r1-lb-rhi-bgp

nix
sudo pkg install hapee-3.2r1-lb-rhi-bgp

Configure route health injection Jump to heading

In the global section of your configuration, load the module via the module-load directive.

haproxy
global
  ...
  module-load hapee-lb-rhi-bgp.so

haproxy
global
  ...
  module-load hapee-lb-rhi-bgp.so

Add an rhi-bgp section to your configuration. It defines the BGP settings.

In this example, HAProxy Enterprise will announce to its neighbor router at 192.168.0.1 that it is a route for reaching the IPv4 address 192.168.1.10. It continues to announce this route unless the backend named webservers has no servers up.

Example - Load balancer 1
haproxy
rhi-bgp dc1
  hold-time 30
  timeout connect 1s
  timeout open 5s
  timeout reconnect 1s
  timeout keepalive 10s
  timeout min-update-interval 3s
  timeout graceful-restart 5s
  log global
  local-id 192.168.0.101
  local-as 65001
  neighbor 192.168.0.1:179 as 65001
  next-hop-ipv4 192.168.0.101
  acl backend_is_up nbsrv(webservers) gt 0
  rhi-announce addrs 192.168.1.10/32 if backend_is_up

Example - Load balancer 1
haproxy
rhi-bgp dc1
  hold-time 30
  timeout connect 1s
  timeout open 5s
  timeout reconnect 1s
  timeout keepalive 10s
  timeout min-update-interval 3s
  timeout graceful-restart 5s
  log global
  local-id 192.168.0.101
  local-as 65001
  neighbor 192.168.0.1:179 as 65001
  next-hop-ipv4 192.168.0.101
  acl backend_is_up nbsrv(webservers) gt 0
  rhi-announce addrs 192.168.1.10/32 if backend_is_up

Add an rhi-bgp section to your other load balancer to announce another route for reaching the same IP address.

Example - Load balancer 2
haproxy
rhi-bgp dc1
  hold-time 30
  timeout connect 1s
  timeout open 5s
  timeout reconnect 1s
  timeout keepalive 10s
  timeout min-update-interval 3s
  timeout graceful-restart 5s
  log global
  local-id 192.168.0.102
  local-as 65001
  neighbor 192.168.0.1:179 as 65001
  next-hop-ipv4 192.168.0.102
  acl backend_is_up nbsrv(webservers) gt 0
  rhi-announce addrs 192.168.1.10/32 if backend_is_up

Example - Load balancer 2
haproxy
rhi-bgp dc1
  hold-time 30
  timeout connect 1s
  timeout open 5s
  timeout reconnect 1s
  timeout keepalive 10s
  timeout min-update-interval 3s
  timeout graceful-restart 5s
  log global
  local-id 192.168.0.102
  local-as 65001
  neighbor 192.168.0.1:179 as 65001
  next-hop-ipv4 192.168.0.102
  acl backend_is_up nbsrv(webservers) gt 0
  rhi-announce addrs 192.168.1.10/32 if backend_is_up

Intercept traffic destined for the IP Jump to heading

Remember that both of your load balancers must be able to receive packets at the same IP address defined in the route shared with your router, for example 192.168.1.10/32, but without causing a conflict on the network. Here are two ways to accomplish that:

Add a second address to the load balancer's loopback network interface

The IP must be handled on each server’s loopback interface to accept connections, but can’t be advertised on the network or it will be identified as an IP conflict by some network components. To avoid IP address conflicts, disable ARP for IP addresses managed by the loopback interface.

Perform these steps on both load balancers:

Edit the HAProxy Enterprise configuration file, /etc/hapee-3.2/hapee-lb.cfg:
```
hapee-lb.cfg
haproxy
frontend www
  bind 192.168.1.10:80 name http
  bind 192.168.1.10:443 name https ssl crt site.pem
```
```
hapee-lb.cfg
haproxy
frontend www
  bind 192.168.1.10:80 name http
  bind 192.168.1.10:443 name https ssl crt site.pem
```
- In the frontend section, define one or more bind lines that listen at the IP address you created a route for, such as 192.168.1.10. This address shouldn’t be assigned to any network interfaces.
- Each load balancer should be assigned the same IP address.

Save the changes and then reload the service.

Reload the service

nix
sudo systemctl reload hapee-3.2r1-lb

nix
sudo systemctl reload hapee-3.2r1-lb

Manage the IP address through a loopback interface.

Edit the file /etc/network/interfaces. Add a new iface section for the lo interface and add the address under it:

interfaces
text
# The loopback network interface
auto lo
iface lo inet loopback
iface lo inet static
  address 192.168.1.10/32

interfaces
text
# The loopback network interface
auto lo
iface lo inet loopback
iface lo inet static
  address 192.168.1.10/32

Edit the netplan YAML configuration file located in /etc/netplan. The configuration file is probably the one having the lowest number and has a name like 00-installer-config.yaml or 01-netcfg.yaml.

Edit the netplan YAML file, adding an lo section under the ethernets level:

01-netcfg.yaml
yaml
network:
  ethernets:
    lo:
      dhcp4: false
      addresses:
      - "192.168.1.10/32"

01-netcfg.yaml
yaml
network:
  ethernets:
    lo:
      dhcp4: false
      addresses:
      - "192.168.1.10/32"

Then use sudo netplan try and sudo netplan apply before rebooting to make sure the configuration is valid. Ignore warnings about Open vSwitch.

To persist the IP address on RHEL 9.2 or newer, use NetworkManager. Previous versions didn’t support managing the loopback interface with NetworkManager.

nix
sudo nmcli connection modify lo +ipv4.addresses 192.168.1.10/32
sudo nmcli con up 'lo'

nix
sudo nmcli connection modify lo +ipv4.addresses 192.168.1.10/32
sudo nmcli con up 'lo'

To persist the IP address on RHEL systems older than 9.2, create a new service for loading it at boot:

nix
sudo touch /etc/systemd/system/01-static-ip.service
sudo vi /etc/systemd/system/01-static-ip.service

nix
sudo touch /etc/systemd/system/01-static-ip.service
sudo vi /etc/systemd/system/01-static-ip.service

Add the following lines to the service file:

01-static-ip.service
text
[Unit]
Description=Add static IP to loopback
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the address
ExecStart=-/usr/sbin/ip address add 192.168.1.10/32 dev lo
[Install]
WantedBy=multi-user.target

01-static-ip.service
text
[Unit]
Description=Add static IP to loopback
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the address
ExecStart=-/usr/sbin/ip address add 192.168.1.10/32 dev lo
[Install]
WantedBy=multi-user.target

Set the service to start on boot:

nix
sudo systemctl enable 01-static-ip.service

nix
sudo systemctl enable 01-static-ip.service

Add the following lines to /etc/sysctl.conf.

text
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

text
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

Restart the HAProxy Enterprise server.

Configure transparent proxying

Perform these steps on both load balancers to enable transparent proxying:

Edit the HAProxy Enterprise configuration file, /etc/hapee-3.2/hapee-lb.cfg:
```
hapee-lb.cfg
haproxy
frontend www
  bind 192.168.1.10:80 name http transparent
  bind 192.168.1.10:443 name https ssl crt site.pem transparent
```
```
hapee-lb.cfg
haproxy
frontend www
  bind 192.168.1.10:80 name http transparent
  bind 192.168.1.10:443 name https ssl crt site.pem transparent
```
- In the frontend section, define one or more bind lines that listen at the IP address you created a route for, such as 192.168.1.10. This address shouldn’t be assigned to any network interfaces.
- Because the IP address isn’t configured on the network interface, add the transparent argument. This indicates that the IP address should be bound even though it doesn’t belong to the local machine. Packets targeting this address will be intercepted as if the address were locally configured. This feature uses the Linux kernel’s TPROXY feature.
- Each load balancer should be assigned the same IP address.

Save the changes and then reload the service.

Reload the service

nix
sudo systemctl reload hapee-3.2r1-lb

nix
sudo systemctl reload hapee-3.2r1-lb

Add firewall rules that intercept packets that have a destination IP address matching a listening socket, which in this case is our transparent proxied IP address. Also add policy-based routing rules to deliver the traffic locally.

Create firewall and routing rules:

nix
sudo iptables -t mangle -N DIVERT
sudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
sudo iptables -t mangle -A PREROUTING -p udp -m socket -j DIVERT
sudo iptables -t mangle -A DIVERT -j MARK --set-mark 1
sudo iptables -t mangle -A DIVERT -j ACCEPT
sudo ip rule add fwmark 1 lookup 100
sudo ip route add local 0.0.0.0/0 dev lo table 100

nix
sudo iptables -t mangle -N DIVERT
sudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
sudo iptables -t mangle -A PREROUTING -p udp -m socket -j DIVERT
sudo iptables -t mangle -A DIVERT -j MARK --set-mark 1
sudo iptables -t mangle -A DIVERT -j ACCEPT
sudo ip rule add fwmark 1 lookup 100
sudo ip route add local 0.0.0.0/0 dev lo table 100

To make the iptables changes persistent after reboot, use the iptables-save command. It saves the changes and configures the system to restore them at reboot.

nix
sudo apt install iptables-persistent
sudo su -c 'iptables-save > /etc/iptables/rules.v4'

nix
sudo apt install iptables-persistent
sudo su -c 'iptables-save > /etc/iptables/rules.v4'

To verify the IP tables rules, use the iptables command. Note that the output may show tcp or the number 6, and udp or the number 17:

nix
sudo iptables -L -v -n -t mangle

nix
sudo iptables -L -v -n -t mangle

output
text
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
1941  335K DIVERT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            socket
   0     0 DIVERT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            socket
...
Chain DIVERT (1 references)
pkts bytes target     prot opt in     out     source               destination
1941  335K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK set 0x1
1941  335K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

output
text
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
1941  335K DIVERT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            socket
   0     0 DIVERT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            socket
...
Chain DIVERT (1 references)
pkts bytes target     prot opt in     out     source               destination
1941  335K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK set 0x1
1941  335K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

To make the policy route (ip rule) and route table (ip route) changes persist after reboot, your next step depends on whether or not your system uses netplan. Typically, Ubuntu uses it, but Debian doesn’t.

If your system uses netplan, persist the policy route (ip rule) and route table (ip route) changes in the netplan YAML configuration file located in /etc/netplan. The configuration file is probably the one having the lowest number and has a name like 00-installer-config.yaml or 01-netcfg.yaml.

Edit the netplan YAML file, adding an lo section under the ethernets level:

01-netcfg.yaml
yaml
network:
  ethernets:
    lo:
      routing-policy:
      - to: 0.0.0.0/0
        mark: 1
        table: 100
      routes:
      - to: 0.0.0.0/0
        type: local
        table: 100

01-netcfg.yaml
yaml
network:
  ethernets:
    lo:
      routing-policy:
      - to: 0.0.0.0/0
        mark: 1
        table: 100
      routes:
      - to: 0.0.0.0/0
        type: local
        table: 100

Then, use sudo netplan try and sudo netplan apply before rebooting to ensure the configuration is valid. Ignore warnings about Open vSwitch.

If your system doesn’t use netplan, persist the changes by creating a new service for loading them at boot. Create the systemd service file:

nix
sudo touch /etc/systemd/system/01-static-route.service
sudo vi /etc/systemd/system/01-static-route.service

nix
sudo touch /etc/systemd/system/01-static-route.service
sudo vi /etc/systemd/system/01-static-route.service

Add the following lines to the service file:

01-static-route.service
text
[Unit]
Description=Add route table 100
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the route table and rule
ExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100
ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100
[Install]
WantedBy=multi-user.target

01-static-route.service
text
[Unit]
Description=Add route table 100
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the route table and rule
ExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100
ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100
[Install]
WantedBy=multi-user.target

Set the service to start on boot:

nix
sudo systemctl enable 01-static-route.service

nix
sudo systemctl enable 01-static-route.service

Restart the system. The saved settings will be restored after the restart.

Create firewall rules:

nix
sudo firewall-cmd --permanent --direct --add-chain ipv4 mangle DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p tcp -m socket -j DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p udp -m socket -j DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 0 -j MARK --set-mark 1
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 1 -j ACCEPT

nix
sudo firewall-cmd --permanent --direct --add-chain ipv4 mangle DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p tcp -m socket -j DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p udp -m socket -j DIVERT
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 0 -j MARK --set-mark 1
sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 1 -j ACCEPT

Reload the firewall:

nix
sudo firewall-cmd --reload

nix
sudo firewall-cmd --reload

To persist the IP routing rules, create a new service for loading them at boot:

nix
sudo touch /etc/systemd/system/01-static-route.service
sudo vi /etc/systemd/system/01-static-route.service

nix
sudo touch /etc/systemd/system/01-static-route.service
sudo vi /etc/systemd/system/01-static-route.service

Add the following lines to the service file:

01-static-route.service
text
[Unit]
Description=Add route table 100
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the route table and rule
ExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100
ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100
[Install]
WantedBy=multi-user.target

01-static-route.service
text
[Unit]
Description=Add route table 100
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
# create the route table and rule
ExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100
ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100
[Install]
WantedBy=multi-user.target

Set the service to start on boot:

nix
sudo systemctl enable 01-static-route.service

nix
sudo systemctl enable 01-static-route.service

Restart the system.

To verify the rule table, use the ip command:

nix
sudo ip rule ls

nix
sudo ip rule ls

output
text
...
32765:  from all fwmark 0x1 lookup 100
...

output
text
...
32765:  from all fwmark 0x1 lookup 100
...

To verify the route table, use the ip command:

nix
sudo ip route ls table 100

nix
sudo ip route ls table 100

output
text
local default dev lo scope host

output
text
local default dev lo scope host

At this point, you have completed the setup of the Route Health Injection module.

Reference Jump to heading

This section describes the module’s configuration settings and commands.

rhi-bgp section Jump to heading

An rhi-bgp section represents a BGP router that announces or withdraws routes, but ignores all updates from neighboring routers. You can define multiple rhi-bgp sections. Each contains rules for when to add routes.

ruleset: a group of rules. A file that you specify with the rhi-config directive or rhi-legacy-config directive is a named ruleset. All inline rhi-announce rules in an rhi-bgp section are placed into an anonymous ruleset.
rule: a condition for when to add one or more routes. The state of a rule depends upon the condition’s evaluated result. By default, all of a rule’s routes inherit the owning rule’s state.
route: a path on the network to reach a destination IP address. The RHI-BGP module announces or withdraws routes depending on the rule’s state or the route’s own state if forced via the HAProxy Runtime API set rhi-bgp command.

The rhi-bgp section supports the following directives.

Directive	Description
`acl <aclname> <criterion> [flags] [operator] <value> ...`	Defines an access control list. See the configuration reference for acl.
`hold-time <time>`	Defines the announced BGP hold-time value, in seconds. This value is used in the `OPEN` messages sent to neighbors. The RHI module and the neighbor each announce a number and the smaller of the two is used. The calculated hold-time value indicates the maximum number of seconds that can elapse between the receipt of successive `KEEPALIVE` and/or `UPDATE` messages from the neighbor. `<time>`: the announced BGP hold-time value in seconds. It must be either zero or any value between 3 and 65535.
`local-as <as>`	Defines the Autonomous System number that this RHI-BGP instance uses. `<as>`: a 4-byte unsigned integer.
`local-id <id>`	Defines the BGP Identifier that this RHI-BGP instance uses. `<id>`: the BGP identifier. It can be an IPv4 address or a 4-byte unsigned integer.
`log global \| log <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>] [profile <prof>] <facility> [<level> [<minlevel>]]`	Enables per-instance logging of events and traffic. See the configuration reference for log.
`neighbor <addr> [ as <uint32> ] [ source <addr>[:<port>] ] [ namespace <name> ] [ tcp-md5sig <password> ]`	Define a neighboring router this instance will peer with and share routes. `addr`: the neighbor router’s IP address. `as <uint32>` : the neighbor router’s Autonomous System number. A 4-byte AS number is supported. If not set, the neighbor’s AS number must be the same as the AS number of the RHI-BGP instance (equal to the `local-as` value). `source <addr>[:<port>]`: an IP address followed by an optional port. This sets the source address to use when connecting to the neighbor. `namespace <name>`: the network namespace name. On Linux, it’s possible to specify which network namespace a socket will belong to. This argument makes it possible to explicitly bind a neighbor to a namespace different from the default one. Please refer to your operating system’s documentation to find more details about network namespaces. `tcp-md5sig <password>`: a password string used to sign every TCP segments with a 16-byte MD5 digest. This argument is available only on Linux. This protects the BGP session against spoofing.
`next-hop-ipv4 <ipv4>`	Specifies which IPv4 address to use to announce IPv4 routes. If this directive isn’t configured, no IPv4 routes will be announced. By default, it’s unset.
`next-hop-ipv6 <ipv6>`	Specifies which IPv6 address to use to announce IPv6 routes. If this directive isn’t configured, no IPv6 routes will be announced. By default, it’s unset.
`rhi-announce auto [ { if \| unless } <condition> ]` `rhi-announce label <name> [ { if \| unless } <condition> ]` `rhi-announce <network> [<network>...] [ { if \| unless } <condition> ]`	Defines one or more routes to announce or withdraw from all neighbors, depending on the condition’s evaluation result. There are three ways to configure the routes this RHI-BGP instance will monitor: `auto`: monitors all listening addresses (`bind` directives) configured accross all frontends. `label <name>`: monitors all listening addresses (`bind` directives) configured with the given `label` argument accross all frontends. you can monitor a list of space-separated network ranges following the format `{<ipv4>,<ipv6>}[/<mask>]`.
`rhi-config <file>`	Defines a file containing a list of routes to announce or withdraw from all neighbors. `<file>`: a file containing a list of routes to monitor. Each line of this file represents a rule with the following format: `<network> [<network>...] [ { if \| unless } <condition> ]`. Empty lines or lines beginning with a `#` are ignored. See `rhi-announce` for details.
`rhi-legacy-config <file>`	Defines a file containing a list of routes to announce or withdraw from all neighbors. This file uses the same format as the legacy RHI module. a file containing a list of routes to monitor. Each line of this file represents a rule with the following format: `<network>[,<network>,[...]] = <agg>(<b:\|f:><name>[,<b:\|f:><name>,[...]])`. Empty lines or lines beginning with a `#` are ignored.
`timeout connect <timeout>`	Sets the time to wait to connect. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. It defaults to 500 milliseconds.
`timeout graceful-restart <timeout>`	Sets the restart time announced in the BGP Graceful Restart capability and specifies how long the neighbors would wait for the BGP session to re-establish after a restart before deleting stale routes. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. It defaults to 10 seconds.
`timeout keepalive <timeout>`	Sets the delay between sending of two `KEEPALIVE` messages. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. The effective value depends on the negotiated hold time, as it’s scaled to maintain proportion between the keepalive time and the hold time. If hold time is zero, the keepalive timeout is disabled and no `KEEPALIVE` is sent. If the keepalive timeout is lower than half of the negotiated hold time, it’s used. In all other cases, the default value is used. It defaults to one-third of the negotiated hold time.
`timeout min-update-interval <timeout>`	Sets the minimum time between sending two `UPDATE` messages. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. It defaults to 5 seconds.
`timeout open <timeout>`	Sets the delay between the sending of the `OPEN` message and the receipt of the neighbor’s `OPEN` message. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. It defaults to 1 second.
`timeout reconnect <timeout>`	Sets the time to wait before retrying a failed attempt to connect. `<timeout>`: a timeout value specified in milliseconds by default, but can be in any other unit if the number is suffixed by the unit. See the configuration reference for Time format. It defaults to 1 second.

rhi-config file Jump to heading

The file that you pass via the rhi-config directive supports this syntax:

Syntax
text
<network> [<network>...] [ { if | unless } <condition> ]

Syntax
text
<network> [<network>...] [ { if | unless } <condition> ]

my-rhi-config.cfg
text
# Inject the 10.200.200.200/32 address into the route daemon if
# all the backends "be_static" and "be_app" are up.
addrs 10.200.200.200/32 if { nbsrv(be_state) gt 0 } { nbsrv(be_app) gt 0 }

my-rhi-config.cfg
text
# Inject the 10.200.200.200/32 address into the route daemon if
# all the backends "be_static" and "be_app" are up.
addrs 10.200.200.200/32 if { nbsrv(be_state) gt 0 } { nbsrv(be_app) gt 0 }

rhi-legacy-config file Jump to heading

The file that you pass via the rhi-legacy-config directive supports this syntax:

Syntax
text
<network>[,<network>,[...]] = <agg>(<b:|f:><name>[,<b:|f:><name>,[...]])

Syntax
text
<network>[,<network>,[...]] = <agg>(<b:|f:><name>[,<b:|f:><name>,[...]])

my-legacy-rhi-confg.cfg
text
# Inject the 10.200.200.200/32 address into the route daemon if
# all the backends "be_static" and "be_app" are up.
addrs 10.200.200.200/32 = all(b:be_static,b:be_app)

my-legacy-rhi-confg.cfg
text
# Inject the 10.200.200.200/32 address into the route daemon if
# all the backends "be_static" and "be_app" are up.
addrs 10.200.200.200/32 = all(b:be_static,b:be_app)

Runtime API Jump to heading

In this section, we describe the HAProxy Runtime API commands that the RHI module makes available.

add rhi-bgp Jump to heading

Adds a new rule to the <rs> ruleset of the rhi-bgp section <name>. Use the rhi-config file rules format.

Syntax
text
add rhi-bgp <name> ruleset <rs> rule addrs <addr/mask> [<addr>/<mask> ...] [ { if | unless } <condition> ]

Syntax
text
add rhi-bgp <name> ruleset <rs> rule addrs <addr/mask> [<addr>/<mask> ...] [ { if | unless } <condition> ]

Example:

Add a rule

nix
echo -e "add rhi-bgp dc1 ruleset /etc/hapee-3.2/my-rhi-config.cfg rule addrs 192.168.1.11/32 if { nbsrv(webservers) gt 0 }" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "add rhi-bgp dc1 ruleset /etc/hapee-3.2/my-rhi-config.cfg rule addrs 192.168.1.11/32 if { nbsrv(webservers) gt 0 }" | sudo socat stdio tcp4-connect:127.0.0.1:9999

del rhi-bgp Jump to heading

Deletes the given rule or route from the rhi-bgp section <name>. The rule is removed once withdrawn from all neighbor routers. You can delete only rules and routes from an rhi-config file ruleset.

Syntax
text
del rhi-bgp <name> [ rule <id> | route <id> ]

Syntax
text
del rhi-bgp <name> [ rule <id> | route <id> ]

Example:

Delete the rule that has the ID of 1.

nix
echo -e "del rhi-bgp dc1 rule 1" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "del rhi-bgp dc1 rule 1" | sudo socat stdio tcp4-connect:127.0.0.1:9999

set rhi-bgp Jump to heading

Change the state of the rule or the route with the identifier <id> in the rhi-bgp instance <name>. In a given rhi-bgp section, all rules and routes have a unique identifier. Set the state to UP, DOWN, or AUTO. A rule or a route can have one of the following states:

State	Description
`DEL`	Deleted. The rule or route was remove via the HAProxy Runtime API. For a route, it means that it will be withdrawn and then removed. For a rule, it means that all of its routes will be withdrawn and then removed.
`DOWN`	Down. The rule or route was evaluated as down. For a route, it means it will be withdrawn. For a rule, it means that all of its routes will be withdrawn.
`F_DOWN`	Forced down. The rule or route’s state was set to down via the HAProxy Runtime API. For a route, it means that it will be withdrawn. For a rule, it means that all of its routes will be withdrawn.
`F_UP`	Forced up. The rule or route’s state was set to up via the HAProxy Runtime API. For a route, it means that it will be announced. For a rule, it means that all of its routes will be announced.
`UNK`	Unknown. The rule or route’s state hasn’t been evaluated yet.
`UP`	Up. The rule or route was evaluated as up. For a route, it means that it will be announced. For a rule, it means that all of its routes will be announced.

Syntax
text
set rhi-bgp <name> [ route <id> | rule <id> ] { UP | DOWN | AUTO }

Syntax
text
set rhi-bgp <name> [ route <id> | rule <id> ] { UP | DOWN | AUTO }

Example:

Change the state to down for the rule with the ID of 0. This command returns no output.

nix
echo -e "set rhi-bgp dc1 rule 0 DOWN" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "set rhi-bgp dc1 rule 0 DOWN" | sudo socat stdio tcp4-connect:127.0.0.1:9999

show rhi-bgp Jump to heading

Returns information about existing rhi-bgp sections. Without arguments, it lists all rhi-bgp sections.

Syntax
text
show rhi-bgp [ name [ info | ruleset [rs] | route [id] | rules [id] | stats ] ]

Syntax
text
show rhi-bgp [ name [ info | ruleset [rs] | route [id] | rules [id] | stats ] ]

Examples:

List all rhi-bgp sections.

nix
echo -e "show rhi-bgp" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
* RHI-BGP section 'dc1' (id=3232249871, as=65001)

output
text
* RHI-BGP section 'dc1' (id=3232249871, as=65001)

Show the details of the rhi-bgp section named dc1.

nix
echo -e "show rhi-bgp dc1" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp dc1" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
RHI-BGP section 'dc1'
  local-id: 3232249871
  local-as: 65001
  hold-time: 30 seconds
  supported capabilities: ipv4-unicast graceful-restart ASN4
  next-hop-v4: 192.168.0.101
  next-hop-v6:
  neighbor <0>: 192.168.0.1 65001
      remote-id: 3232249873
      remote-as: 65001
      hold-time: 30 seconds
      remote capabilities: ipv4-unicast ASN4

output
text
RHI-BGP section 'dc1'
  local-id: 3232249871
  local-as: 65001
  hold-time: 30 seconds
  supported capabilities: ipv4-unicast graceful-restart ASN4
  next-hop-v4: 192.168.0.101
  next-hop-v6:
  neighbor <0>: 192.168.0.1 65001
      remote-id: 3232249873
      remote-as: 65001
      hold-time: 30 seconds
      remote capabilities: ipv4-unicast ASN4

List rules.

nix
echo -e "show rhi-bgp dc1 rule" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp dc1 rule" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
#id  [state] (ruleset)                          description
0    [   UP] (/etc/hapee-3.2/my-rhi-config.cfg) loaded from /etc/hapee-3.2/my-rhi-config.cfg line 1 - type=ACLS/ADDRS - last-change=1h40m refcount=3
1    [   UP] (/etc/hapee-3.2/my-rhi-config.cfg) loaded from /etc/hapee-3.2/my-rhi-config.cfg line 0 - type=ACLS/ADDRS - last-change=5m59s refcount=3

output
text
#id  [state] (ruleset)                          description
0    [   UP] (/etc/hapee-3.2/my-rhi-config.cfg) loaded from /etc/hapee-3.2/my-rhi-config.cfg line 1 - type=ACLS/ADDRS - last-change=1h40m refcount=3
1    [   UP] (/etc/hapee-3.2/my-rhi-config.cfg) loaded from /etc/hapee-3.2/my-rhi-config.cfg line 0 - type=ACLS/ADDRS - last-change=5m59s refcount=3

Show the status of the rule with an ID of 0.

nix
echo -e "show rhi-bgp dc1 rule 0" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp dc1 rule 0" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
0: type=ACLS/ADDRS - state=UP - last-change=29s - refcount=4 - ruleset=- (loaded from /etc/hapee-3.2/hapee-lb.cfg line 59)
    routes: #id  addr
            0    192.168.1.10/32

output
text
0: type=ACLS/ADDRS - state=UP - last-change=29s - refcount=4 - ruleset=- (loaded from /etc/hapee-3.2/hapee-lb.cfg line 59)
    routes: #id  addr
            0    192.168.1.10/32

Show the status of the route with the ID of 0.

nix
echo -e "show rhi-bgp dc1 route 0" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp dc1 route 0" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
0: 192.168.1.10/32 - state=UP - last-change=14m11s - refcount=3 - rule=0 - ruleset=- (loaded from /etc/hapee-3.2/hapee-lb.cfg line 59)

output
text
0: 192.168.1.10/32 - state=UP - last-change=14m11s - refcount=3 - rule=0 - ruleset=- (loaded from /etc/hapee-3.2/hapee-lb.cfg line 59)

Show statistics for the rhi-bgp section named dc1.

nix
echo -e "show rhi-bgp dc1 stats" | sudo socat stdio tcp4-connect:127.0.0.1:9999

nix
echo -e "show rhi-bgp dc1 stats" | sudo socat stdio tcp4-connect:127.0.0.1:9999

output
text
RHI-BGP section 'dc1'
Total number of routes set to 'UKN' state: 0
Total number of routes set to 'UP' state: 1
Total number of routes set to 'DOWN' state: 0
Total number of routes set to 'F_UP' state: 0
Total number of routes set to 'F_DOWN' state: 0
Total number of routes set to 'DEL' state: 0
neighbor <0>: 192.168.0.1
  Total number of sessions: 1
  Total number of announces: 1
  Total number of withdraws: 0

output
text
RHI-BGP section 'dc1'
Total number of routes set to 'UKN' state: 0
Total number of routes set to 'UP' state: 1
Total number of routes set to 'DOWN' state: 0
Total number of routes set to 'F_UP' state: 0
Total number of routes set to 'F_DOWN' state: 0
Total number of routes set to 'DEL' state: 0
neighbor <0>: 192.168.0.1
  Total number of sessions: 1
  Total number of announces: 1
  Total number of withdraws: 0

Product Documentation

API Documentation

Configuration Documentation

Install the VM image

Route health injection

Concepts Jump to heading

How ECMP routing works Jump to heading

How Route Health Injection works Jump to heading

Prerequisites Jump to heading

Install and configure RHI Jump to heading

Install the RHI module Jump to heading

Configure route health injection Jump to heading

Intercept traffic destined for the IP Jump to heading

Reference Jump to heading

rhi-bgp section Jump to heading

rhi-config file Jump to heading

rhi-legacy-config file Jump to heading

Runtime API Jump to heading

add rhi-bgp Jump to heading

del rhi-bgp Jump to heading

set rhi-bgp Jump to heading

show rhi-bgp Jump to heading

Install the VM image

Product Documentation

API Documentation

Configuration Documentation

Concepts Jump to heading #

How ECMP routing works Jump to heading #

How Route Health Injection works Jump to heading #

Prerequisites Jump to heading #

Install and configure RHI Jump to heading #

Install the RHI module Jump to heading #

Configure route health injection Jump to heading #

Intercept traffic destined for the IP Jump to heading #

Reference Jump to heading #

rhi-bgp section Jump to heading #

rhi-config file Jump to heading #

rhi-legacy-config file Jump to heading #

Runtime API Jump to heading #

add rhi-bgp Jump to heading #

del rhi-bgp Jump to heading #

set rhi-bgp Jump to heading #

show rhi-bgp Jump to heading #

Privacy Settings

Concepts Jump to heading

How ECMP routing works Jump to heading

How Route Health Injection works Jump to heading

Prerequisites Jump to heading

Install and configure RHI Jump to heading

Install the RHI module Jump to heading

Configure route health injection Jump to heading

Intercept traffic destined for the IP Jump to heading

Reference Jump to heading

rhi-bgp section Jump to heading

rhi-config file Jump to heading

rhi-legacy-config file Jump to heading

Runtime API Jump to heading

add rhi-bgp Jump to heading

del rhi-bgp Jump to heading

set rhi-bgp Jump to heading

show rhi-bgp Jump to heading