Enterprise modules
Route health injection
The Route Health Injection (RHI) service interacts with the BIRD Internet Routing Daemon to start or stop the flow of traffic to this HAProxy Enterprise node depending on the health of the network and your load balanced servers. RHI is useful for scaling out an HAProxy Enterprise cluster in an active/active manner, ensuring high availability by relying on well established routing network protocols.
The RHI service adds this HAProxy Enterprise node as a route in BIRD’s configuration using a custom route table named volatile. BIRD then broadcasts these routes to peer routers using the BGP, RIP, or OSPF protocol. If either a frontend or a backend is down, then the RHI service removes the route from the volatile table, notifying BIRD to stop advertising this HAProxy Enterprise node as a route on the network, diverting the flow of traffic to the other HAProxy Enterprise node in the active-active cluster. You can configure ECMP on your router to load balance traffic to both HAProxy Enterprise nodes via the advertised routes.
Configure Route Health Injection Jump to heading
-
Install the RHI module using your package manager:
nixsudo apt-get install hapee-extras-rhinixsudo apt-get install hapee-extras-rhinixsudo yum install hapee-extras-rhinixsudo yum install hapee-extras-rhinixsudo zypper install hapee-extras-rhinixsudo zypper install hapee-extras-rhinixsudo pkg install hapee-extras-rhinixsudo pkg install hapee-extras-rhiThis installs the hapee-extras-route package too, which is our version of the BIRD Internet Routing Daemon. The daemon is stored as
/opt/hapee-extras/sbin/hapee-route
. -
Create a socket for the Runtime API.
The RHI service needs to connect to the Runtime API to collect information about the health of your frontends and backends. Add a new
stats socket
line to theglobal
section of your HAProxy Enterprise configuration. This exposes the Runtime API as the socket/var/run/hapee-extras/hapee-lb.sock
:haproxyglobalstats socket /var/run/hapee-extras/hapee-lb.sock user hapee-lb group hapee mode 660haproxyglobalstats socket /var/run/hapee-extras/hapee-lb.sock user hapee-lb group hapee mode 660The socket path
The RHI service expects the Runtime API socket to be
/var/run/hapee-extras/hapee-lb.sock
. However, you can change the path that the RHI service expects by setting the variableHAPEE_LB_SOCKET
in the following file:- On Debian/Ubuntu:
/etc/default/hapee-extras-rhi
- On RHEL:
/etc/sysconfig/hapee-extras-rhi
For example:
hapee-extras-rhitextHAPEE_LB_SOCKET="/var/run/hapee-2.9/hapee-lb.sock"hapee-extras-rhitextHAPEE_LB_SOCKET="/var/run/hapee-2.9/hapee-lb.sock"Still, you must ensure that this same path is configured in your HAProxy Enterprise configuration file:
hapee-lb.cfghaproxyglobalstats socket /var/run/hapee-2.9/hapee-lb.sock user hapee-lb group hapee mode 660hapee-lb.cfghaproxyglobalstats socket /var/run/hapee-2.9/hapee-lb.sock user hapee-lb group hapee mode 660 - On Debian/Ubuntu:
-
Configure the RHI service.
-
Edit the file
/etc/hapee-extras/hapee-rhi.cfg
.The default configuration contains an example:
hapee-rhi.cfgtext# Inject the 10.200.200.200/32 address into the route daemon if# all the backends "be_static" and "be_app" are up.10.200.200.200/32 = all(b:be_static,b:be_app)hapee-rhi.cfgtext# Inject the 10.200.200.200/32 address into the route daemon if# all the backends "be_static" and "be_app" are up.10.200.200.200/32 = all(b:be_static,b:be_app)This file contains a list of routes that the RHI service should add to BIRD, but only if a given rule returns true; a rule checks the status of one or more frontends or backends to see if they are up or down. A backend is treated as down if all servers fail their health checks or if you manually disable the servers. A frontend is down if you disable it manually.
For example, the following line uses the
all
rule to announce the192.168.1.10/32
IP only when both thebe_static
andbe_app
backend are up and running. When the condition is false, the IP is removed from the list of advertised routes:hapee-rhi.cfgtext192.168.1.10/32 = all(b:be_static,b:be_app)hapee-rhi.cfgtext192.168.1.10/32 = all(b:be_static,b:be_app)Or, the following line uses the
any
rule to advertise the IP if either thebe_app
orbe_app2
backends are up and running:hapee-rhi.cfgtext192.168.1.10/32 = any(b:be_app,b:be_app2)hapee-rhi.cfgtext192.168.1.10/32 = any(b:be_app,b:be_app2) -
Save the configuration and then enable and restart the service:
nixsudo systemctl enable hapee-extras-rhisudo systemctl restart hapee-extras-rhinixsudo systemctl enable hapee-extras-rhisudo systemctl restart hapee-extras-rhi
-
-
Add
iptables
rules to route traffic to the HAProxy Enterprise socket:nixsudo iptables -t mangle -N DIVERTsudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERTsudo iptables -t mangle -A DIVERT -j MARK --set-mark 1sudo iptables -t mangle -A DIVERT -j ACCEPTsudo ip rule add fwmark 1 lookup 100sudo ip route add local 0.0.0.0/0 dev lo table 100nixsudo iptables -t mangle -N DIVERTsudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERTsudo iptables -t mangle -A DIVERT -j MARK --set-mark 1sudo iptables -t mangle -A DIVERT -j ACCEPTsudo ip rule add fwmark 1 lookup 100sudo ip route add local 0.0.0.0/0 dev lo table 100To learn more about these commands, see the transparent proxy topic in the Linux kernel documentation.
-
Edit the HAProxy Enterprise configuration file,
/etc/hapee-2.9/hapee-lb.cfg
:-
In the
frontend
section, change the addresses on thebind
lines so that they use a new IP address that is not yet assigned to one of the network interfaces. -
Because the IP addresses are not actually configured on the network interface, configure transparent binding by adding the
transparent
argument to thebind
lines. This indicates that the IP address should be bound even though it does not belong to the local machine. Packets targeting this address will be intercepted as if the address were locally configured. This feature uses the kernel’s TPROXY feature, which has been available since Linux kernel 2.2.
haproxyfrontend wwwbind 192.168.1.10:80 name http transparentbind 192.168.1.10:443 name https ssl crt site.pem transparenthaproxyfrontend wwwbind 192.168.1.10:80 name http transparentbind 192.168.1.10:443 name https ssl crt site.pem transparent -
-
Save the changes and then restart the service.
nixsudo systemctl reload hapee-2.9-lbnixsudo systemctl reload hapee-2.9-lbEach HAProxy Enterprise node should be assigned the same IP address.
-
Configure BIRD for BGP or OSPF, which are used to advertise routes to peer routers.
-
Edit the file
/etc/hapee-extras/hapee-route.cfg
. -
Add a section for either BGP or OSPF, depending on which protocol you intend to use for advertising routes to peers. Within it, add an
export
line that advertises routes from the volatile table,vol1
.An example BGP configuration section:
hapee-route.cfgtextprotocol bgp r1 {local 192.168.0.101 as 65001;neighbor 192.168.0.1 as 65001;graceful restart on;import none;# advertise the IP routeexport where proto = "vol1";}hapee-route.cfgtextprotocol bgp r1 {local 192.168.0.101 as 65001;neighbor 192.168.0.1 as 65001;graceful restart on;import none;# advertise the IP routeexport where proto = "vol1";}In this example:
- the
local
directive refers to the IP address assigned to this HAProxy Enterprise node’s network interface and assigns the Autonomous System Number 65001. - the
neighbor
directive refers to the layer 3 device, such as the gateway router, with which we are establishing a BGP session. - the
neighbor
directive refers to the layer 3 device, such as the gateway router, with which we are establishing a BGP session.
An example OSPF configuration section:
hapee-route.cfgtextprotocol ospf anycast {tick 2;import none;# advertise the IP routeexport where proto = "vol1";area 0.0.0.0 {stub no;interface "eth0" {hello 10;retransmit 6;cost 10;transmit delay 5;dead count 4;wait 50;type broadcast;};};}hapee-route.cfgtextprotocol ospf anycast {tick 2;import none;# advertise the IP routeexport where proto = "vol1";area 0.0.0.0 {stub no;interface "eth0" {hello 10;retransmit 6;cost 10;transmit delay 5;dead count 4;wait 50;type broadcast;};};} - the
-
Save the configuration and then restart the service:
nixsudo systemctl restart hapee-extras-routenixsudo systemctl restart hapee-extras-route -
Repeat these steps for the
hapee-extras-route6
service if using IPv6.
-
-
Verify that RHI added a route to BIRD by calling the
show route
command. The IP should display.nixsudo /opt/hapee-extras/bin/hapee-route-cli show routenixsudo /opt/hapee-extras/bin/hapee-route-cli show routeoutputtextBIRD 1.6.3 ready.192.168.1.10/32 dev auto [vol1 18:54:15] * (0)outputtextBIRD 1.6.3 ready.192.168.1.10/32 dev auto [vol1 18:54:15] * (0)When you disable all servers in the backend, the command should not return this route.
-
Configure ECMP on your peer router to distribute traffic to the bound IP.
Volatile table Jump to heading
BIRD stores routes in routing tables, with each table associated with a particular protocol such as BGP or OSPF. The RHI service adds its own table named volatile, which allows it to add routes to BIRD dynamically.
To see the volatile table definition, edit the BIRD configuration.
-
Edit the file
/etc/hapee-extras/hapee-route.cfg
. -
Scroll down to the
protocol volatile
section.hapee-route.cfgtextprotocol volatile vol1 {# gateway <ip>}hapee-route.cfgtextprotocol volatile vol1 {# gateway <ip>}By default, BIRD announces routes through the gateway configured on the network interface, but you can specify a different network gateway by uncommenting the
gateway
directive and typing its IP address.For example:
hapee-route.cfgtextprotocol volatile vol1 {gateway 192.168.1.244}hapee-route.cfgtextprotocol volatile vol1 {gateway 192.168.1.244}You can add more volatile tables to support advertising routes for different frontends:
hapee-route.cfgtextprotocol volatile vol1 {}protocol volatile vol2 {}hapee-route.cfgtextprotocol volatile vol1 {}protocol volatile vol2 {}Then, prefix each route in the RHI service’s configuration,
/etc/hapee-extras/hapee-rhi.cfg
, with a table’s name:hapee-rhi.cfgtextvol1%192.168.1.10/32 = all(b:be_static,b:be_app)vol2%192.168.1.11/32 = any(b:k8s_servers)hapee-rhi.cfgtextvol1%192.168.1.10/32 = all(b:be_static,b:be_app)vol2%192.168.1.11/32 = any(b:k8s_servers)
Rules syntax Jump to heading
This section describes the syntax of the RHI configuration file.
text
<network>[,<network>,[...]] = <agg>(<b:|f:><name>[,<b:|f:><name>,[...]])
text
<network>[,<network>,[...]] = <agg>(<b:|f:><name>[,<b:|f:><name>,[...]])
Argument | Description |
---|---|
<network> |
[%<protoname>]{<ipv4>,<ipv6>}[/<mask>] Specify an IPv4 or IPv6 CIDR subnet or list of several comma-delimited subnets. If you do not specify any subnet mask, RHI applies the /32 mask. For advanced configuration, you can supply the name of a volatile table in the %<protoname> section (default is vol1 ). |
<agg> |
Aggregation function: all - Returns true if all listed proxies are active. any - Returns true if at least one of the proxies listed is active. never - Always false. For debugging purposes. |
<b:|f:> |
Prefix of either b for backend or f for frontend. |
<name> |
Name of the frontend or backend. |
See also Jump to heading
Do you have any suggestions on how we can improve the content of this page?