Client IP preservation
Enable transparent proxying
For scenarios where you’re proxying TCP traffic and cannot use the PROXY Protocol, you can use transparent proxying to preserve the client’s IP address when connecting to backend servers. This requires changing the default gateway on your backend servers so that return traffic passes through the load balancer, which may not be feasible in all cases.
To enable transparent proxying:
-
On the load balancer, edit
/etc/sysctl.conf
and add:nixnet.ipv4.ip_forward = 1net.ipv4.ip_nonlocal_bind = 1nixnet.ipv4.ip_forward = 1net.ipv4.ip_nonlocal_bind = 1 -
Restart the
systemd-sysctl
service.nixsudo systemctl restart systemd-sysctlnixsudo systemctl restart systemd-sysctl -
Add routing rules that intercept packets with a destination IP address matching a listening socket, which in this case is the client’s IP address. Also add policy-based routing rules to deliver the traffic locally.
Create firewall and routing rules:
nixsudo iptables -t mangle -N DIVERTsudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERTsudo iptables -t mangle -A PREROUTING -p udp -m socket -j DIVERTsudo iptables -t mangle -A DIVERT -j MARK --set-mark 1sudo iptables -t mangle -A DIVERT -j ACCEPTsudo ip rule add fwmark 1 lookup 100sudo ip route add local 0.0.0.0/0 dev lo table 100nixsudo iptables -t mangle -N DIVERTsudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERTsudo iptables -t mangle -A PREROUTING -p udp -m socket -j DIVERTsudo iptables -t mangle -A DIVERT -j MARK --set-mark 1sudo iptables -t mangle -A DIVERT -j ACCEPTsudo ip rule add fwmark 1 lookup 100sudo ip route add local 0.0.0.0/0 dev lo table 100To make the
iptables
changes persistent after reboot, use theiptables-save
command. It saves the changes and configures the system to restore them at reboot.nixsudo apt install iptables-persistentsudo su -c 'iptables-save > /etc/iptables/rules.v4'nixsudo apt install iptables-persistentsudo su -c 'iptables-save > /etc/iptables/rules.v4'To verify the IP tables rules, use the
iptables
command. Note that the output may showtcp
or the number 6, andudp
or the number 17:nixsudo iptables -L -v -n -t manglenixsudo iptables -L -v -n -t mangleoutputtextChain PREROUTING (policy ACCEPT 0 packets, 0 bytes)pkts bytes target prot opt in out source destination1941 335K DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket0 0 DIVERT udp -- * * 0.0.0.0/0 0.0.0.0/0 socket...Chain DIVERT (1 references)pkts bytes target prot opt in out source destination1941 335K MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK set 0x11941 335K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0outputtextChain PREROUTING (policy ACCEPT 0 packets, 0 bytes)pkts bytes target prot opt in out source destination1941 335K DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket0 0 DIVERT udp -- * * 0.0.0.0/0 0.0.0.0/0 socket...Chain DIVERT (1 references)pkts bytes target prot opt in out source destination1941 335K MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK set 0x11941 335K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0To make the policy route (
ip rule
) and route table (ip route
) changes persist after reboot, your next step depends on whether or not your system uses netplan. Typically, Ubuntu uses it, but Debian doesn’t.-
If your system uses netplan, persist the policy route (
ip rule
) and route table (ip route
) changes in the netplan YAML configuration file located in/etc/netplan
. The configuration file is probably the one having the lowest number and has a name like00-installer-config.yaml
or01-netcfg.yaml
.Edit the netplan YAML file, adding an
lo
section under theethernets
level:01-netcfg.yamlyamlnetwork:ethernets:lo:routing-policy:- to: 0.0.0.0/0mark: 1table: 100routes:- to: 0.0.0.0/0type: localtable: 10001-netcfg.yamlyamlnetwork:ethernets:lo:routing-policy:- to: 0.0.0.0/0mark: 1table: 100routes:- to: 0.0.0.0/0type: localtable: 100Then, use
sudo netplan try
andsudo netplan apply
before rebooting to ensure the configuration is valid. Ignore warnings about Open vSwitch. -
If your system doesn’t use netplan, persist the changes by creating a new service for loading them at boot. Create the
systemd
service file:nixsudo touch /etc/systemd/system/01-static-route.servicesudo vi /etc/systemd/system/01-static-route.servicenixsudo touch /etc/systemd/system/01-static-route.servicesudo vi /etc/systemd/system/01-static-route.serviceAdd the following lines to the service file:
01-static-route.servicetext[Unit]Description=Add route table 100Wants=network-online.targetAfter=network-online.target[Service]Type=oneshot# create the route table and ruleExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100[Install]WantedBy=multi-user.target01-static-route.servicetext[Unit]Description=Add route table 100Wants=network-online.targetAfter=network-online.target[Service]Type=oneshot# create the route table and ruleExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100[Install]WantedBy=multi-user.targetSet the service to start on boot:
nixsudo systemctl enable 01-static-route.servicenixsudo systemctl enable 01-static-route.serviceRestart the system. The saved settings will be restored after the restart.
Create firewall rules:
nixsudo firewall-cmd --permanent --direct --add-chain ipv4 mangle DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p tcp -m socket -j DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p udp -m socket -j DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 0 -j MARK --set-mark 1sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 1 -j ACCEPTnixsudo firewall-cmd --permanent --direct --add-chain ipv4 mangle DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p tcp -m socket -j DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING 0 -p udp -m socket -j DIVERTsudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 0 -j MARK --set-mark 1sudo firewall-cmd --permanent --direct --add-rule ipv4 mangle DIVERT 1 -j ACCEPTReload the firewall:
nixsudo firewall-cmd --reloadnixsudo firewall-cmd --reloadTo persist the IP routing rules, create a new service for loading them at boot:
nixsudo touch /etc/systemd/system/01-static-route.servicesudo vi /etc/systemd/system/01-static-route.servicenixsudo touch /etc/systemd/system/01-static-route.servicesudo vi /etc/systemd/system/01-static-route.serviceAdd the following lines to the service file:
01-static-route.servicetext[Unit]Description=Add route table 100Wants=network-online.targetAfter=network-online.target[Service]Type=oneshot# create the route table and ruleExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100[Install]WantedBy=multi-user.target01-static-route.servicetext[Unit]Description=Add route table 100Wants=network-online.targetAfter=network-online.target[Service]Type=oneshot# create the route table and ruleExecStart=-/usr/sbin/ip route add local 0.0.0.0/0 dev lo table 100ExecStart=-/usr/sbin/ip rule add fwmark 1 lookup 100[Install]WantedBy=multi-user.targetSet the service to start on boot:
nixsudo systemctl enable 01-static-route.servicenixsudo systemctl enable 01-static-route.serviceRestart the system.
To verify the rule table, use the
ip
command:nixsudo ip rule lsnixsudo ip rule lsoutputtext...32765: from all fwmark 0x1 lookup 100...outputtext...32765: from all fwmark 0x1 lookup 100...To verify the route table, use the
ip
command:nixsudo ip route ls table 100nixsudo ip route ls table 100outputtextlocal default dev lo scope hostoutputtextlocal default dev lo scope host -
-
Give the load balancer the capability to bind to an address with transparent proxying. You can either:
-
Add
setcap cap_net_raw
to theglobal
section.haproxyglobalsetcap cap_net_rawhaproxyglobalsetcap cap_net_raw -
Or run as the root user by removing the
user
andgroup
directives from theglobal
section.
-
-
Add a
source
directive to thebackend
section.- The first parameter sets the IP address to bind to before connecting to the server. You can set it to
0.0.0.0
for the load balancer to choose the appropriate address to reach the destination. - Set
usesrc clientip
to present the client’s IP address to the server.
haproxybackend serversmode tcpsource 0.0.0.0 usesrc clientipserver web1 192.168.56.12:80 checkhaproxybackend serversmode tcpsource 0.0.0.0 usesrc clientipserver web1 192.168.56.12:80 check - The first parameter sets the IP address to bind to before connecting to the server. You can set it to
-
On the backend server, make the load balancer the default gateway. You can either:
-
Set a default gateway for traffic originating from an IP range. Below, we set the default gateway to
192.168.56.10
, which is the load balancer’s address, for traffic originating on the192.168.56.0/24
IP range. Change the address and network adapter,eth1
, for your server.nixsudo ip rule add from 192.168.56.0/24 table 10sudo ip route add default via 192.168.56.10 dev eth1 table 10nixsudo ip rule add from 192.168.56.0/24 table 10sudo ip route add default via 192.168.56.10 dev eth1 table 10 -
Or change the default gateway for all traffic, which will affect SSH too.
nixsudo ip route delete defaultsudo ip route add default via 192.168.56.10 dev eth1nixsudo ip route delete defaultsudo ip route add default via 192.168.56.10 dev eth1
-
Your backend servers should now be able to receive the traffic and see the source address as the client’s.
Do you have any suggestions on how we can improve the content of this page?