Load balance UDP
UDP load balancing reference
This page applies to:
- HAProxy ALOHA 17.0 and newer
This page explains the keywords that you can use when configuring UDP load balancing.
Directives Jump to heading
This section describes the directives that you can add to a udp-lb configuration section.
Configuring udp-lb sections
udp-lb sections don’t inherit settings from defaults sections. You must define the following directives explicitly within each appropriate udp-lb section.
accepted-payload-size Jump to heading
The accepted-payload-size directive sets the maximum UDP datagram payload size in bytes. The default value is 1472, and the maximum value allowed is 65507.
Syntaxtextaccepted-payload-size <number>
Syntaxtextaccepted-payload-size <number>
acl Jump to heading
This section applies to:
- HAProxy ALOHA 17.5 and newer
- HAProxy Enterprise 3.2r1 and newer
The acl directive configures an ACL expression. See ACLs for more implementation details.
Syntaxtextacl <name> <criterion> [<ACL flags>] [<operator>] <value>
Syntaxtextacl <name> <criterion> [<ACL flags>] [<operator>] <value>
balance Jump to heading
The balance directive sets the load balancing algorithm.
Syntaxtextbalance <algorithm>
Syntaxtextbalance <algorithm>
The UDP module supports the following algorithms for balance:
static-rrroundrobinleastconnfirstsourcerandom
default-server Jump to heading
The default-server directive sets default parameters that will apply to all server lines within the same section. For a list of supported parameters, see default-server options.
Syntaxtextdefault-server [param*]
Syntaxtextdefault-server [param*]
dgram-bind Jump to heading
The dgram-bind directive configures a datagram listener to receive messages to forward. Addresses must be in IPv4 or IPv6 form, followed by a port.
Syntaxtextdgram-bind <addr>:<port> [param*]
Syntaxtextdgram-bind <addr>:<port> [param*]
The dgram-bind directive supports these bind parameters:
maxconnnamespaceniceshardsthreadtransparent
hash-balance-factor Jump to heading
This section applies to:
- HAProxy ALOHA 17.0 and newer
- HAProxy Enterprise 3.1r1 and newer
The hash-balance-factor directive specifies the balancing factor for bounded-load consistent hashing. See hash-balance-factor for more details.
Syntaxtexthash-balance-factor <factor>
Syntaxtexthash-balance-factor <factor>
hash-type Jump to heading
This section applies to:
- HAProxy ALOHA 17.0 and newer
- HAProxy Enterprise 3.1r1 and newer
The hash-type directive specifies a method to use for mapping hashes to servers. See hash-type for more details.
Syntaxtexthash-type <method> <function> <modifier>
Syntaxtexthash-type <method> <function> <modifier>
log Jump to heading
This section applies to:
- HAProxy ALOHA 17.0 and newer
- HAProxy Enterprise 3.0r1 and newer
The log directive enables per-instance logging of events. For requests, the source is the client’s IP/port, and the destination is the listener’s IP/port. For responses, the source is the listener’s IP/port, and the destination is the client’s IP/port.
For details, see log reference.
Syntaxtextlog <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>] <facility> [<level> [<minlevel>]]
Syntaxtextlog <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>] <facility> [<level> [<minlevel>]]
log global Jump to heading
This section applies to:
- HAProxy ALOHA 17.0 and newer
- HAProxy Enterprise 3.0r1 and newer
The log global directive sets the instance’s logging parameters to be the same as the global ones.
Syntaxtextlog global
Syntaxtextlog global
log-tag Jump to heading
This section applies to:
- HAProxy ALOHA 17.0 and newer
- HAProxy Enterprise 3.0r1 and newer
The log-tag directive sets the log tag string to use for all outgoing logs.
Syntaxtextlog-tag <string>
Syntaxtextlog-tag <string>
maxconn Jump to heading
The maxconn directive sets the maximum number of concurrent connections. Once the limit is reached, all datagrams that would initiate a new UDP connection will be dropped.
Syntaxtextmaxconn <integer>
Syntaxtextmaxconn <integer>
option persist Jump to heading
This section applies to:
- HAProxy ALOHA 17.5 and newer
- HAProxy Enterprise 3.2r1 and newer
The option persist directive enables forced persistence on downed servers by forcing the datagram to be sent to the downed server(s) first.
Syntaxtextoption persist
Syntaxtextoption persist
option tcp-check Jump to heading
The option tcp-check directive performs health checks using TCP connection attempts.
Syntaxtextoption tcp-check
Syntaxtextoption tcp-check
option udp-check Jump to heading
The option udp-check directive performs health checks by sending a zero-byte UDP datagram to the target server’s address and port. It then reads the UDP socket’s kernel error queue. If the kernel delivers an ICMP Destination Unreachable error in response, the server is marked as down, meaning the host actively rejected the probe. If there’s no error, the server is marked as up. Note that there is no ICMP echo involved. However, any intermediary firewalls must permit ICMP Unreachable messages to return to the load balancer or else the UDP module, not receiving the error, will assume the server is up, even if it isn’t. If your firewall cannot allow this type of traffic, then it’s better to use option tcp-check if possible.
This is the default type of health check.
Syntaxtextoption udp-check
Syntaxtextoption udp-check
udp-check directives
In HAProxy Enterprise 3.4r1 / HAProxy ALOHA 18.5 and newer, you can define actions for health checks. When one or more udp-check directives are configured in the udp-lb section, the default check is replaced: the load balancer runs the send and expect sequence you defined and marks the server as up only when every reply matches its expect action. See udp-check for the action syntax.
proxy-requests Jump to heading
The proxy-requests directive sets the number of expected datagrams per client session. Since UDP isn’t a connection-oriented protocol, the UDP module must keep track of a client’s session such that it can route the response datagrams from an upstream server back to the correct client. Each session is indexed by the 4-tuple consisting of source IP/port and destination IP/port corresponding to the datagram.
-
If this option isn’t set, then the load balancer will forward all datagrams from the client to the same backend server as long as the client is considered alive. If the client becomes inactive, their session expires, and the next time they send a datagram, the load balancer will again choose a server based on the load balancing algorithm.
-
If this option is set to a value greater than 0, then session stickiness is disabled and the load balancer will choose the backend server on every
<number>datagrams received. For example, if set toproxy-requests 1, then a destination server will be rotated after each datagram received from the client.
Syntaxtextproxy-requests <number>
Syntaxtextproxy-requests <number>
proxy-responses Jump to heading
The proxy-responses directive sets the number of expected responses from the server. Sessions last until the timeout is reached or the expected number of responses has been received. If a zero value is specified, all responses from the server will be ignored and not forwarded back to the client. If a value isn’t specified, the number of expected responses is set to unlimited.
Syntaxtextproxy-responses <number>
Syntaxtextproxy-responses <number>
server Jump to heading
The server directive configures a target server.
Syntaxtextserver <name> <address>[:[port]] [param*]
Syntaxtextserver <name> <address>[:[port]] [param*]
source Jump to heading
The source directive sets the source address for outgoing connections. The <addr> and optional <port> will be used for binding before connecting to the server. The <addr2> and <port2> are presented to the server when connections are forwarded in full transparent proxy mode. If client or clientip is set, the load balancer will present the client’s IP address and port, or the client’s IP address only.
Syntax for setting source addresstextsource <addr>[:<port>] [usesrc { <addr2>[:<port2>] | client | clientip } ]
Syntax for setting source addresstextsource <addr>[:<port>] [usesrc { <addr2>[:<port2>] | client | clientip } ]
Syntax for setting interface nametextsource <addr>[:<port>] [interface <name>]
Syntax for setting interface nametextsource <addr>[:<port>] [interface <name>]
tcp-check Jump to heading
The tcp-check directive configures TCP health checking.
Syntaxtexttcp-check <option> [param*]
Syntaxtexttcp-check <option> [param*]
Supported options are:
commentconnectsendsend-lfsend-binarysend-binary-lfexpectset-varset-var-fmtunset-var
timeout check Jump to heading
This section applies to:
- HAProxy Enterprise 3.4r1 and newer
- HAProxy ALOHA 18.5 and newer
The timeout check directive sets the maximum time to wait for a udp-check reply sequence to complete. It defaults to the check interval, the inter value on the server line, when not set.
Syntaxtexttimeout check <timeout>
Syntaxtexttimeout check <timeout>
timeout client Jump to heading
The timeout client directive sets the maximum inactivity time on the client side. The default is 10 seconds, but the ideal setting depends on your traffic and application. For example, if you have a large amount of traffic and a large number of client IP addresses and ports, you can lower the value in order to avoid tracking a high number of connections unnecessarily.
Syntaxtexttimeout client <timeout>
Syntaxtexttimeout client <timeout>
timeout server Jump to heading
The timeout server directive sets the maximum inactivity time on the server side.
Syntaxtexttimeout server <timeout>
Syntaxtexttimeout server <timeout>
track Jump to heading
This section applies to:
- HAProxy Enterprise 3.4r1 and newer
- HAProxy ALOHA 18.5 and newer
The track directive makes a server’s health status follow the health of another server. A server that tracks another server runs no health check of its own; its status mirrors the tracked server’s. This is useful when a service is exposed over both TCP and UDP, so that the UDP server is taken out of rotation whenever its TCP counterpart fails its check.
Syntaxtexttrack <backend>/<server>
Syntaxtexttrack <backend>/<server>
The tracked server must have a health check or an agent enabled. A server can’t run a health check and track another server at the same time. Any type of health check can be tracked, including a TCP check, an HTTP check, or a udp-check send and expect sequence on another udp-lb server.
udp-check Jump to heading
This section applies to:
- HAProxy Enterprise 3.4r1 and newer
- HAProxy ALOHA 18.5 and newer
The udp-check directive configures a datagram health-check sequence. Each send action transmits a probe datagram and each expect action waits for one reply datagram and matches it. The check passes only when every expect matches. All servers in the udp-lb section are checked with the same sequence.
Syntaxtextudp-check <action> [param*]
Syntaxtextudp-check <action> [param*]
Actions Jump to heading
Configure one sequence of actions in each udp-lb section. All servers in the section are checked with the same sequence.
| Action | Description |
|---|---|
comment <msg> |
Attaches a description to the check. The load balancer uses it in logs and on the stats socket. |
send <string> |
Sends one datagram with the given payload. |
send-binary <hexstring> |
Sends one datagram with a payload given as hexadecimal digits, for example 70696e67. |
expect [min-recv <n>] [comment <msg>] [ok-status <s>] [error-status <s>] [tout-status <s>] [!] <match> <pattern> |
Waits for one reply datagram and matches it against the pattern. The check fails when the reply does not match. An
Set
Prefix a match with |
Check results Jump to heading
The check reports one of the following results, visible on the stats page, the Runtime API, and the dashboard:
| Status | Meaning |
|---|---|
L7OK |
The server replied and every expect matched. |
L7RSP |
The server replied, but a reply didn’t match, or the reply was larger than tune.bufsize. |
L7TOUT |
The server didn’t reply within timeout check. |
L4CON |
The load balancer couldn’t send the probe or the kernel reported a socket error; typically the error is ICMP Unreachable when the server port is closed. |
Rules for valid sequences Jump to heading
The load balancer rejects a udp-lb section at startup when the actions can never produce a meaningful result:
- The sequence must contain at least one
expectaction. A sequence with onlysendactions verifies nothing, because a closed UDP port doesn’t respond to a datagram with an error the check can observe. - The sequence must contain at least one
sendaction, placed before the firstexpect. Until a datagram is sent, the server doesn’t know the check exists and never replies, so anexpectwithout a precedingsendcan only time out. - A sequence of only
commentactions is rejected for both of the reasons above.
Additional behaviors to keep in mind:
- Each
expectconsumes exactly one reply datagram. UDP has no stream to reassemble. - A reply larger than
tune.bufsizecan’t be matched. The check fails with the messageresponse too largerather than evaluating a truncated reply. - Log-format payloads such as
send-lf, and theconnectandset-varactions supported bytcp-check, aren’t available toudp-checkdirectives. option tcp-checktakes priority. If bothoption tcp-checkandudp-checkdirectives are configured, the load balancer logs a warning and theudp-checkdirectives never run.timeout checkbounds the time the check waits for the reply sequence to complete. It defaults to the check interval, theintervalue on theserverline.
udp-request Jump to heading
This section applies to:
- HAProxy ALOHA 18.5 and newer
- HAProxy Enterprise 3.4r1 and newer
The udp-request directive invokes a per-datagram action before selecting a server. Actions are evaluated in the order they appear for each incoming request datagram. Add an if or unless clause to invoke the action based on an ACL condition.
Syntaxtextudp-request <action> [{ if | unless } <condition>]
Syntaxtextudp-request <action> [{ if | unless } <condition>]
Use any of the following actions:
| Action | Description |
|---|---|
accept |
Accept the datagram and skip evaluating any more actions. |
dgram-drop |
Same as reject. |
reject |
Reject (drop) the datagram and skip evaluating any more actions. |
sc-add-gpc(<idx>,<sc>) <expr> |
Add the integer value resulting from <expr> to the general purpose counter that has index <idx> and is associated with sticky counter <sc>. General purpose counters are integer values that you can increment. |
sc-inc-gpc(<idx>,<sc>) |
Increment by one the general purpose counter that has index <idx> and is associated with sticky counter <sc>. |
sc-inc-gpc0([<sc>]) |
Increment by one the general purpose counter 0 associated with sticky counter <sc>. |
sc-inc-gpc1([<sc>]) |
Increment by one the general purpose counter 1 associated with sticky counter <sc>. |
sc-set-gpt(<idx>,<sc>) <expr> |
Set a general purpose tag that has index <idx> and is associated with sticky counter <sc> to the expression <expr>. General purpose tags are integer values that you can set to a value, such as 0 or 1 for indicating on/off or to flag a client. |
sc-set-gpt0([<sc>]) <expr> |
Set the general purpose tag 0 associated with sticky counter <sc> to the expression <expr>. |
set-var(<name>) <expr> |
Set a variable named <name> to the result of a sample expression. The txn.* and req.* scopes are reset on every datagram, so a key derived from the payload is evaluated afresh per datagram (the way HTTP resets the txn scope per request). Use the sess.* scope for state that must persist across datagrams within the same UDP connection. |
set-var-fmt(<name>) <fmt> |
Set a variable named <name> using a log-format string. |
track-sc<N> <key> [table <name>] |
Track a stick-table entry for sticky counter <N>. This enables you to use the sc-* actions and the sc* fetch methods. Define the stick table in a separate TCP/HTTP backend section and reference it via the table argument. |
unset-var(<name>) |
Unset the variable named <name>. |
Here’s an example of how to restrict the rate at which a client can create connections. The conn_rate counter counts new UDP connections, so this limits how fast a client, tracked by source IP address, can open new flows. A source/destination address tuple accounts for a UDP connection. It counts once for stick table counters like conn_cnt, conn_cur, and sess_cnt, no matter how many datagrams it carries. The gauges drop when the connection expires (timeout client) or its batch completes.
haproxybackend udp_rate_limitstick-table type ip size 100k expire 30s store conn_rate(10s)udp-lb dns_proxydgram-bind *:5353proxy-responses 1udp-request track-sc0 src table udp_rate_limitudp-request reject if { sc0_conn_rate(udp_rate_limit) gt 100 }server dns1 8.8.8.8:53server dns2 8.8.4.4:53
haproxybackend udp_rate_limitstick-table type ip size 100k expire 30s store conn_rate(10s)udp-lb dns_proxydgram-bind *:5353proxy-responses 1udp-request track-sc0 src table udp_rate_limitudp-request reject if { sc0_conn_rate(udp_rate_limit) gt 100 }server dns1 8.8.8.8:53server dns2 8.8.4.4:53
The next example shows how to limit the datagram rate itself. There isn’t a built-in per-datagram counter. Instead, count each datagram by using a general purpose counter. The udp-request sc-inc-gpc0 directive increments the general purpose counter before the udp-request reject action so rejected datagrams keep feeding the rate, like when tracking the HTTP request rate with http_req_rate.
haproxybackend udp_dgram_limitstick-table type ip size 100k expire 30s store gpc0_rate(10s)udp-lb dns_proxydgram-bind *:5353proxy-responses 1udp-request track-sc0 src table udp_dgram_limitudp-request sc-inc-gpc0udp-request reject if { sc0_gpc0_rate(udp_dgram_limit) gt 100 }server dns1 8.8.8.8:53server dns2 8.8.4.4:53
haproxybackend udp_dgram_limitstick-table type ip size 100k expire 30s store gpc0_rate(10s)udp-lb dns_proxydgram-bind *:5353proxy-responses 1udp-request track-sc0 src table udp_dgram_limitudp-request sc-inc-gpc0udp-request reject if { sc0_gpc0_rate(udp_dgram_limit) gt 100 }server dns1 8.8.8.8:53server dns2 8.8.4.4:53
In the next example, we show how to track RADIUS clients. A RADIUS client multiplexes many transactions over one socket, and multi-round exchanges (e.g. Access-Challenge) must keep communicating with the same server. Leave proxy-responses unset so that any number of responses will be forwarded and the session will last until a timeout; use balance source so the client is re-bound to the same server whenever the binding is re-created.
haproxybackend radius_usersstick-table type string len 64 size 100k expire 5m store gpc0_rate(1m)udp-lb radius_proxydgram-bind *:1812balance sourcetimeout client 30stimeout server 5sacl is_access_request req.payload(0,1) -m bin 01acl user_name_first req.payload(20,1) -m bin 01udp-request track-sc0 req.payload_lv(21,1,-2) table radius_users if is_access_request user_name_firstudp-request sc-inc-gpc0 if is_access_request user_name_firstudp-request reject if { sc0_gpc0_rate(radius_users) gt 60 }server radius1 10.0.0.10:1812server radius2 10.0.0.20:1812
haproxybackend radius_usersstick-table type string len 64 size 100k expire 5m store gpc0_rate(1m)udp-lb radius_proxydgram-bind *:1812balance sourcetimeout client 30stimeout server 5sacl is_access_request req.payload(0,1) -m bin 01acl user_name_first req.payload(20,1) -m bin 01udp-request track-sc0 req.payload_lv(21,1,-2) table radius_users if is_access_request user_name_firstudp-request sc-inc-gpc0 if is_access_request user_name_firstudp-request reject if { sc0_gpc0_rate(radius_users) gt 60 }server radius1 10.0.0.10:1812server radius2 10.0.0.20:1812
Stick table counters Jump to heading
When you track a UDP request with track-sc<N>, the stick table entry records the following counters:
| Counters | Description |
|---|---|
conn_cnt, conn_rate(<period>) |
Number of UDP connections, and the rate of new connections. A connection is a source/destination address tuple. It counts once, no matter how many datagrams it carries. |
conn_cur |
Number of currently active connections. It drops when the connection expires (timeout client) or its batch completes. |
sess_cnt, sess_rate(<period>) |
Number of sessions, and the rate of new sessions. Accounted the same way as connections for UDP traffic. |
bytes_in_cnt, bytes_in_rate(<period>) |
Bytes received from clients. Counted on reception, including datagrams later rejected by udp-request rules, the same way TCP counts bytes before a content rule denies them. |
bytes_out_cnt, bytes_out_rate(<period>) |
Bytes forwarded from the server back to the client. |
There is no built-in per-datagram counter. To count or rate-limit datagrams, increment a general purpose counter with sc-inc-gpc0 or sc-inc-gpc1 on each datagram and track gpc0_rate or gpc1_rate in the table. The per-datagram rate limiting example above shows this pattern.
Info
Counters that only HTTP updates, such as http_req_rate, don’t update for UDP traffic.
You can inspect the counters with the Runtime API command show table followed by the table name. This example shows the entry for a client that sent five datagrams, each of them carrying a seven byte payload and receiving a twelve byte response, with udp-request sc-inc-gpc0 configured:
nixecho -e "show table udp_stk" | \sudo socat stdio unix-connect:/var/run/hapee-3.4/hapee-lb.sock
nixecho -e "show table udp_stk" | \sudo socat stdio unix-connect:/var/run/hapee-3.4/hapee-lb.sock
nixecho -e "show table udp_stk" | \sudo socat stdio /var/run/haproxy.stat
nixecho -e "show table udp_stk" | \sudo socat stdio /var/run/haproxy.stat
outputtext# table: udp_stk, type: ip, size:102400, used:10xaaaafb0d39e0: key=127.0.0.1 use=1 exp=57983 shard=0 gpc0=5 gpc0_rate(10000)=5 conn_cnt=1 conn_rate(10000)=1 conn_cur=1 sess_cnt=1 sess_rate(10000)=1 bytes_in_cnt=35 bytes_in_rate(10000)=35 bytes_out_cnt=60 bytes_out_rate(10000)=60
outputtext# table: udp_stk, type: ip, size:102400, used:10xaaaafb0d39e0: key=127.0.0.1 use=1 exp=57983 shard=0 gpc0=5 gpc0_rate(10000)=5 conn_cnt=1 conn_rate(10000)=1 conn_cur=1 sess_cnt=1 sess_rate(10000)=1 bytes_in_cnt=35 bytes_in_rate(10000)=35 bytes_out_cnt=60 bytes_out_rate(10000)=60
The five datagrams arrived over a single UDP connection, so conn_cnt, conn_rate, sess_cnt, and sess_rate each count one. The sc-inc-gpc0 action ran once per datagram, so gpc0 counts five. The bytes counters show the five incoming datagrams at 35 bytes in total and the five responses at 60 bytes.
use-server Jump to heading
This section applies to:
- HAProxy ALOHA 17.5 and newer
- HAProxy Enterprise 3.2r1 and newer
The use-server directive assigns a specified server if/unless a condition is matched. For example, use a target UDP server based on the source IP.
If there is more than one use-server directive configured, they are evaluated in their declaration order. The first use-server directive that matches a condition will assign the specified server, unless the current session has already assigned that server. If no condition is valid, the unassigned servers will use other load balancing mechanisms.
When <server> is a simple name, the load balancer checks it against existing servers in the configuration and reports an error if the specified server doesn’t exist. If <server> is in a custom log format, no check is performed when parsing the configuration. If the load balancer can’t resolve a valid server name at runtime, but a use-server condition was met, no other use-server directive is applied, and the load balancer will fall back to load balancing.
Syntaxtextuse-server <server> { if | unless } <condition>
Syntaxtextuse-server <server> { if | unless } <condition>
In this example, we use use-server to select the DNS server based on an ACL condition.
haproxyudp-lb dns_proxydgram-bind *:5353proxy-responses 1# google.com lookupacl is_google req.payload(12,12) -m bin 06676f6f676c6503636f6d00use-server google_dns if is_googleserver google_dns 8.8.8.8:53server default_dns 1.1.1.1:53
haproxyudp-lb dns_proxydgram-bind *:5353proxy-responses 1# google.com lookupacl is_google req.payload(12,12) -m bin 06676f6f676c6503636f6d00use-server google_dns if is_googleserver google_dns 8.8.8.8:53server default_dns 1.1.1.1:53
Fetch methods Jump to heading
This section applies to:
- HAProxy ALOHA 18.5 and newer
- HAProxy Enterprise 3.4r1 and newer
This section describes fetch methods that return information about the current UDP request. You can use fetch methods in if / unless conditions when configuring an action with udp-request and when setting which server to use with use-server. The fetch methods become available only when you use these directives.
Connection-oriented fetches (they begin with fc_* and bc_*) aren’t available, since a datagram flow has no connection behind it, so they never produce a sample and a rule using one never matches; this produces a warning when the configuration is read.
The req.payload and req.len sample fetches apply only to udp-lb sections. The former returns the datagram payload and the latter returns the payload length. Note that req.payload returns SMP_T_BIN type data. String-based matchers (-m sub, -m beg, -m reg) perform a BIN-to-STR cast that truncates at the first NULL byte. For binary protocols (e.g. DNS), use -m bin with the hex converter at known offsets, or extract a NULL-free slice with specific offset and length (e.g. req.payload(13,6) -m str foobar).
The following categories of fetch methods are supported:
- L4 client info:
src,src_port,dst,dst_port,src_is_local,dst_is_local - Frontend info:
fe_name,fe_id,fe_conn,fe_req_rate,fe_sess_rate - Backend/server stats:
nbsrv(<be>),srv_is_up(<srv>),be_conn(<be>),be_sess_rate(<be>),srv_conn(<srv>) - Request data:
req.payload(<offset>,<length>),req.len - Sticky counters:
sc0_conn_cnt,sc0_conn_rate,sc0_get_gpc0, etc. - Variables:
var(txn.*),var(req.*),var(sess.*) - Internal state:
date,rand,thread, etc.