Searching HAProxy Enterprise 1.9r1
Stick Table Aggregator
Stick Table Aggregator
About the Stick Table Aggregator
The HAProxy Enterprise Stick Table Aggregator leverages HAProxy Enterprise stick tables to provide real-time, cluster-wide tracking of network activity.
Stick tables
HAProxy Enterprise instances store client session stickiness information as key-value structures in stick tables. Typical stick table keys are client IP addresses, and client identifiers (32 bits integers).
Soft and hard restarts
Current stick table entries are reloaded when an HAProxy Enterprise peer is restarted.
- Soft restart
-
The HAProxy Enterprise peer that restarts replicates relevant stick table content from the local shutting-down HAProxy Enterprise process.
- Hard restart
-
The HAProxy Enterprise peer that restarts replicates relevant stick table content from remote peers, if any.
See also
The Stick Table Aggregator
The Stick Table Aggregator stores stick table entries, just like HAProxy Enterprise instances. It sends aggregate entries in a distinct stick table that it sends back to the HAProxy Enterprise peers it listens to.
HAProxy Enterprise peers then store a local, unsynchronized stick table, and an aggregate stick table provided by the Stick Table Aggregator.
The Stick Table Aggregator supports peers protocol versions 2.0 and 2.1.
Installing the Stick Table Aggregator
Install the Stick Table Aggregator using your packaging manager.
To install the Stick Table Aggregator on Debian based systems, run:
$ sudo apt-get install hapee-extras-stktagg20
Managing the Stick Table Aggregator process
To start, stop, restart the Stick Table Aggregator, or display its status:
$ sudo systemctl <start|stop|restart|status> hapee-extras-stktagg
Running the Stick Table Aggregator
You can run the /opt/hapee-extras/sbin/hapee-stktagg
command with the following options:
Option | Description | Notes |
---|---|---|
| Displays help. | |
| Enters debug mode. | Cancels |
| Loads the configuration from file. | Mandatory |
| Starts the process as a daemon. | |
| Sets the effective group ID. | |
| Sets the effective user ID. |
Configuring a single level Stick Table Aggregator
The sections below will guide you through aggregating stick tables from several HAProxy Enterprise instances to a new stick table.
Edit the hapee-lb.cfg HAProxy Enterprise configuration files
-
Replace the HAProxy Enterprise instances' peer lines but the current instance's by a new line corresponding to the Stick Table Aggregator.
- Unaggregated HAProxy Enterprise peers A, B, and C
-
HAProxy Enterprise peers A, B, and C's configuration files:
# /etc/hapee-2.0/hapee-lb.cfg global stats socket /var/run/hapee-2.0/hapee-lb.sock defaults mode http frontend mytable.local bind *:80 stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers http-request track-sc0 src http-request deny deny_status 200 peers mypeers bind *:54321 peer A peer B peer C
- Aggregated HAProxy Enterprise peers A, B, and C
-
In HAProxy Enterprise A's configuration file, replace
peer
lines B and C bypeer
line AGGR:peers mypeers bind *:54321 server A server AGGR 192.168.122.13:54321
- In HAProxy Enterprise B's configuration file, replace
peer
-
lines A and C by
peer
line AGGR:peers mypeers bind *:54321 server B server AGGR 192.168.122.13:54321
- In HAProxy Enterprise C's configuration file, replace
peer
-
lines A and B by
peer
line AGGR:peers mypeers bind *:54321 server C server AGGR 192.168.122.13:54321
Note
Alternatively, you can define stick tables in the
peers
section:# /etc/hapee-2.0/hapee-lb.cfg global stats socket /var/run/hapee-2.0/hapee-lb.sock defaults mode http peers mypeers bind *:54321 server A server AGGR 192.168.122.13:54321 table mytable.local type ip size 100 expire 1h store http_req_rate(1h) table mytable.aggr type ip size 100 expire 1h store http_req_rate(1h)
-
Create a new backend with only the very same
stick-table
line as the one used to create the original, unaggregated stick table.The name of the new backend must comply with the naming convention you set up for the aggregate stick table on the
from
line of the Stick Table Aggregator's configuration file.Stick Table Aggregator's
from
line:from .local to .aggr
Original frontend or backend on HAProxy Enterprise instances:
frontend mytable.local
New backend on HAProxy Enterprise instances:
backend mytable.aggr
Note
Since the HAProxy Enterprise instances host both:
their original stick table, and,
the aggregate stick table,
you must create a new backend to listen to the aggregate stick table.
Warning
- Since the new backend section must not generate entries, it
-
only contains the
stick-table
line.
- Unaggregated HAProxy Enterprise peers A, B, and C
-
HAProxy Enterprise peers A, B, and C's configuration files contain the following:
frontend mytable.local bind *:80 stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers http-request track-sc0 src http-request deny deny_status 200
Aggregated HAProxy Enterprise peers A, B, and C
backend mytable.aggr stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers
Full aggregated HAProxy Enterprise peers A's configuration file:
1 # /etc/hapee-2.0/hapee-lb.cfg 2 global 3 stats socket /var/run/hapee-2.0/hapee-lb.sock 4 5 defaults 6 mode http 7 8 frontend mytable.local 9 bind *:80 10 stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers 11 http-request track-sc0 src 12 http-request deny deny_status 200 13 14 backend mytable.aggr 15 stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers 16 17 peers mypeers 18 bind *:54321 19 server A 20 server AGGR 192.168.122.13:54321
Note
The only difference between HAProxy Enterprise peers A, B, and C's configuration files occurs on line 19.
Warning
The local and the aggregate
stick-table
lines must be exactly the same (lines 10 and 15 in the example above).
HAProxy Enterprise peers stop synchronizing their stick tables, and communicate only with the Stick Table Aggregator.
Edit the stktagg.cfg Stick Table Aggregator configuration file
Declare the HAProxy Enterprise instances as peers.
Declare the Stick Table Aggregator itself as a
local
peer
.-
Create a naming convention for aggregate stick tables on the
from
line.To aggregate any <string>.local stick tables from HAProxy Enterprise peers A, B and C to stick table <string>.aggr on a Stick Table Aggregator whose name is AGGR:
# /etc/hapee-extras/hapee-stktagg.cfg global stats socket /tmp/stktagg.socket aggregations AGGR peer A 192.168.122.222:54321 peer B 192.168.122.76:54321 peer C 192.168.122.235:54321 peer AGGR 192.168.122.13:54321 local from .local to .aggr
Tip
The default suffix used within the Real-Time Dashboard when referencing aggregate tables is .aggr.
Configuring multilevel Stick Table Aggregators
You can hierarchically organize Stick Table Aggregators in a tree with as many levels as needed.
You can thus aggregate stick tables located in different geographical areas, for example.
Stick Table Aggregators A, B, C, and D are organized as follows:

Any <string>.local stick tables hosted by level 0 HAProxy Enterprise peers are aggregated by level 1 Stick Table Aggregators to stick tables <string>.aggr.
Tip
The default suffix used within the Real-Time Dashboard when referencing aggregate tables is .aggr.
Level 0 | Level 1 | Level 2 | |||
---|---|---|---|---|---|
HAProxy Enterprise peer | Stick table | Stick Table Aggregator | Aggregate stick table | Stick Table Aggregator | Aggregate stick table |
hB.1 | mytable.local | B | mytable.aggr | A | mytable.l2 |
hB.2 | mytable.local | ||||
hC.1 | mytable.local | C | mytable.aggr | ||
hC21 | mytable.local | ||||
hD.1 | mytable.local | D | mytable.aggr | ||
hD.2 | mytable.local |
Upward and downward peers
The up
and down
parameters specify whether a peer is upstream or downstream, relative to the current Stick Table Aggregator.

Aggregation from downward peers
A Stick Table Aggregator that receives updates from downward Stick Table Aggregators or HAProxy Enterprise peers forwards the aggregate results to all peers, including the peer that sent the updates.

Aggregation from upward peers
A Stick Table Aggregator that receives updates from upward Stick Table Aggregators does not aggregate them and forwards them to its downward peers if they match a forward rule.

Redundancy
The group
peer
line setting creates groups of upward Stick Table Aggregators peers. They are thus seen as a single upward Stick Table Aggregator peer by downward Stick Table Aggregators.
The double link between the level 1 Stick Table Aggregators and the upward, level 0 Stick Table Aggregator peers provides redundancy.
Stick Table Aggregators A, B, AGGR.1, and AGGR.2 are organized as follows:
If Stick Table Aggregator AGGR.1 fails, AGGR.2 continues to aggregate and distribute stick tables.
The table below lists on which peers the various local and aggregate stick tables are created:
Level 0 | Level 1 | Level 2 | |||
---|---|---|---|---|---|
HAProxy Enterprise peer | Stick table | Stick Table Aggregator | Aggregate stick table | Stick Table Aggregator | Aggregate stick table |
hA.1 | mytable.local | A | mytable.aggr | AGGR.1 | mytable.l2 |
hA.2 | mytable.local | ||||
hB.1 | mytable.local | B | mytable.aggr | AGGR.2 | mytable.l2 |
hB.2 | mytable.local |
The group
peer
line setting takes only one group identifier string as an argument.
Here are the configuration files of each Stick Table Aggregator in the diagram above.
We list only peers
(without socket addresses) and from
lines.
Stick Table Aggregator A's configuration file:
peer AGGR.1 <socket address> up group 1
peer AGGR.2 <socket address> up group 1
peer A <socket address> local
peer hA.1 <socket address>
peer hA.2 <socket address>
from any to .aggr
Stick Table Aggregator B's configuration file:
peer AGGR.1 <socket address> up group 1
peer AGGR.2 <socket address> up group 1
peer B <socket address> local
peer hB.1 <socket address>
peer hB.2 <socket address>
from any to .aggr
Stick Table Aggregator AGGR.1's configuration file:
peer A <socket address> down
peer B <socket address> down
peer AGGR.1 <socket address> local
from .aggr to .l2
Stick Table Aggregator AGGR.2's configuration file:
peer A <socket address> down
peer B <socket address> down
peer AGGR.2 <socket address> local
from .aggr to .l2
Stick Table Aggregator configuration file reference
At startup, the Stick Table Aggregator reads a mandatory text configuration file (/etc/hapee-1.9r1/hapee-stktagg.cfg
by default), similar to HAProxy Enterprise configuration files.
It comprises sections made up of one or several lines. Each line is identified by its first keyword and may comprise one or several keywords.
Note
In the rest of this document, we often refer to supported sections lines by their first keyword.
You can comment out lines by preceding them with a pound sign (#).
You can split lines by escaping end of lines with a backslash character (\).
Two sections are supported:
- global
-
Optional. Must be unique, if provided.
- aggregations
-
Mandatory. There may be several
aggregations
sections in the same configuration file.
global
[cpu-map set]
[source IP:port]
[stats socket IP:port]
aggregations <identifier>
peer <name> <ip:port> [local | up | down] [group group_id]
from any to <.suffix>|from <.suffix1,.suffix2,...> to <.suffix> [accept-no-suffix]
[forward <.suffix1,.suffix2...>]
global section (optional)
The configuration supports only one (optional) global
section.
global
[cpu-map set]
[source IP:port]
[stats socket IP:port]
cpu-map (optional)
Same meaning as for HAProxy Enterprise.
Configures the CPU affinity of the Stick Table Aggregator processes.
Comprises one or several CPU sets, each CPU set being a CPU number, or a range of CPU numbers.
CPU sets must be separated by a comma (,).
Select CPUs 0, 5, 6, 7 and 8:
cpu-map 0,5-8
source (optional)
Binds to a specific socket address for outgoing connections.
source
lines are declared as follows:
- IPv4 addresses
-
source <ipv4>[:<port>]
Port is optional. Defaults to 0 for random ports.
- IPv6 addresses
-
source <ipv6>:<port>
Port is mandatory if the address is not enclosed by square brackets ([]).
source [<ipv6>][:<port>]
The port is optional if the address is enclosed by square brackets ([]).
stats socket (optional)
Same meaning as for HAProxy Enterprise.
Creates a listener with which you can interact through a TCP or Unix domain socket with the Stick Table Aggregator at runtime.
Connect to 127.0.0.1 on port 11111 to access runtime CLI:
stats socket 127.0.0.1:11111
Use Unix domain socket /tmp/foo to access runtime CLI:
stats socket /tmp/foo
The following commands are available:
show aggrs
Lists all aggregation debugging information.
Gives the list of peers and aggregate tables for all aggregations sections in the configuration file.
The following fields are available:
- update
-
Displays the last key that was updated in the Stick Table Aggregator table.
- push
-
Displays the last key that was pushed to a given peer.
Always the same value as
update
in normal conditions.The
ack
field displays the last key that was acknowledged by a given peer. Usually a bit behind with actual traffic. - state
-
Protocol state of the peers session with a given peer.
A working peers session shows a state of
0x7
.
List aggregations:
$ sudo nc -U /tmp/foo <<< "show aggrs"
aggregations agg_example
table 'in' aggregated to 'in.aggr' (expire: 600000 nb_entries: 1)
update: 0x41f03e58 used: 0x41f03e68
peer 'buster1' (0) state: 0x7 push: 0x41f03e58 ack: 0x41f03a20 sync: 0x41f03a30
peer 'buster2' (1) state: 0x7 push: 0x41f03e58 ack: 0x41f03a20 sync: 0x41f03a30
peer 'buster3' (2) state: 0x7 push: 0x41f03e58 ack: 0x41f03a20 sync: 0x41f03a30
peer 'buster4' (3) state: 0x7 push: 0x41f03e58 ack: 0x41f03a20 sync: 0x41f03a30
show table
show table <aggregation ID> <name>
Dumps the content of a Stick Table Aggregator's aggregate stick table.
For each key, the aggregate entry and the entry sent by each peer are displayed.
Expired entries remain visible, and are reused when the same key is used again.
$ sudo nc -U /tmp/foo <<< "show table agg_example in.aggr"
key=192.168.122.1 http_req_cnt=86747 http_req_rate=(60000)=75040 exp=600000
buster1: http_req_cnt=62816 http_req_rate=(60000)=51107 exp=533281
buster2: http_req_cnt=0 http_req_rate=(60000)=0 expired
buster3: http_req_cnt=0 http_req_rate=(60000)=0 expired
buster4: http_req_cnt=23931 http_req_rate=(60000)=23931 exp=600000
help
Displays a help message.
aggregations section (mandatory)
An aggregations
section declaration keyword must be followed by a mandatory identifier (a string) that is only used internally by the stick table parsers.
Each aggregations
section comprises:
A list of peers with at least one
peer
lineA unique
from
line
aggregations <identifier>
peer <name> <ip:port> [local | up | down] [group group_id]
from any to <.suffix>|from <.suffix1,.suffix2,...> to <.suffix> [accept-no-suffix]
[forward <.suffix1,.suffix2...>]
Note
You do not need to configure stick tables on the Stick Table Aggregator since it reads stick table configurations from its remote peer.
peer (mandatory)
peer
lines are declared as follows:
peer <peer name> <ip:port> [local | up | down] [group <group_id>]
[ssl] [crt <path>] [ca-file <path] [verify none | optional | required]
A unique, mandatory peer
line with the local
keyword identifies the listening peer that listens to incoming remote peer connections.
peer
lines without the local
keyword identify remote peers.
name (mandatory)
Name of the peer.
ip:port (mandatory)
Socket address of the peer.
up | down (optional)
Optional, exclusive keywords for multilevel settings.
group (optional)
Optional keyword for multilevel settings.
ssl (optional)
Enables TLS/SSL.
crt (optional)
Certificate to present to the remote peer.
ca-file (optional)
File containing the trusted CAs in PEM format.
verify (optional)
none
-
Verification is disabled.
optional
-
A client certificate is requested, but verification is performed only if the remote peer provides a certificate in the response.
required
-
The remote peer certificate is mandatory in the response and verification is always performed.
from (mandatory)
Append a suffix to HAProxy Enterprise peers stick table names to name aggregate stick tables.
First syntax
from any to <.suffix>
Concatenate the names of the stick tables to be aggregated with <.suffix>, ignoring existing suffixes, if any.
Second syntax
from <.suffix1,.suffix2,...> to <.suffix> [accept-no-suffix]
Replaces any suffix found in the list of suffixes by <.suffix>.
<.suffix1,.suffix2,...> is a comma-separated list of suffixes with no spaces.
The
accept-no-suffix
optional keyword also appends the suffix to stick tables with no suffix.
Stick Table Aggregator stktagg_1 aggregates stick tables mytable
hosted by hostA and hostB to mytable.aggr.
global
cpu-map 2-3
aggregations agg_1
peer hostA 192.168.122.222:54321
peer hostB 192.168.122.76:54321
peer stktagg_1 192.168.122.13:54321 local
from any to .aggr
If we edit the from
line as follows:
from .cpu,.proc to .aggr
the Stick Table Aggregator creates the following aggregate stick tables:
Unaggregated stick table | Aggregated stick table |
---|---|
mytable1.cpu | mytable1.aggr |
mytable2.proc | mytable2.aggr |
If we edit the from
line as follows:
from .cpu,.proc to .aggr accept-no-suffix
the Stick Table Aggregator creates the following aggregate stick tables:
Unaggregated stick table | Aggregated stick table |
---|---|
mytable | mytable.aggr |
mytable1.cpu | mytable1.aggr |
mytable2.proc | mytable2.aggr |
Tip
The default suffix used within the Real-Time Dashboard when referencing aggregate tables is .aggr.
forward (optional)
Only for multilevel settings.
forward <.suffix1,.suffix2...>
The stick table with those suffixes are regarded as forwarded from upward to downward peers.
Note
Updates to these stick tables from a downward peer are ignored.
Aggregation for the middle level:
aggregations agg_1 peer hostA 192.168.122.222:54321 peer hostB 192.168.122.76:54321 peer stktagg 192.168.122.13:54321 local peer upstream 192.168.122.235:54321 up from any to .aggr forward .aggr
Aggregation for the top level:
aggregations agg_1 peer upstream 192.168.122.235:54321 local peer stktagg 192.168.122.13:54321 down [...] from .aggr to .l2
Tip
The default suffix used within the Real-Time Dashboard when referencing aggregate tables is .aggr.