Terminology

stick-table source

A stick-table whose entries must be aggregated to others.

stick-table destination

A stick-table whose entries are the results of an aggregation.

Install the stick-table aggregator

As with other HAProxy Enterprise modules, you must install the HAProxy Enterprise stick-table aggregator using the Linux distribution packaging manager.

The stick-table aggregator needs to be run on a system which isn't running an HAProxy instance that needs to be a part of the cluster (e.g. doesn't have stick tables which need to be aggregated by this daemon); this is due to a technical limitation of the peers protocol where announcements from the same IP address will be ignored (and which we hope to resolve in a future release).

Manage the Stick Table Aggregator process

You can start, stop, or restart the stick-table aggregator using the System V init script /etc/init.d/hapee-1.9r1-stktagg or using systemd as follows:

$ sudo service <start|stop|restart|status> hapee-extras-stktagg

By default the stick table aggregator will start as root; to change this behavior, you can modify the file /etc/default/hapee-1.9r1-stktagg/ file to start it with hapee-stktagg as a user and hapee as group as follows:

STKTAGG_OPTIONS="-D -U hapee-stktagg -G hapee"

Configure the stick-table aggregator

The following table lists the command line options:

Other options

:~# echo "show table aggr_c3811-anycast src_table.aggr" |socat tcp-connect:127.0.0.1:4545 -|head
key=192.168.1.123 gpt0=0 gpc0=0 gpc0_rate=(10000)=0 conn_rate=(10000)=0 http_req_cnt=0 http_req_rate=(10000)=0 http_err_cnt=0 http_err_rate=(10000)=0 gpc1=0 gpc1_rate=(10000)=0
key=192.168.1.124 gpt0=0 gpc0=0 gpc0_rate=(10000)=0 conn_rate=(10000)=0 http_req_cnt=0 http_req_rate=(10000)=0 http_err_cnt=0 http_err_rate=(10000)=0 gpc1=0 gpc1_rate=(10000)=0

Configuring HAProxy

You integrate an HAProxy peer with the aggregator by adding the aggregator peer information to the peers section of the HAProxy configuration, define a stick table which the node will use, and a second stick table to hold the received aggregated values.

The configuration file

The configuration file for the stick-table aggregator is located at /etc/hapee-1.9r1/hapee-stktagg.cfg and is required for it to operate.

This file is similar to other HAProxy configuration files containing sections made up of several lines:

• Each line is identified by its first keyword and may be made of one or several keywords. In this document we reference sections/lines by the first keyword used (e.g. 'peer').

• Lines may be commented out using the hash character (#).

• The end of the line can be exited using a backslash character (\) to split a line into several.

Global section

This section currently supports only one line: cpu-map. It has the same meaning as cpu-map setting for HAProxy and is used to configure the CPU affinity of the stick-table aggregator process(es). It is made of one or several CPU sets, each CPU set being a CPU number, or a range of CPU numbers. These CPU sets must be separated by a comma character (,).

# Select 0 and from 5 up to 8 CPUs.
cpu-map 0,5-8

Aggregations section

An aggregations section must follow the rules below:

• The aggregations keyword (starting the section) must be followed by an identifier (e.g. aggregations example_com). This name is an internal identifier and is only used for identification and referencing inside this configuration file.

• Supports only two lines keywords:

  • peer

  • from

• Has one or more peer lines (HAProxy servers it will listen to stick-table activity from)

• Has one from line (telling it what the name of the new aggregated table is)

peer <name> <ip:port> [local | up | down] [group group_id]

global
   cpu-map 2-3

aggregations agg_1
   peer hostA        192.168.0.200:10000
   peer hostB        192.168.0.201:10001
   peer stktagg_1    192.168.0.1:9000   local #IP of the host running this configuration file
   from any to .aggr

...
 from .cpu,.proc to .aggr

peers aggr
peer haproxy1 192.168.1.10:10000
peer stkaggr1 192.168.1.20:10000

frontend web
bind *:80
http-request track-sc0 src table src_table
default_backend web

backend src_table
stick-table type ip size 1m expire 10m store conn_rate(10s),http_req_cnt,http_err_cnt,http_req_rate(10s),http_err_rate(10s) peers aggr

backend src_table.aggr
stick-table type ip size 1m expire 10m store conn_rate(10s),http_req_cnt,http_err_cnt,http_req_rate(10s),http_err_rate(10s) peers aggr

global
cpu-map 1
source 192.168.1.20:0
stats socket 127.0.0.1:4545

aggregations agg_example # HAProxy peers
peer haproxy1 192.168.1.10:10000
peer haproxy2 192.168.1.11:10000
peer haproxy3 192.168.1.12:10000

Multi-level stick-table aggregators

Stick-table aggregators may be cascaded and organized hierarchically in a tree. This can be useful to aggregate stick-tables located in different places (datacenters).

Two rules apply when forwarding aggregation results between aggregator peers:

• A peer aggregator that receives updates from downward peers (aggregators or not) forwards the aggregation results to all its peers without any exception, even to the peer that sent the updates.

• A peer aggregator which receives updates from upward aggregators aggregates them and forwards the results to its downward peers.

For example with such an organization of four A, B, C, and D aggregators:

                A                     Level 0:   for .aggr to .cluster
              _/|\_
            _/  |  \_
          _/    |    \_
        _/      |     \_
      _/        |      \_
     /          |        \_
    B           C          D          Level 1:   for any to .aggr
   / \         / \        / \
  /   \       /   \      /   \
hB.1  hB.2  hC.1  hC.2  hD.1 hD.2     Level 2:   (HAProxy peers)

• Any xxx stick-table hosted by hB.1 & 2 (resp. hC.1 & 2 and hD.1 & 2) HAProxy peers would be aggregated by B (resp. C and D) with xxx.aggr as the aggregated stick-table.

• At level 0 in the tree, peer A aggregator is seen as another peer from level 1 aggregators; hence it would receive the aggregation results of level 2 downward HAProxy peer xxx stick-tables, presented as xxx.aggr stick-table entries.

• These aggregations results would also be aggregated again to stick-table xxx.cluster and redistributed to level 1 B, C and D peers using the setting from .aggr to .cluster.

• Consequently, all level 2 peers would receive their own copy of xxx.cluster stick-table contents which would be the aggregation results of B, C and D xxx.aggr stick-tables.

Upward aggregator peers

There is another multilevel feature which can add a redundancy service to the aggregation services provided by aggregators. You can group upward aggregator peers to be viewed as a unique upward aggregator peer from its downward aggregators peers using the line setting group on the peer line.

This configuration looks like this:

       AGGR.1   AGGR.2            Level 0:   from .aggr to .cluster
          / \   / \
         /   \ /   \
        /     X     \
       /    _/ \_    \
      /   _/     \_   \
     /___/         \___\
    A                   B         Level 1:   from any to .aggr
   / \                 / \
  /   \               /   \
hA.1  hA.2          hB.1  hB.2    Level 2:   (HAProxy peers)

• In this double tree organization of stick-table aggregators, if you remove AGGR.1, the remaining tree is similar to the previous tree, but with only two branches instead of three.

• Similarly, if you remove AGGR.2 from this tree, you also get two branches instead of 3.

• Hence, AGGR.1 & 2 aggregators offer the same aggregations as the previous A aggregator: they aggregate xxx.aggr stick-tables of A and B, respectively, which are themselves aggregations results of hA.1 & 2 and hB.1 & 2 xxx stick-tables, respectively.

• The double link between the level 1 aggregator and their two upward level 0 aggregator peers offers a redundancy service. If one of the level 0 aggregators fails, the remaining one continue its aggregation services.

• The group peer line setting takes only one string as argument: a group identifier.