HAProxy Enterprise Documentation 2.7r1

Multi-level setup

You can aggregate stick tables from other Stick Table Aggregators. This helps manage stick tables located in different locations, or AWS availability zones, for example.

As an example, we will consider the following setup:

https://cdn.haproxy.com/documentation/hapee/latest/assets/cluster-wide-tracking-multi-level-setup-1a888c49b593b8883fa211ddc216f14b03dfb0de036dd57fe414e2bbf16e06e1.png

The top-level aggr3 Stick Table Aggregator will sum the counters from the intermediate aggr1 and aggr2 aggregate stick tables. It will then send the top-level aggregate stick table to all HAProxy Enterprise nodes.

Configure the top-level Stick Table Aggregator

We will erase the content of the default /etc/hapee-extras/hapee-stktagg.cfg top-level Stick Table Aggregator configuration file, then implement the following sandbox configuration:

global
  stats socket /tmp/stktagg.socket

aggregations myl2
  peer aggr3     192.168.56.113:22222 local
  peer aggr1     192.168.56.111:11111 down
  peer aggr2     192.168.56.112:11111 down
  from .aggr to .l2
  1. Enable the Stick Table Aggregator CLI in the global section.

    The stats socket directive enables a CLI that lets you view data that the aggregator has stored.

    Access the API locally

    global
      stats socket /tmp/stktagg.socket

    Access the API remotely

    global
      stats socket ipv4@192.168.56.113:9999
  2. Create an aggregations section with the name of your choice, and make the top-level Stick Table Aggregator listen to incoming remote peer connections through the local keyword.

    aggregations myl2
      peer aggr3 192.168.56.113:22222 local
  3. Make the top-level Stick Table Aggregator listen to Stick Table Aggregators that are one level lower, denoted by the down keyword.

    aggregations myl2
      peer aggr3 192.168.56.113:22222 local
      peer aggr1 192.168.56.111:11111 down
      peer aggr2 192.168.56.112:11111 down
  4. Aggregate stick tables <name>.aggr to <name>.l2

    aggregations myl2
      peer aggr3 192.168.56.113:22222 local
      peer aggr1 192.168.56.111:11111 down
      peer aggr2 192.168.56.112:11111 down
      from .aggr to .l2

    The top-level Stick Table Aggregator will aggregate <name>.aggr stick tables from the intermediate Stick Table Aggregators to the <name>.l2 stick table.

Configure the intermediate Stick Table Aggregators

We will edit the /etc/hapee-extras/hapee-stktagg.cfg Stick Table Aggregator configuration file we created in the Single-level setup section. We will implement the following sandbox configuration:

global
  stats socket /tmp/stktagg.socket

aggregations myaggr
  peer aggr1  192.168.56.111:11111 local
  peer hapee1 192.168.56.101:44444
  peer hapee2 192.168.56.102:44444
  from .uncombined to .aggr
  peer aggr3  192.168.56.113:11111 up
  forward .l2

We will populate the file from top to bottom.

  1. Add a new peer directive for the top-level Stick Table Aggregator.

    The upper-level Stick Table Aggregator is denoted by the up keyword.

    Each Stick Table Aggregator is only aware of the HAProxy Enterprise nodes it manages and of the top-level Stick Table Aggregator.

    global
      stats socket /tmp/stktagg.socket
    
    aggregations myaggr
      peer aggr1  192.168.56.111:11111 local
      peer hapee1 192.168.56.101:44444
      peer hapee2 192.168.56.102:44444
      from .uncombined to .aggr
      peer aggr3  192.168.56.113:11111 up
  2. Make the intermediate Stick Table Aggregator forward the aggregate stick table from the top-level aggregator to HAProxy enterprise nodes through the forward directive.

    Intermediate Stick Table Aggregator aggr1

    global
      stats socket /tmp/stktagg.socket
    
    aggregations myaggr
      peer aggr1  192.168.56.111:11111 local
      peer hapee1 192.168.56.101:44444
      peer hapee2 192.168.56.102:44444
      from .uncombined to .aggr
      peer aggr3  192.168.56.113:11111 up
      forward .l2

    Intermediate Stick Table Aggregator aggr2

    global
      stats socket /tmp/stktagg.socket
    
    aggregations myaggr
      peer aggr2  192.168.56.112:11111 local
      peer hapee3 192.168.56.103:44444
      peer hapee4 192.168.56.104:44444
      from .uncombined to .aggr
      peer aggr3  192.168.56.113:11111 up
      forward .l2

The intermediate Stick Table Aggregators' configuration files are mostly identical, but:

  • The peer directives in the aggregations section are specific to each intermediate Stick Table Aggregator.

  • The peer directives containing the up keyword are the same.

Configure the HAProxy Enterprise nodes

We will edit the /etc/hapee-2.7/hapee-lb.cfg HAProxy Enterprise configuration files we created in the Single-level setup section. We will implement the following sandbox configuration:

HAProxy Enterprise node hapee1
global
  stats socket /var/run/hapee-2.7/hapee-lb.sock

defaults
  mode http

frontend mytable.uncombined
  bind *:80
  stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers
  http-request track-sc0 src
  http-request deny deny_status 200

backend mytable.aggr
  stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers

backend mytable.l2
  stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers

peers mypeers
  bind 0.0.0.0:44444
  server hapee1
  server aggr1 192.168.56.111:11111

On the HAProxy Enterprise nodes, the configuration files are the same, except for the server directive in the peers section. The local server name must match the name specified by the -L command line argument or the localpeer name specified in the global section (preferred methods) or the system hostname command (default method).

HAProxy Enterprise node hapee1

peers mypeers
  bind 0.0.0.0:44444
  server hapee1
  server aggr1 192.168.56.111:11111

HAProxy Enterprise node hapee2

peers mypeers
  bind 0.0.0.0:44444
  server hapee2
  server aggr1 192.168.56.111:11111

HAProxy Enterprise node hapee3

peers mypeers
  bind 0.0.0.0:44444
  server hapee3
  server aggr2 192.168.56.112:11111

HAProxy Enterprise node hapee4

peers mypeers
  bind 0.0.0.0:44444
  server hapee4
  server aggr2 192.168.56.112:11111
  1. Define the mytable.l2 stick table, either in a backend section or in the peers section.

    Stick table definitions in a backend section
    backend mytable.l2
      stick-table type ip size 100 expire 1h store http_req_rate(1h) peers mypeers
    Stick table definitions in peers section
    peers mypeers
      bind 0.0.0.0:44444
      server hapee1
      server aggr1 192.168.56.111:11111
      table mytable.uncombined type ip size 100 expire 1h store http_req_rate(1h)
      table mytable.aggr       type ip size 100 expire 1h store http_req_rate(1h)
      table mytable.l2         type ip size 100 expire 1h store http_req_rate(1h)

Next up

Highly-available setup