HAProxy Enterprise Documentation 1.8r2

Single-level setup

As an example, we will consider the following setup:

https://cdn.haproxy.com/documentation/hapee/1-8r2/assets/cluster-wide-tracking-single-level-setup-c11861ecfca68578d69b7317d67dc1d5a6c8cfb251640d02f95b6a224b97ca7a.png

The Stick Table Aggregator adds up the counters from HAPRoxy Enterprise nodes' uncombined stick tables and sends an aggregate stick table to all nodes.

Configure the Stick Table Aggregator

We will erase the content of the default /etc/hapee-extras/hapee-stktagg.cfg Stick Table Aggregator configuration file on server aggr1, then implement the following sandbox configuration:

global
  stats socket /tmp/stktagg.socket

aggregations myaggr
  peer hapee1 192.168.56.101:44444
  peer hapee2 192.168.56.102:44444
  peer aggr1  192.168.56.111:11111 local
  from .uncombined to .aggr

The Stick Table Aggregator configuration file is very similar to traditional HAProxy Enterprise configuration files.

We will populate the file from top to bottom.

  1. Enable the Stick Table Aggregator CLI in the global section.

    The stats socket directive enables a CLI that lets you view data that the aggregator has stored.

    Access the API locally

    global
      stats socket /tmp/stktagg.socket

    Access the API remotely

    global
      stats socket ipv4@192.168.56.111:9999
  2. Create an aggregations section with the name of your choice, and make the aggr1 Stick Table Aggregator listen to HAProxy Enterprise nodes hapee1 and hapee2.

    aggregations myaggr
      peer hapee1 192.168.56.101:44444
      peer hapee2 192.168.56.102:44444
  3. Make the Stick Table Aggregator listen to incoming remote peer connections through the local keyword.

    aggregations myaggr
      peer hapee1 192.168.56.101:44444
      peer hapee2 192.168.56.102:44444
      peer aggr1  192.168.56.111:11111 local
  4. Define the stick table mapping.

    The Stick Table Aggregator works by combining the data from normal stick tables into aggregated stick tables.

    In your HAProxy Enterprise configuration, you will store data in the normal tables, as usual, using http-request track-sc0, but base ACL logic, such as whether to deny a request, on the aggregated tables.

    You'll use suffixes on your stick table names to declare which tables are uncombined vs aggregated. For example, you might add the suffix .uncombined to your normal stick tables and .aggr to the aggregated ones.

    Integration with the HAProxy Enterprise Real-time Dashboard

    You can analyze aggregate data in your favorite dashboard. If you use the HAProxy Enterprise Real-Time Dashboard, consider using the .aggr suffix. It is the default suffix the dashboard uses to reference aggregate stick tables.

    There are no mandatory names, but the labels you choose must match the from line in your Stick Table Aggregator configuration file.

    The table below describes the ways you can set the from line and how you would name your uncombined and aggregated tables:

    From

    To

    from directive

    <name>.uncombined

    <name>.aggr

    from .uncombined to .aggr

    <name>.<any suffix>

    <name>.aggr

    from any to .aggr

    <name>.<suffix from a list>

    <name>.aggr

    from .cpu,.proc to .aggr

    <name>.uncombined and <name>

    <name>.aggr

    from .uncombined to .aggr accept-no-suffix
  5. Start the Stick Table Aggregator service with systemctl:

    $ sudo systemctl start hapee-extras-stktagg

Configure the HAProxy Enterprise nodes

We will erase the content of the default /etc/hapee-1.8/hapee-lb.cfg HAProxy Enterprise configuration files on servers hapee1 and hapee2, then implement the sandbox configurations below.

We will populate the file from top to bottom.

HAProxy Enterprise node hapee1

global
  stats socket /var/run/hapee-1.8/hapee-lb.sock

defaults
  mode http

# Demo frontend that returns HTTP 200 OK responses
  frontend fe_main
  bind *:80
  http-request track-sc0 src table mypeers/mytable.uncombined
  http-request deny deny_status 200

peers mypeers
  server hapee1
  server aggr1 192.168.56.111:11111
  table mytable.uncombined type ip size 100 expire 1h store http_req_rate(1h)
  table mytable.aggr       type ip size 100 expire 1h store http_req_rate(1h)

HAProxy Enterprise node hapee2

global
  stats socket /var/run/hapee-1.8/hapee-lb.sock

defaults
  mode http

# Demo frontend that returns HTTP 200 OK responses
frontend fe_main
  bind *:80
  http-request track-sc0 src table mypeers/mytable.uncombined
  http-request deny deny_status 200

peers mypeers
  server hapee2
  server aggr1 192.168.56.111:11111
  table mytable.uncombined type ip size 100 expire 1h store http_req_rate(1h)
  table mytable.aggr       type ip size 100 expire 1h store http_req_rate(1h)

The HAProxy Enterprise nodes' configuration files are the same, except for one server directive in the peers section.

  1. Enable the HAProxy Enterprise Runtime API.

    You can use the Runtime API to display the stick tables metrics, locally through a Unix domain socket, or remotely through an IP address and port.

    Access the API locally

    global
      stats socket /var/run/hapee-1.8/hapee-lb.sock

    Access the API remotely

    global
      stats socket ipv4@192.168.56.101:9999 level admin
  2. Make the HAProxy Enterprise node operate at layer 4 or 7.

    In our example, we will track the HTTP request rate, so we make HAProxy Enterprise act as a layer 7 proxy.

    defaults
      mode http
  3. Declare the current HAProxy Enterprise node and the Stick Table Aggregator in the peers section.

    In an active-active cluster, no HAProxy Enterprise node should be aware of other nodes, to avoid peer nodes overwriting each other's stick tables.

  4. Define a stick table counter in a frontend section.

    The new http-request directive states that we want to store the sc0 sticky counter in the stick table we specify with the table parameter.

    We specify the mypeers/mytable.uncombined stick table, i.e, in the mytable.uncombined stick table defined in the mypeers peers section.

    # Demo frontend that returns HTTP 200 OK responses
    frontend fe_main
      bind *:80
      http-request track-sc0 src table mypeers/mytable.uncombined
      http-request deny deny_status 200

    The bind directive assigns a listener to all IP addresses on the server on port 80. We'll use the directive's ssl and crt arguments later on to encrypt traffic between HAProxy Enterprise nodes and the aggregator.

    In this example, the stick table will track IP addresses and the corresponding HTTP request rates over the last hour.

    The last line of the snippet is meant just for testing: we deny access but return status 200 OK.

  5. Declare stick tables mytable.uncombined and mytable.aggr in the peers section on all HAProxy Enterprise nodes:

    HAProxy Enterprise node

    hapee1

    peers mypeers
      server hapee1
      server aggr1 192.168.56.111:11111
      table  mytable.uncombined type ip size 100 expire 1h store http_req_rate(1h)
      table  mytable.aggr       type ip size 100 expire 1h store http_req_rate(1h)

    HAProxy Enterprise node

    hapee2

    peers mypeers
      server hapee2
      server aggr1 192.168.56.111:11111
      table  mytable.uncombined type ip size 100 expire 1h store http_req_rate(1h)
      table  mytable.aggr       type ip size 100 expire 1h store http_req_rate(1h)

Note

You can also define stick tables in backend sections. This method will be deprecated soon, however.

global
  stats socket /var/run/hapee-1.8/hapee-lb.sock

defaults
  mode http

frontend mytable.uncombined
  bind *:80
  stick-table  type ip size 100 expire 1h store http_req_rate(1h) peers mypeers
  http-request track-sc0 src
  http-request deny deny_status 200

backend mytable.aggr
  stick-table  type ip size 100 expire 1h store http_req_rate(1h) peers mypeers

peers mypeers
  server hapee1
  server aggr1 192.168.56.111:11111

Next up

Multi-level setup