HAProxy Enterprise Documentation 2.1r1

Load Shedding with NS1

When you run HAProxy Enterprise in more than one availability zone, you may direct traffic to the geographically closest load balancer instance. For example, you may use Anycast network routing to send clients to the nearest instance to them. But where should a client be sent when the closest load balancer is experiencing high latency or is down?

Load shedding is a mechanism that allows you to send clients to a more distant load balancer if the closest one is too busy. Here you will learn how to configure load shedding at the DNS layer. You will use the NS1 DNS service to monitor your load balancers and shed load to an alternate load balancer when the closest instance becomes too busy or is down.

Prerequisites

  • An NS1 account from https://ns1.com/

  • A domain name to add DNS records for

  • Two or more geographically dispersed instances of HAProxy Enterprise

Configure the NS1 records

When there are several IP addresses (i.e answer) for a given zone, NS1 must choose the best one based on metadata associated with each answer. Set up metadata to inform this decision.

  1. In the NS1 Customer Portal, set up a DNS zone (e.g. www.foo.com) and create A records within it for each of your load balancers.

  2. For each answer, click Edit Answer Metadata to display a settings window.

    [Edit the answer metadata]
  3. Set the appropriate Geographic region (e.g. "US-EAST").

    [Set the geographic region]
  4. Check that you associate each answer with a region.

    [Check the region added]
  5. Add metadata for the Up/down status and set it to "Up". This takes into consideration the status of the load balancer in addition to its geographic location.

Configure data feeds for HAProxy

Use the NS1 API to create data feeds where HAProxy Enterprise can push data.

  1. Go to the NS1 portal and generate an API key. This key needs the following permissions:

    1. push to datafeeds

    2. manage datasources

    3. manage datafeeds

  2. Create a new data source with curl to call the /v1/data/sources NS1 API endpoint. Make sure you set your API key for the X-NSONE-Key HTTP header:

    $ curl -sH 'X-NSONE-Key: <API_KEY>' \
        -X PUT 'https://api.nsone.net/v1/data/sources' \
        -d '{"sourcetype": "nsone_v1", "name": "HA_PROXY_CONNECT"}'
    
    {"status": "ok", "name": "HA_PROXY_CONNECT", "feeds": [], "config": {}, "id": "760e670096f4f59dec045bed383aac5c", "sourcetype": "nsone_v1"}
  3. Create a new data feed for each of your load balancers:

    1. Call the /v1/data/feeds/[id] NS1 API endpoint.

    2. Set id in the URL to the "id" returned from the previous step.

    In the following example, we set name and label to "us-east", but you can choose any value. NS1 uses this when it selects the data feed to monitor for the current number of connections:

    $ curl -sH 'X-NSONE-Key: <API_KEY>' \
         -X PUT 'https://api.nsone.net/v1/data/feeds/760e670096f4f59dec045bed383aac5c' \
         -d '{"name": "us-east", "config": {"label": "us-east"}, "destinations": []}'
  4. Repeat the previous step to create a data feed for each load balancer, but change the name and label for each (e.g. "us-east", "us-west").

  5. Verify that the data feeds are listed in the NS1 Customer Portal on the Integrations window.

Associate DNS answers with data feeds

Add metadata to associate your A record answers with the data feeds.

  1. In the NS1 Customer Portal, edit the A records for the zone.

  2. Click Edit Answer Metadata for each answer and select Active connections.

  3. Click the "Feed" NS1 Feed Icon button to toggle the source of the data to become a feed.

  4. Choose the appropriate data feed for the current answer.

    [Choose the data feed]
  5. Add record metadata that applies to all answers:

    1. Specify a Low watermark that sets the threshold for when NS1 can begin shedding traffic away from a load balancer.

    2. Specify a High watermark that sets the threshold for when NS1 must completely stop sending traffic to a load balancer.

    [Set watermarks]

Configure a filter chain

When NS1 decides which IP address to return for a DNS query, it bases its decision on a chain of filters. Each filter discards answers that fail to match some criteria. Set a filter chain to inform NS1 on how to make its decision.

  1. In the NS1 Customer Portal, edit your A records and click Edit Filter Chain.

  2. Add the following filters:

    • Up

    • Geotarget Regional

    • Shed Load

    • Select First N

  3. In the Active Filters window, select:

    • the Shed Load filter and choose connections from the drop-down list.

    • the Select First N filter and set its value to 1.

  4. Save the filter chain. Verify that the filters are listed in the correct order as shown in the image below.

    [Verify the filter chain]

Configure HAProxy Enterprise to send data

Use the HAProxy Enterprise Send Metrics module to send the count of active connections for each load balancer to NS1.

  1. Log into each load balancer server and install the Send Metrics module.

  2. Edit the HAProxy Enterprise configuration file to send connection information to NS1 by adding the following to the global section:

    global
        # ...
        module-path /opt/hapee-1.8/modules
        module-load hapee-lb-send-metrics.so
        send-metrics-url POST https://api.nsone.net/v1/feed/760e670096f4f59dec045bed383aac5c xdelay 1m 5s 1s 1s timeout 100ms retries 3 log verify none
        send-metrics-header 'X-NSONE-Key: <API_KEY>'
        send-metrics-content-type application/json
        send-metrics-data '{ "us-east": { "connections": "%ac" }}'
  3. Be sure to update the URL with your data source ID, the X-NSONE-Key header with your API key, and the label sent with the send-metrics-data line.

  4. Reload HAProxy Enterprise.

  5. Verify that the HAProxy Enterprise access logs show that the Send Metrics module sent data successfully:

    Send Metrics: metrics data successfully updated (1/0).
  6. Repeat this procedure for each load balancer.


Next up

Active/Standby Clustering