The HAProxy Data Plane API is a program that runs alongside HAProxy to enable you to configure fully the HAProxy load balancer at runtime.

This feature benefits a number of use cases that are suited for a dynamically-generated configuration.

The best example is using HAProxy as the data plane of a service mesh in a microservices architecture, where a central control plane manages the configuration of a set of sidecar proxies.

This section shows you how to install and configure the Data Plane API and get started with a few examples.

See also the API specification documentation for more details.

Prerequisites

  • HAProxy Community 1.9 or newer (2.0 if using Process Manager)

  • HAProxy Enterprise 1.9 or newer

  • Working knowledge of HTTP and using REST APIs

Installing the Data Plane API

Install API with HAProxy Enterprise

The Data Plane API comes as a system package which you can install with apt or yum.

The following is an example of an installation on an Ubuntu 18.04 server.

  1. Store your HAProxy Enterprise key in a variable named haproxy_key and add the HAProxy apt repository key:

    $ haproxy_key=[YOUR KEY]
    $ curl -fsSL https://www.haproxy.com/download/hapee/key/$haproxy_key-common/HAPEE-key-1.9r1.asc | sudo apt-key add -
  2. Run the add-apt-repository command to include the following package repositories:

    $ sudo add-apt-repository "deb [arch=amd64] https://www.haproxy.com/download/hapee/key/${haproxy_key}-common/1.9r1/ubuntu-18.04/amd64/ bionic main"
    $ sudo add-apt-repository "deb [arch=amd64] https://www.haproxy.com/download/hapee/key/${haproxy_key}-plus/1.9r1/ubuntu-18.04/amd64/ bionic main"
    $ sudo add-apt-repository "deb [arch=amd64] https://www.haproxy.com/download/hapee/key/${haproxy_key}-plus/extras/ubuntu-18.04/amd64/ bionic main"
  3. Install the Data Plane API package, as follows:

    $ sudo apt install hapee-extras-dataplane-api

    This adds the program here: /opt/hapee-extras/sbin/hapee-dataplane-api.

Install API with HAProxy Community

The Data Plane API comes as a program that you can download from Github.

  • After the download, set its permissions to run as an executable and copy it to a folder on your PATH:

    $ chmod +x dataplaneapi
    $ sudo cp dataplaneapi /usr/local/bin/

Configure user access to HAProxy

The Data Plane API requires you to enable basic authentication, which means that any user invoking its methods must provide valid credentials.

Usernames and passwords are stored in the HAProxy configuration file inside a userlist section. A userlist is a top-level section like frontend and backend.

  1. Add the following to the HAProxy configuration file /etc/hapee-1.9/hapee-lb.cfg:

    userlist dataplane-api
        user dataplaneapi insecure-password mypassword

    In this example, we added a single user named dataplaneapi with the password mypassword.

  2. You can also encrypt the password with the mkpasswd command from the whois package.

    $ sudo apt install -y whois
    $ mkpasswd -m sha-256 mypassword
    $5$aVnIFECJ$2QYP64eTTXZ1grSjwwdoQxK/AP8kcOflEO1Q5fc.5aA
  3. Copy and paste the encrypted password into your configuration file:

    userlist dataplane-api
        user dataplaneapi password $5$aVnIFECJ$2QYP64eTTXZ1grSjwwdoQxK/AP8kcOflEO1Q5fc.5aA

    You do not need to reload HAProxy for the userlist to take effect.

    However, if the Data Plane API is running, stop it and then restart it.

    The API runs in its own process, which allows it to write to and reload the HAProxy configuration as needed.

Configure access to the HAProxy socket

The Data Plane API needs access to read and write to the HAProxy socket.

To do this, update your HAProxy configuration file so that it has a stats socket line in the global section:

  1. Set the user and group parameters to the system user who will run the API. For example, with HAProxy Community, you might set both to haproxy. These should be set to hapee-lb and hapee, respectively, for HAProxy Enterprise, as shown:

    global
        stats socket /run/haproxy.sock user hapee-lb group hapee mode 660 level admin
  2. If you test the API by running it yourself, you can add your own username to the same group:

    $ sudo usermod -a -G hapee myusername

    Note

    If you do this, make sure that you log out and back in again for the permissions to take effect.

Running the API

View available options

  • Invoke the API with the --help flag to see a list of available options:

    Usage:
        dataplaneapi [OPTIONS]
    
    API for editing and managing HAProxy instances
    
    Application Options:
        --scheme=                            the listeners to enable, this can be repeated and defaults to the schemes in the swagger spec
        --cleanup-timeout=                   grace period for which to wait before killing idle connections (default: 10s)
        --graceful-timeout=                  grace period for which to wait before shutting down the server (default: 15s)
        --max-header-size=                   controls the maximum number of bytes the server will read parsing the request header's keys and values, including the request line. It does not limit the size of the request body. (default: 1MiB)
        --socket-path=                       the unix socket to listen on (default: /var/run/data-plane.sock)
        --host=                              the IP to listen on (default: localhost) [$HOST]
        --port=                              the port to listen on for insecure connections, defaults to a random value [$PORT]
        --listen-limit=                      limit the number of outstanding requests
        --keep-alive=                        sets the TCP keep-alive timeouts on accepted connections. It prunes dead TCP connections ( e.g. closing laptop mid-download) (default: 3m)
        --read-timeout=                      maximum duration before timing out read of the request (default: 30s)
        --write-timeout=                     maximum duration before timing out write of the response (default: 60s)
        --tls-host=                          the IP to listen on for tls, when not specified it's the same as --host [$TLS_HOST]
        --tls-port=                          the port to listen on for secure connections, defaults to a random value [$TLS_PORT]
        --tls-certificate=                   the certificate to use for secure connections [$TLS_CERTIFICATE]
        --tls-key=                           the private key to use for secure connections [$TLS_PRIVATE_KEY]
        --tls-ca=                            the certificate authority file to be used with mutual tls auth [$TLS_CA_CERTIFICATE]
        --tls-listen-limit=                  limit the number of outstanding requests
        --tls-keep-alive=                    sets the TCP keep-alive timeouts on accepted connections. It prunes dead TCP connections ( e.g. closing laptop mid-download)
        --tls-read-timeout=                  maximum duration before timing out read of the request
        --tls-write-timeout=                 maximum duration before timing out write of the response
    
    HAProxy options:
        -c, --config-file=                   Path to the haproxy configuration file (default: /etc/haproxy/haproxy.cfg)
        -u, --userlist=                      Userlist in HAProxy configuration to use for API Basic Authentication (default: controller)
        -b, --haproxy-bin=                   Path to the haproxy binary file (default: haproxy)
        -d, --reload-delay=                  Minimum delay between two reloads (in s) (default: 5)
        -r, --reload-cmd=                    Reload command
            --reload-retention=              Reload retention in days, every older reload id will be deleted (default: 1)
        -t, --transaction-dir=               Path to the transaction directory (default: /tmp/haproxy)
        -m, --master-runtime=                Path to the master Runtime API socket
    
    Logging options:
        --log-to=[stdout|file]               Log target, can be stdout or file (default: stdout)
        --log-file=                          Location of the log file (default: /var/log/dataplaneapi/dataplaneapi.log)
        --log-level=
        [trace|debug|info|warning|error]     Logging level (default: warning)
        --log-format=[text|JSON]             Logging format (default: text)
    
    Show version:
        -v, --version                        Version and build information
    
    Help Options:
        -h, --help                           Show this help message

Start the Data Plane API with HAProxy Enterprise

  • Run:

    $ /opt/hapee-extras/sbin/hapee-dataplane-api \
    --host 0.0.0.0 \
    --port 5555 \
    --haproxy-bin /opt/hapee-1.9/sbin/hapee-lb \
    --config-file /etc/hapee-1.9/hapee-lb.cfg \
    --reload-cmd "systemctl reload hapee-1.9-lb" \
    --reload-delay 5 \
    --userlist dataplane-api

Start the Data Plane API with HAProxy Community

  • Run:

    $ dataplaneapi \
    --host 0.0.0.0 \
    --port 5555 \
    --haproxy-bin $(which haproxy) \
    --config-file /etc/haproxy/haproxy.cfg \
    --reload-cmd "systemctl reload haproxy" \
    --reload-delay 5 \
    --userlist dataplane-api

Display the API in a browser

  • With the API running, open a browser and access the OpenAPI specification on port 5555, such as http://localhost:5555/v1/specification.

    • This returns a JSON document that defines all available API methods.

    • In Firefox, you get a human-readable version with collapsible sections. The paths node contains the URLs that map to each method.

      [Firefox displays the specification file in a friendly way]

Test methods

You can use the curl command to test calling various methods.

  • Call the info method, which returns process information:

    $ curl -X GET --user dataplaneapi:mypassword http://localhost:5555/v1/services/haproxy/info
    
    {"haproxy":{"processes":1,"release_date":"2019-04-02","time":"2019-04-08T22:07:39.291Z","uptime":1006,"version":"1.9.1-1.0.0-200.372"}}

    If you get this error:

    {"code":500,"message":"dial unix /run/haproxy.sock: connect: permission denied"}

    This means that the user who runs the API does not have access to the HAProxy socket. Check that you added it to the HAProxy group and log out and back in again.

Note

The objective of the Data Plane API is to manage the HAProxy configuration file completely. So, whenever you edit the configuration manually, make sure that you restart the API or send it a SIGUSR2 signal so that it can read the file again. See below for examples on how to use the commands.

Using the HAProxy Process Manager

When using HAProxy 2.0 or later, you can use the HAProxy Process Manager to start the Data Plane API. The Process Manager adds a new section called program to the HAProxy configuration to start external processes when HAProxy starts.

Configure HAProxy to start the API

The following configuration shows how to tell HAProxy to start the Data Plane API whenever its own master process is started or reloaded:

  1. Update your HAProxy configuration:

    program api
        command dataplaneapi --host 0.0.0.0 --port 5555 --haproxy-bin /usr/sbin/haproxy --config-file /etc/haproxy/haproxy.cfg --reload-cmd "systemctl reload haproxy" --reload-delay 5 --userlist controller
  2. For this to work, you must run HAProxy in master-worker mode by adding the master-worker directive to your global section (or adding the -W command-line argument). Then when you view the status of HAProxy, you'll see the new program running alongside the HAProxy worker processes.

    $ sudo systemctl restart haproxy
    $ sudo systemctl status haproxy
    
    Main PID: 1274 (haproxy)
        Tasks: 6
    Memory: 5.5M
        CPU: 2.838s
    CGroup: /system.slice/haproxy.service
        1274 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -sf 2768 -x /run/haproxy/admin.sock
        2768 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -sf 2662 -x /run/haproxy/admin.sock
        2830 /opt/hapee-extras/sbin/hapee-dataplane-api --host 0.0.0.0 --port 5555 -b /usr/local/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r systemctl reload haproxy -u controller
        2831 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -sf 2768 -x /run/haproxy/admin.sock
  1. If HAProxy runs inside a Docker container in master-worker mode, you can use the kill -SIGUSR2 [PID] command to reload only the worker processes. In this case, with the Data Plane API under the Process Manager's control and running as a worker process, it will restart as well. To avoid this, use the directive no option start-on-reload to tell the Process Manager not to restart the API during a reload of the HAProxy master process.

    program api
        command /etc/haproxy/dataplaneapi --host 0.0.0.0 --port 5555 --haproxy-bin /usr/sbin/haproxy --config-file /etc/haproxy/haproxy.cfg --reload-cmd "kill -SIGUSR2 1" --reload-delay 5 --userlist controller
        no option start-on-reload

Using the Data Plane API

In this section, we show you the commands that you can use with the Data Plane API.

GET requests

When fetching data with GET requests, you don't need any additional URL parameters. For example, to get a list of frontends in your configuration, use:

$ curl -X GET --user admin:mypassword http://localhost:5555/v1/services/haproxy/configuration/frontends

{"_version":1,"data":[{"default_backend":"be_web","name":"fe_main"}]}

This returns a JSON document with two top-level keys: _version and data.

  • The version figures at the top of the HAProxy configuration file and increments whenever you write data via a POST, PUT or DELETE request. It ensures that changes to the file don't conflict.

  • The data key contains the result of the command, which, in this case, is an array of objects describing each frontend.

POST requests

When you write data, such as with a POST request, you must include the version in the URL. If it doesn't match the current version, you get an error about a version mismatch.

Below is an example of a POST request to add a new frontend:

Tip

It's a good idea to enclose the URL within quotes so that the shell does not interpret any URL parameter prefixed with an ampersand as a request to run the command in the background.

$ curl -X POST --user admin:mypassword \
  -H "Content-Type: application/json" \
  -d '{"name": "test_frontend", "default_backend": "be_web", "mode": "http", "maxconn": 2000}' \
  "http://localhost:5555/v1/services/haproxy/configuration/frontends?version=1"

Use transactions

When you run the query above, you get the following problems:

  • If you didn't define be_web, you get an error stating that the API can't find the backend.

  • The API creates a frontend without a bind line.

To fix these problems, you can execute the following commands only once in the following sequence:

  1. Create the backend be_web

  2. Add servers to the backend

  3. Create the frontend

  4. Add a bind line to the frontend

Each of these operations requires a separate command; and until you've executed them all, you continue to reload your HAProxy configuration with valid, but incomplete, data. The solution is to use a transaction that stores the commands and then applies them all at once.

  1. Initialize a transaction:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    http://localhost:5555/v1/services/haproxy/transactions?version=5

    This returns:

    {"_version":5,"id":"b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd","status":"in_progress"}

    Then, each command running within that transaction includes the transaction_id parameter in the URL. To view all transactions, use the following command:

    $ curl -X GET --user admin:mypassword \
    -H "Content-Type: application/json" \
    "http://localhost:5555/v1/services/haproxy/transactions"
  2. Add a backend:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"name": "test_backend", "mode":"http", "balance": {"algorithm":"roundrobin"}, "httpchk": {"method": "HEAD", "uri": "/check", "version": "HTTP/1.1"}}' \
    "http://localhost:5555/v1/services/haproxy/configuration/backends?transaction_id=b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd"
  3. Add a server to the backend:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"name": "server1", "address": "127.0.0.1", "port": 8080, "check": "enabled", "maxconn": 30, "weight": 100}' \
    "http://localhost:5555/v1/services/haproxy/configuration/servers?backend=test_backend&transaction_id=b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd"
  4. Add the frontend:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"name": "test_frontend", "mode": "http", "default_backend": "test_backend", "maxconn": 2000}' \
    "http://localhost:5555/v1/services/haproxy/configuration/frontends?transaction_id=b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd"
  5. Add a bind line to the frontend:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"name": "http", "address": "*", "port": 80}' \
    "http://localhost:5555/v1/services/haproxy/configuration/binds?frontend=test_frontend&transaction_id=b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd"
  6. After the transaction is complete, apply the changes:

    $ curl -X PUT --user admin:mypassword \
    -H "Content-Type: application/json" \
    "http://localhost:5555/v1/services/haproxy/transactions/b9a0ecd7-b8ae-4ef8-8865-0cd5e38396cd"

Monitor Configuration Reloads

The Data Plane API is started with the --reload-delay or -d flag to set the time interval between reloads. For example, if you set the delay to be five seconds, then every five seconds the program checks whether there have been any changes that require a reload of HAProxy. If none are found, the API waits another five seconds and checks again. All changes that happen during the interval between checks wait for the next check before they are applied. In other words, a command does not trigger a reload. Reloads happen automatically at regular intervals as needed.

Use the /services/haproxy/reloads endpoint to get a list of in-progress and failed reloads. This can be used to verify that the configuration was successfully updated.

It is possible to reload HAProxy immediately by passing the force_reload=true URL parameter when invoking a command.

Configuration example

The following example adds an ACL named is_api to a frontend named test_frontend.

  1. Add an ACL named is_api:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"id": 0, "acl_name": "is_api", "criterion": "path_beg", "value": "/api"}' \
    "http://localhost:5555/v1/services/haproxy/configuration/acls?parent_type=frontend&parent_name=test_frontend&version=4"

    Result:

    frontend test_frontend
        mode http
        maxconn 2000
        acl is_api path_beg /api
        default_backend test_backend
  2. Add a use_backend line that references the is_api ACL:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"id": 0, "cond": "if", "cond_test": "is_api", "name": "test_backend"}' \
    "http://localhost:5555/v1/services/haproxy/configuration/backend_switching_rules?frontend=test_frontend&version=5"

    Result:

    frontend test_frontend
        mode http
        maxconn 2000
        acl is_api path_beg /api
        use_backend test_backend if is_api
        default_backend test_backend
  3. Delete the use_backend line (note that we pass the id of 0 in the URL):

    $ curl -X DELETE --user admin:mypassword \
    -H "Content-Type: application/json" \
    "http://localhost:5555/v1/services/haproxy/configuration/backend_switching_rules/0?frontend=test_frontend&version=6"
  4. Add an inline ACL that denies all requests except those from localhost:

    $ curl -X POST --user admin:mypassword \
    -H "Content-Type: application/json" \
    -d '{"id": 0, "cond": "unless", "cond_test": "{ src 127.0.0.1 }", "type": "deny"}' \
    "http://localhost:5555/v1/services/haproxy/configuration/http_request_rules?parent_type=frontend&parent_name=test_frontend&version=7"

    The HAProxy configuration now looks like this:

    frontend test_frontend
        mode http
        maxconn 2000
        acl is_api path_beg /api
        http-request deny unless { src 127.0.0.1 }
        default_backend test_backend

Summary

This section introduced you to the HAProxy Data Plane API, which enables you to configure HAProxy by making HTTP requests to a process that runs externally to HAProxy.

Methods exist to create proxies, backends, servers, ACLs, stick table rules, and more.

See the API specification documentation for more information about the available commands.