Advanced features

Use Consul service discovery

This section will guide you through integrating HAProxy Data Plane API and HashiCorp Consul. We will pull information from the Consul service discovery registry through the HAProxy Data Plane API and use it to update the load balancer’s configuration.

You can:

  • query Consul for the IP addresses of your internal services
  • fill these addresses into the load balancer configuration in real time

Overview Jump to heading

For this setup, you will need three servers:

  • a Consul server to host the service discovery registry
  • an HAProxy, HAProxy Enterprise, or HAProxy ALOHA server with the Data Plane API enabled
  • a web server to serve as an example application to load balance

Consul service discovery diagram

Create the Consul server Jump to heading

To create the Consul server:

  1. On your Consul server machine, install Consul by following the official deployment guide.

  2. Configure it as a Consul server by creating the file /etc/consul.d/server.hcl.

    bash
    sudo touch /etc/consul.d/server.hcl
    sudo chown --recursive consul:consul /etc/consul.d
    sudo chmod 640 /etc/consul.d/server.hcl
    bash
    sudo touch /etc/consul.d/server.hcl
    sudo chown --recursive consul:consul /etc/consul.d
    sudo chmod 640 /etc/consul.d/server.hcl
  3. Add the following contents to the file. Note especially that server is true and bootstrap_expect is 1, since we are deploying a single-server cluster. Change bind_addr and client_addr to be your server’s IP address.

    server.hcl
    hcl
    datacenter = "dc1"
    server = true
    bootstrap_expect = 1
    bootstrap = true
    data_dir = "/opt/consul"
    # your Consul server's IP address
    bind_addr = "192.168.56.21"
    client_addr = "192.168.56.21"
    ui_config {
    enabled = true
    }
    log_level = "INFO"
    server.hcl
    hcl
    datacenter = "dc1"
    server = true
    bootstrap_expect = 1
    bootstrap = true
    data_dir = "/opt/consul"
    # your Consul server's IP address
    bind_addr = "192.168.56.21"
    client_addr = "192.168.56.21"
    ui_config {
    enabled = true
    }
    log_level = "INFO"
  4. Enable and start the Consul service.

    bash
    sudo systemctl enable consul
    sudo systemctl start consul
    bash
    sudo systemctl enable consul
    sudo systemctl start consul
  5. Verify that the Consul server is up and running by opening the Consul Web UI, which listens at port 8500, in a browser. It may take some time for the service to become available.

    Consul server UI

Add a service to the Consul registry Jump to heading

To add a service to Consul so that the HAProxy Data Plane API automatically adds it to the load balancer configuration:

  1. On your web application server, install Consul by following the official deployment guide.

  2. Configure it as a Consul agent by creating the file /etc/consul.d/consul.hcl.

    bash
    sudo touch /etc/consul.d/consul.hcl
    sudo chown --recursive consul:consul /etc/consul.d
    sudo chmod 640 /etc/consul.d/consul.hcl
    bash
    sudo touch /etc/consul.d/consul.hcl
    sudo chown --recursive consul:consul /etc/consul.d
    sudo chmod 640 /etc/consul.d/consul.hcl
  3. Add the following contents to the file:

    consul.hcl
    hcl
    datacenter = "dc1"
    server = false
    data_dir = "/opt/consul/"
    bind_addr = "192.168.56.22"
    retry_join = ["192.168.56.21"]
    enable_syslog = true
    log_level = "INFO"
    consul.hcl
    hcl
    datacenter = "dc1"
    server = false
    data_dir = "/opt/consul/"
    bind_addr = "192.168.56.22"
    retry_join = ["192.168.56.21"]
    enable_syslog = true
    log_level = "INFO"
    • Set bind_addr to this web server’s IP address.
    • Set retry_join to the Consul server’s IP address. The agent uses this to join the Consul cluster.
  4. Enable and start the Consul service.

    bash
    sudo systemctl enable consul
    sudo systemctl start consul
    bash
    sudo systemctl enable consul
    sudo systemctl start consul
  5. Assuming that you have a web application running at port 80, you can register it with the local Consul agent by creating a JSON manifest. For example:

    example.json
    json
    {
    "service": {
    "name": "example",
    "port": 80
    }
    }
    example.json
    json
    {
    "service": {
    "name": "example",
    "port": 80
    }
    }

    In this example, we set name to example. This is the name that will appear in the Consul registry. Also, it will appear in the name of the backend in the load balancer configuration.

  6. Register the service with Consul:

    bash
    consul services register example.json
    bash
    consul services register example.json

    The example service should appear on the Services tab in the Consul Web UI.

Configure the HAProxy Data Plane API Jump to heading

To enable the Data Plane API to pull service discovery information from the Consul server, follow these steps:

  1. On the load balancer, make a POST request to the Data Plane API’s /v2/services_discovery/consul endpoint. This tells the API to begin monitoring Consul’s service registry.

    bash
    curl -u admin:adminpwd \
    -H 'Content-Type: application/json' \
    -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
    }' http://127.0.0.1:5555/v2/service_discovery/consul
    bash
    curl -u admin:adminpwd \
    -H 'Content-Type: application/json' \
    -d '{
    "address": "192.168.56.21",
    "port": 8500,
    "enabled": true,
    "retry_timeout": 10
    }' http://127.0.0.1:5555/v2/service_discovery/consul
  2. When you add services to Consul, the Data Plane API will automatically create a backend in the configuration. In the example below, a backend named consul-backend-192.168.56.21-8500-example was created automatically from Consul’s service registry. It has a single registered server at the IP address 192.168.56.22:

    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    backend consul-backend-192.168.56.21-8500-example
    server SRV_V2u4y 192.168.56.22:80 check weight 128
    server SRV_1NQXA 127.0.0.1:80 disabled weight 128
    server SRV_DWcAP 127.0.0.1:80 disabled weight 128
    server SRV_IIcqg 127.0.0.1:80 disabled weight 128
    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    backend consul-backend-192.168.56.21-8500-example
    server SRV_V2u4y 192.168.56.22:80 check weight 128
    server SRV_1NQXA 127.0.0.1:80 disabled weight 128
    server SRV_DWcAP 127.0.0.1:80 disabled weight 128
    server SRV_IIcqg 127.0.0.1:80 disabled weight 128

    The Data Plane API has also added several disabled server lines. When you register more instances of the same service, the Data Plane API fills in disabled server slots. You can thus scale up or down without a reload, in most cases.

  3. Add a frontend section that routes traffic to this backend pool of servers. For example:

    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    frontend example
    bind :80
    default_backend consul-backend-192.168.56.21-8500-example
    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    frontend example
    bind :80
    default_backend consul-backend-192.168.56.21-8500-example

Filter which services are load balanced Jump to heading

You can control which services from Consul’s service registry get added to the load balancer’s configuration.

Set the service_allowlist and service_denylist fields when configuring service discovery. They each take an array of service names to restrict which services to allow or deny from load balancing.

The example below adds the app1 and app2 services to the load balancer configuration, but not app3 and app4. We make a PUT request to update the Consul server we registered before:

bash
curl -X PUT \
-u admin:adminpwd \
-H 'Content-Type: application/json' \
-d '{
"address": "192.168.56.21",
"port": 8500,
"enabled": true,
"retry_timeout": 10,
"service_allowlist": ["app1", "app2"],
"service_denylist": ["app3", "app4"]
}' http://127.0.0.1:5555/v2/service_discovery/consul
bash
curl -X PUT \
-u admin:adminpwd \
-H 'Content-Type: application/json' \
-d '{
"address": "192.168.56.21",
"port": 8500,
"enabled": true,
"retry_timeout": 10,
"service_allowlist": ["app1", "app2"],
"service_denylist": ["app3", "app4"]
}' http://127.0.0.1:5555/v2/service_discovery/consul

If this page was useful, please, Leave the feedback.