HAProxy Enterprise Documentation 2.1r1

Consul Service Discovery

This section will guide you through integrating HAProxy Enterprise and Consul.

You can:

  • query Consul for the IP addresses of your internal services, and

  • fill these addresses into HAProxy Enterprise's configuration in real time.

https://cdn.haproxy.com/documentation/hapee/2-1r1/assets/consul-service-discovery-diagram-5a7f370a519ab9c9c26cfca30362fd4ce0af18de7797333b925cee2e4561e648.png

We will pull configuration changes from the Consul service discovery registry through the HAProxy Enterprise Data Plane API.

See also

HashiCorp's Consul

Example setup

We will consider an example setup with 3 machines with the following characteristics.

Hostname

IP address

Role

hapee-lb

192.168.50.20

hapee-icon

HAProxy Enterprise load balancer

consul-server

192.168.50.21

consul-icon

Consul server

webserver

192.168.50.22

web-icon

Web server

Note

To implement the example setup in a virtual environment, you can run the following Vagrantfile:

Vagrant.configure("2") do |config|

    config.vm.define "hapee-lb" do |server|
      server.vm.box = "ubuntu/bionic64"
      server.vm.hostname = "hapee-lb"
      server.vm.network "private_network", ip: "192.168.50.20"
    end

    config.vm.define "consul-server" do |server|
      server.vm.box = "ubuntu/bionic64"
      server.vm.hostname = "consul-server"
      server.vm.network "private_network", ip: "192.168.50.21"
    end

    config.vm.define "webserver" do |server|
      server.vm.box = "ubuntu/bionic64"
      server.vm.hostname = "webserver"
      server.vm.network "private_network", ip: "192.168.50.22"
    end

end

See also

HashiCorp's Vagrant

Install the Consul Server

We will install a Consul server on a machine and run it as a Systemd service.

  1. consul-icon Download and install Consul on your Consul server machine.

    For example, on a Debian-based Linux distribution:

    $ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
    $ sudo apt-add-repository \
        "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
    $ sudo apt-get update && sudo apt-get install consul
  2. consul-icon Replace the content of the /etc/consul.d/consul.hcl configuration file with the following.

    datacenter = "dc1"
    server = true
    data_dir = "/opt/consul/"
    bind_addr = "192.168.50.21"
    client_addr = "192.168.50.21"
    bootstrap = true
    bootstrap_expect = 1
    
    ui_config {
      enabled = true
    }
    
    enable_syslog = true
    log_level = "INFO"

    Notable fields

    Value

    Description

    datacenter

    Consul datacenter name

    The private networking environment all Consul agents belong to.

    server

    true

    This Consul agent will run in server mode.

    bind_addr and client_addr

    Consul server machine's static IP address

    IP address Consul agents will connect to when they join the cluster.

  3. consul-icon Start Consul.

    $ sudo systemctl enable consul
    $ sudo systemctl start consul
  4. To check if the Consul server is up and running, open the Consul web UI at http://<Consul Server IP address>:8500 in a web browser.

    Check available nodes and services.

    https://cdn.haproxy.com/documentation/hapee/2-1r1/assets/consul-services-nodes-9da6578138bd664e227c215c080a9b5a1f95d85a4a5cc31bf3be28b3e0ab32f1.gif

See also

Consul documentation

Install a Consul agent on your web server machine

We will install and run a Consul agent on the web server machine.

  1. web-icon Install Consul and create the following /etc/consul.d/consul.hcl file.

    datacenter = "dc1"
    server = false
    data_dir = "/opt/consul/"
    bind_addr = "192.168.50.22"
    retry_join = ["192.168.50.21"]
    enable_syslog = true
    log_level = "INFO"

    Notable fields

    Value

    Description

    datacenter

    Consul datacenter name

    The private networking environment all Consul agents belong to.

    server

    false

    This Consul agent will not run in server mode.

    bind_addr

    Web server machine's static IP address

    IP address Consul agents will connect to when they join the cluster.

    retry_join

    Web server machine's static IP address

    IP address of the Consul server, which the agent uses to join the Consul cluster.

  2. web-icon Start the Consul agent.

    $ sudo systemctl enable consul
    $ sudo systemctl start consul
  3. To check if the Consul agent is up and running, open the Consul web UI at http://<Consul Server IP address>:8500 in a web browser.

    Check available nodes.

    https://cdn.haproxy.com/documentation/hapee/2-1r1/assets/consul1-0a60f069c38674d4b480ef187c838545cea7944c019b9a58337d4467e4babb2c.png

The webserver node was created but does not contain any service as yet.

Register the web server as a service

We will register the web server as a Consul service.

  1. web-icon Create a Consul service configuration file on the web server machine.

    For example, create the web.json file with the following contents:

    {
      "service": {
    
        "name": "web",
        "port": 80
      }
    }

    Field

    Description

    port

    Port of the listening web server.

    name

    Name that will display in the Consul registry.

    To add more servers to the same backend, register more services with Consul. Keep the same name, web, in their JSON files.

  2. web-icon Register your web server as a Consul service.

    $ consul services register web.json
  3. To check if the service was registered, open the Consul web UI at http://<Consul Server IP address>:8500 in a web browser.

    To check the details of the service, click its name in the Services tab.

    https://cdn.haproxy.com/documentation/hapee/2-1r1/assets/consul3-b2686ca060e9a7d5d4ee98b28e268ca6df8b52343e14b3fc5c87c438c0ae5ec3.png

See also

Consul Services

Enable service discovery in HAProxy Enterprise

We will install the HAProxy Enterprise Data Plane API on the load balancer machine.

  1. hapee-icon Follow the steps to install the HAProxy Data Plane API on your HAProxy Enterprise server.

  2. hapee-icon To enable HAProxy Enterprise to use the Consul API, send a POST request to the HAProxy Enterprise Data Plane API's /v2/services_discovery/consul endpoint.

    $ curl -u admin:adminpwd \
        -H 'Content-Type: application/json' \
        -d '{
          "address": "192.168.50.21",
          "port": 8500,
          "enabled": true,
          "retry_timeout": 10
        }' http://192.168.50.20:5555/v2/service_discovery/consul

    The body of the request is a JSON string with the following fields:

    Field

    Description

    address

    The Consul server's IP address.

    port

    The Consul server's HTTP API port (i.e. 8500).

    enabled

    Whether to enable this integration.

    retry_timeout

    Duration in seconds in-between requesting data from the Consul server.

    The HAProxy Enterprise Data Plane API now begins monitoring Consul's service catalog.

  3. hapee-icon Check the /etc/hapee-2.1/hapee-lb.cfg file.

    A new backend section contains the web server.

    backend consul-backend-192.168.50.21-8500-web
      server SRV_V2u4y 192.168.50.22:80 check weight 128
      server SRV_1NQXA 127.0.0.1:80 disabledweight 128
      server SRV_DWcAP 127.0.0.1:80 disabledweight 128
      server SRV_IIcqg 127.0.0.1:80 disabledweight 128

    The HAProxy Enterprise Data Plane API has also added several disabled server lines.

    When you register more instances of the same service, HAProxy Enterprise fills in disabled server slots. You can thus scale up or down without a reload, in most cases.

You can now configure a frontend section that routes traffic to this backend pool of servers.

See also

Data Plane API installation