Advanced features

AWS EC2 service discovery

The HAProxy Data Plane API scales your load balancer configuration to dynamically fit the current capacity of your pool of EC2 instances.

The HAProxy Data Plane API:

  • Regularly polls the AWS EC2 API.
  • Automatically creates backends.
  • Populates your backends with the server addresses of your EC2 instances.
  • Uses the HAProxy Runtime API to minimize reloads during scaling events.

Consul service discovery diagram

See also

Add resource tags to your EC2 instances Jump to heading

Add the following resource tags to either:

  • EC2 instances
  • an AWS Auto Scaling Group launch template

The Data Plane API will discover tagged instances and populate a backend section in your load balancer configuration with their IP addresses. The instances may have been created either individually or from a launch template.

Tag Description Requisiteness Example value
HAProxy:Service:Name A custom service name. Mandatory MyApp
HAProxy:Service:Port Port number that you want to send traffic to. Mandatory 80
HAProxy:Instance:Port Overrides the service port for a single instance. Optional 8080

For example, in a launch template:

AWS Launch template resource tags

See also

Enable Service Discovery on the load balancer Jump to heading

To enable service discovery:

  1. Add a section such as the following to the Data Plane API configuration file:

    dataplaneapi.yml
    yaml
    service_discovery:
    aws_regions:
    - accesskeyid: KAB[...]2X6
    description: Production environment
    enabled: true
    ipv4address: private
    name: my-disc
    region: eu-west-3
    retrytimeout: 10
    secretaccesskey: qjT[...]uXV
    serverslotsbase: 10
    serverslotsgrowthincrement: 0
    serverslotsgrowthtype: exponential
    dataplaneapi.yml
    yaml
    service_discovery:
    aws_regions:
    - accesskeyid: KAB[...]2X6
    description: Production environment
    enabled: true
    ipv4address: private
    name: my-disc
    region: eu-west-3
    retrytimeout: 10
    secretaccesskey: qjT[...]uXV
    serverslotsbase: 10
    serverslotsgrowthincrement: 0
    serverslotsgrowthtype: exponential

    Alternatively, you can run a curl command with JSON keys such as the following:

    nix
    curl -H 'Content-Type: application/json' \
    -u admin:adminpwd \
    -d '{
    "access_key_id":"KAB[...]2X6",
    "description":"Production environment",
    "enabled":true,
    "ipv4_address":"private",
    "name":"my-disc",
    "region":"eu-west-3",
    "secret_access_key":"qjT[...]uXV",
    "retry_timeout":10}' \
    http://127.0.0.1:5555/v2/service_discovery/aws
    nix
    curl -H 'Content-Type: application/json' \
    -u admin:adminpwd \
    -d '{
    "access_key_id":"KAB[...]2X6",
    "description":"Production environment",
    "enabled":true,
    "ipv4_address":"private",
    "name":"my-disc",
    "region":"eu-west-3",
    "secret_access_key":"qjT[...]uXV",
    "retry_timeout":10}' \
    http://127.0.0.1:5555/v2/service_discovery/aws

    This command sends configuration and authentication data to the Data Plane API’s AWS service discovery endpoint. Upon successful registration, it returns a JSON result and populates the Data Plane API configuration file.

    In this example, as we do not specify values for the server_slots_base, server_slots_growth_increment, and server_slots_growth_type directives, default values will be used.

    Configuration directive JSON directive Description
    AccessKeyID access_key_id AWS Access Key ID
    Description description Service discovery description
    Enabled enabled true or false. If set to false, the Data Plane API will not update server sections in discovered backend sections. Data can become outdated if newer EC2 instances launch or if an existing instance reboots with a new IPv4 address.
    IPV4Address ipv4_address private (for the private network, reachable inside the AWS VPC) or public.
    Name name Service discovery name
    Region region AWS region
    RetryTimeout retry_timeout Interval of time in seconds between the reconciliation and the following.
    SecretAccessKey secret_access_key AWS Secret Access Key
    ServerSlotsBase server_slots_base The minimum amount of server entries per backend section. Defaults to 10.
    ServerSlotsGrowthIncrement server_slots_growth_increment The number of additional slots that are allocated for server entries if there are additional entries. Defaults to 0.
    ServerSlotsGrowthType server_slots_growth_type Function type to implement when the number of server slots increases: exponential or linear. Defaults to exponential.

    EC2 instances with attached IAM roles

    If the EC2 instance where the Data Plane API is running has an IAM Role attached (as AmazonEC2ReadOnlyAccess), you do not need to specify credentials.

  2. Restart the service:

    If running the API via the HAProxy Process Manager, restart HAProxy.

    nix
    sudo systemctl restart hapee-extras-dataplaneapi
    nix
    sudo systemctl restart hapee-extras-dataplaneapi

    In the Services tab, restart the dataplaneapi service.

  3. Check the load balancer configuration file. A new backend section displays:

    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    backend aws-eu-west-3-my-disc-MyApp-80
    server SRV_ONGQw 172.31.14.35:80 check weight 128
    server SRV_5hi3l 127.0.0.1:80 disabled weight 128
    server SRV_L2rw7 127.0.0.1:80 disabled weight 128
    server SRV_pG3MX 127.0.0.1:80 disabled weight 128
    server SRV_9WFsc 127.0.0.1:80 disabled weight 128
    server SRV_o2UU2 127.0.0.1:80 disabled weight 128
    server SRV_GMuw1 127.0.0.1:80 disabled weight 128
    server SRV_3Id36 127.0.0.1:80 disabled weight 128
    server SRV_6Ivld 127.0.0.1:80 disabled weight 128
    server SRV_jYF88 127.0.0.1:80 disabled weight 128
    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    backend aws-eu-west-3-my-disc-MyApp-80
    server SRV_ONGQw 172.31.14.35:80 check weight 128
    server SRV_5hi3l 127.0.0.1:80 disabled weight 128
    server SRV_L2rw7 127.0.0.1:80 disabled weight 128
    server SRV_pG3MX 127.0.0.1:80 disabled weight 128
    server SRV_9WFsc 127.0.0.1:80 disabled weight 128
    server SRV_o2UU2 127.0.0.1:80 disabled weight 128
    server SRV_GMuw1 127.0.0.1:80 disabled weight 128
    server SRV_3Id36 127.0.0.1:80 disabled weight 128
    server SRV_6Ivld 127.0.0.1:80 disabled weight 128
    server SRV_jYF88 127.0.0.1:80 disabled weight 128

    The name of the new backend comprises the following parts:

    aws-<AWS region>-<Service discovery name>-<HAProxy:Service:Name tag value>-<HAProxy:Service:Port tag value>

    This backend may only have several servers enabled out of the total number of servers specified by the ServerSlotsBase directive. When you register more instances of the same service, the API fills in disabled server slots. You can thus scale up or down without a reload, in most cases.

  4. Edit the new backend as needed. For example, you can specify a load balancing algorithm via the balance directive.

  5. Configure a frontend section that routes traffic to this backend pool of servers. For example:

    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    frontend example
    bind :80
    default_backend aws-eu-west-3-my-disc-MyApp-80
    haproxy.cfg, hapee-lb.cfg, LB Layer7 tab
    haproxy
    frontend example
    bind :80
    default_backend aws-eu-west-3-my-disc-MyApp-80

If this page was useful, please, Leave the feedback.