Integrations

Elastic Stack

On this page

This section illustrates how to connect the load balancer to Elastic Stack, a suite of applications for visualizing metrics along with Metricbeat and its HAProxy module, which is installed on each load balancer server to ship metrics and logs from the load balancer to the Elastic Stack.

Installation Jump to heading

Elastic Stack is a highly scalable and highly configurable suite of applications that can be deployed on a single machine or dozens of machines, depending upon the volume of data that must be processed, the rate at which the data is ingested, as well as the complexity of the output visualizations desired.

  1. Follow the Installing Elasticsearch guide to install Elasticsearch onto a server.

    We will store metrics data in Elasticsearch.

  2. Follow the Installing Kibana guide to install Kibana onto another server.

    We will visualize our data with Kibana.

  3. By default, Elasticsearch is available on port 9200. You can test this using curl:

    nix
    curl -X GET "<elasticsearch_server_ip>:9200"
    nix
    curl -X GET "<elasticsearch_server_ip>:9200"
    output
    text
    {
    "name" : "es01",
    "cluster_name" : "es-docker-cluster",
    "cluster_uuid" : "2PSZIzo_SH-AEDK9nkPELw",
    "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
    "build_date" : "2021-03-18T06:17:15.410153305Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
    },
    "tagline" : "You Know, for Search"
    }
    output
    text
    {
    "name" : "es01",
    "cluster_name" : "es-docker-cluster",
    "cluster_uuid" : "2PSZIzo_SH-AEDK9nkPELw",
    "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
    "build_date" : "2021-03-18T06:17:15.410153305Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
    },
    "tagline" : "You Know, for Search"
    }
  4. Ensure that your load balancer configuration has the Runtime API enabled:

    haproxy
    global
    stats socket ipv4@*:9999 user haproxy group haproxy mode 660 level admin
    haproxy
    global
    stats socket ipv4@*:9999 user haproxy group haproxy mode 660 level admin
  5. Follow the Installing Metricbeat guide to install Metricbeat onto your load balancer server.

  6. Enable the Metricbeat HAProxy module:

    nix
    sudo metricbeat modules enable haproxy
    nix
    sudo metricbeat modules enable haproxy
  7. Edit the file /etc/metricbeat/metricbeat.yml so that it lists your Elasticsearch server under the output.elasticsearch section:

    yaml
    output.elasticsearch:
    hosts: ["<elasticsearch_server_ip>:9200"]
    yaml
    output.elasticsearch:
    hosts: ["<elasticsearch_server_ip>:9200"]
  8. Edit the file /etc/metricbeat/modules.d/haproxy.yml to configure the HAProxy module so that the hosts field includes the address and port where your HAProxy Runtime API is listening:

    yaml
    - module: haproxy
    metricsets: ["info", "stat"]
    period: 10s
    hosts: ["tcp://<haproxy_server_ip>:9999"]
    enabled: true
    yaml
    - module: haproxy
    metricsets: ["info", "stat"]
    period: 10s
    hosts: ["tcp://<haproxy_server_ip>:9999"]
    enabled: true
  9. Restart the Metricbeat service to begin shipping HAProxy metrics to Elasticsearch:

    nix
    sudo service metricbeat restart
    nix
    sudo service metricbeat restart
  10. Open Kibana’s interface, http://<kibana_server_ip>:5601/app/discover#/ using a web browser.

    If everything is working correctly, you will see that your metrics are being collected:

    Kibana metrics

  11. Begin creating visualizations using Kibana’s Lens application, http://<kibana_server_ip>:5601/app/lens/.

    Drag and drop fields to the visualization area and adjust your timeframe.

    In this example, we added a metric for response time during a benchmarking test:

    Kibana response time

If this page was useful, please, Leave the feedback.