HAProxy Enterprise Documentation 1.8r2

Elastic Stack

This guide illustrates how to connect HAProxy Enterprise to Elastic Stack, a suite of applications for visualizing metrics along with Metricbeat and its HAProxy module, which is installed on each HAProxy Enterprise server to ship metrics and logs from HAProxy Enterprise to the Elastic Stack.

Installation

Elastic Stack is a highly scalable and highly configurable suite of applications that can be deployed on a single machine or dozens of machines, depending upon the volume of data that must be processed, the rate at which the data is ingested, as well as the complexity of the output visualizations desired.

  1. Follow the Installing Elasticsearch guide to install Elasticsearch onto a server. We will store metrics data in Elasticsearch.

  2. Follow the Installing Kibana guide to install Kibana onto another server. We will visualize our data with Kibana.

  3. By default, Elasticsearch is available on port 9200. You can test this using curl:

    $ curl -X GET "<elasticsearch_server_ip>:9200"
    {
    "name" : "es01",
    "cluster_name" : "es-docker-cluster",
    "cluster_uuid" : "2PSZIzo_SH-AEDK9nkPELw",
    "version" : {
        "number" : "7.12.0",
        "build_flavor" : "default",
        "build_type" : "docker",
        "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
        "build_date" : "2021-03-18T06:17:15.410153305Z",
        "build_snapshot" : false,
        "lucene_version" : "8.8.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
    },
    "tagline" : "You Know, for Search"
    }
  4. Ensure that your HAProxy Enterprise configuration has the Runtime API enabled:

    global
        stats socket ipv4@*:9999 user haproxy group haproxy mode 660 level admin
  5. Follow the Installing Metricbeat guide to install Metricbeat onto your HAProxy Enterprise server.

  6. Enable the Metricbeat HAProxy module:

    $ sudo metricbeat modules enable haproxy
  7. Edit the file /etc/metricbeat/metricbeat.yml so that it lists your Elasticsearch server under the output.elasticsearch section:

    output.elasticsearch:
       hosts: ["<elasticsearch_server_ip>:9200"]
  8. Edit the file /etc/metricbeat/modules.d/haproxy.yml to configure the HAProxy module so that the hosts field includes the address and port where your HAProxy Runtime API is listening:

    - module: haproxy
      metricsets: ["info", "stat"]
      period: 10s
      hosts: ["tcp://<haproxy_server_ip>:9999"]
      enabled: true
  9. Restart the Metricbeat service to begin shipping HAProxy metrics to Elasticsearch:

    $ sudo service metricbeat restart
  10. Open Kibana's interface, http://<kibana_server_ip>:5601/app/discover#/ using a web browser. If everything is working correctly, you will see that your metrics are being collected:

    https://cdn.haproxy.com/documentation/hapee/1-8r2/assets/kibana-integration-3-400752cee8a82a20833ed8e4e10e760bb83274a8754d6d04bfb49e2ed67cecfe.png
  11. Begin creating visualizations using Kibana's Lens application, http://<kibana_server_ip>:5601/app/lens/. Drag and drop fields to the visualization area and adjust your timeframe. In this example, I have added a metric for response time during a benchmarking test:

    https://cdn.haproxy.com/documentation/hapee/1-8r2/assets/kibana-integration-5-f28f9a4b71cd8e7e76bfbee8da17ab41c5eee6780eea697633124cecb4a2d168.png