Elastic Stack

Installation

Elastic Stack is a highly scalable and highly configurable suite of applications that can be deployed on a single machine or dozens of machines, depending upon the volume of data that must be processed, the rate at which the data is ingested, as well as the complexity of the output visualizations desired.

1. Follow the Installing Elasticsearch guide to install Elasticsearch onto a server.

   We will store metrics data in Elasticsearch.

2. Follow the Installing Kibana guide to install Kibana onto another server.

   We will visualize our data with Kibana.

3. By default, Elasticsearch is available on port 9200. You can test this using curl:

   $ curl -X GET "<elasticsearch_server_ip>:9200"
   {
   "name" : "es01",
   "cluster_name" : "es-docker-cluster",
   "cluster_uuid" : "2PSZIzo_SH-AEDK9nkPELw",
   "version" : {
       "number" : "7.12.0",
       "build_flavor" : "default",
       "build_type" : "docker",
       "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
       "build_date" : "2021-03-18T06:17:15.410153305Z",
       "build_snapshot" : false,
       "lucene_version" : "8.8.0",
       "minimum_wire_compatibility_version" : "6.8.0",
       "minimum_index_compatibility_version" : "6.0.0-beta1"
   },
   "tagline" : "You Know, for Search"
   }

4. Ensure that your HAProxy Enterprise configuration has the Runtime API enabled:

   global
       stats socket ipv4@*:9999 user haproxy group haproxy mode 660 level admin

5. Follow the Installing Metricbeat guide to install Metricbeat onto your HAProxy Enterprise server.

6. Enable the Metricbeat HAProxy module:

   $ sudo metricbeat modules enable haproxy

7. Edit the file /etc/metricbeat/metricbeat.yml so that it lists your Elasticsearch server under the output.elasticsearch section:

   output.elasticsearch:
      hosts: ["<elasticsearch_server_ip>:9200"]

8. Edit the file /etc/metricbeat/modules.d/haproxy.yml to configure the HAProxy module so that the hosts field includes the address and port where your HAProxy Runtime API is listening:

   - module: haproxy
     metricsets: ["info", "stat"]
     period: 10s
     hosts: ["tcp://<haproxy_server_ip>:9999"]
     enabled: true

9. Restart the Metricbeat service to begin shipping HAProxy metrics to Elasticsearch:

   $ sudo service metricbeat restart

10. Open Kibana's interface, http://<kibana_server_ip>:5601/app/discover#/ using a web browser.

    If everything is working correctly, you will see that your metrics are being collected:

    

11. Begin creating visualizations using Kibana's Lens application, http://<kibana_server_ip>:5601/app/lens/.

    Drag and drop fields to the visualization area and adjust your timeframe.

    In this example, I have added a metric for response time during a benchmarking test: