HAProxy Enterprise Documentation 2.4r1

Prometheus

HAProxy Enterprise exposes a Prometheus endpoint that publishes metrics that you can scrape with a Prometheus-compatible agent such as the Prometheus server, Fluentd, Telegraf, and Metricbeat. It is enabled by default, so you only need to configure the IP address and port where it listens.

Configure the Prometheus exporter

Add a new frontend section to your configuration file:

  • Set the IP address and port where you would like to scrape metrics.

  • Include the http-request use-service prometheus-exporter directive to enable the Prometheus exporter.

In the following example, the Prometheus exporter page is available on all IP addresses at port 8405:

frontend prometheus
   bind *:8405
   http-request use-service prometheus-exporter
   no log

Optionally, add a conditional statement to the end of the line to show the metrics page only on a certain URL path. In the next example, the metrics are shown only at the URL path /metrics:

frontend prometheus
   bind *:8405
   http-request use-service prometheus-exporter if { path /metrics }
   no log

Enable scraping using the Prometheus server

Configure your Prometheus server to scrape the HAProxy Enterprise metrics page.

  1. Follow the First Steps with Prometheus guide to install the Prometheus server.

  2. Edit its prometheus.yml file so that it includes a job for scraping HAProxy Enterprise. Add the following to the scrape_configs section, replacing localhost with your load balancer's address. If you run multiple HAProxy Enterprise nodes, include them in the targets array.

    - job_name: 'haproxy-enterprise'
  static_configs:
    - targets: ['localhost:8405']

    By default, the Prometheus server scrapes the URL /metrics. To change this path, set the metrics_path parameter in the scrape_configs section of the Prometheus configuration file.

Host the Prometheus endpoint over HTTPS

Optionally, you can serve the Prometheus endpoint over HTTPS. Follow these steps:

  1. On the HAProxy Enterprise server, edit the load balancer configuration. Add the ssl parameter to the bind line to enable HTTPS. Also, set the crt parameter to the path where you've stored your TLS key and certificate file.

    frontend prometheus
   bind *:8405 ssl crt /etc/hapee-2.4/certs/site.pem
   http-request use-service prometheus-exporter
   no log

  2. On the Prometheus server, edit the prometheus.yml configuration file. Set scheme to https.

    You can also configure the tls_config block to set a CA certificate for verifying the load balancer's certificate, enable client certificates, set a ServerName extension, or disable validation of the load balancer's certificate. See the Prometheus configuration documentation for more information.

    - job_name: 'haproxy-enterprise'
  scheme: https
  static_configs:
    - targets: ['localhost:8405']
  tls_config:
    # Disable validation of server certificate
    insecure_skip_verify: true

Filter metrics

To reduce the volume of metrics returned by the proxy, you can specify filter parameters in the Prometheus configuration YAML. The filters are applied on the HAProxy Enterprise server, thereby reducing the load on the network and the Prometheus server.

Specify scope by application component

Use the scope parameter to specify the component whose metrics you want to scrape:

  • global

  • frontend

  • listener

  • backend

  • server

  • sticktable

  • asterisk (*) = all scopes

  • empty = no metrics

You can specify multiple scope parameters if needed.

This configuration has two scrape jobs: one for Prometheus and another for a HAProxy Enterprise proxy. In both jobs, we leave metrics_path and scheme set to the defaults, /metrics and http, respectively. In the proxy job, we are scraping only frontend and sticktable metrics. The configuration has no alertmanager or load rules sections.

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: 'haproxy-enterprise'
    static_configs:
     - targets: ['192.168.56.31:8405']
    params:
     scope:
      - frontend
      - sticktable

Omit servers in maintenance mode

You can further reduce the volume of metrics being returned by omitting results from servers in maintenance mode. Enable this feature by defining the no-maint parameter. The no-maint parameter takes no arguments.

The no-maint parameter is particularly useful in environments where server templates are used to manage the dynamic provisioning of high numbers of servers.

Define no-maint to omit metrics from servers in maintenance mode.

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: 'haproxy-enterprise'
    static_configs:
     - targets: ['192.168.56.31:8405']
    params:
     scope:
      - frontend
      - sticktable
     no-maint:
      - empty

Scrape health checks only

To scrape only server health status without other server metrics, configure the proxy job with a metric_relabel_configs section, as follows:

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: 'haproxy-enterprise'
     static_configs:
     - targets: ['192.168.56.31:8405']
     metric_relabel_configs:
     - source_labels: [__name__]
       regex: 'haproxy_(process_|frontend_|listener_|backend_|server_check_status).*'
       action: keep

Exported metrics

The following metrics are exported.

Global metrics

  • haproxy_process_active_peers

  • haproxy_process_build_info

  • haproxy_process_busy_polling_enabled

  • haproxy_process_bytes_out_rate

  • haproxy_process_bytes_out_total

  • haproxy_process_connected_peers

  • haproxy_process_connections_total

  • haproxy_process_current_backend_ssl_key_rate

  • haproxy_process_current_connection_rate

  • haproxy_process_current_connections

  • haproxy_process_current_frontend_ssl_key_rate

  • haproxy_process_current_run_queue

  • haproxy_process_current_session_rate

  • haproxy_process_current_ssl_connections

  • haproxy_process_current_ssl_rate

  • haproxy_process_current_tasks

  • haproxy_process_current_zlib_memory

  • haproxy_process_dropped_logs_total

  • haproxy_process_failed_resolutions

  • haproxy_process_frontend_ssl_reuse

  • haproxy_process_hard_max_connections

  • haproxy_process_http_comp_bytes_in_total

  • haproxy_process_http_comp_bytes_out_total

  • haproxy_process_idle_time_percent

  • haproxy_process_jobs

  • haproxy_process_limit_connection_rate

  • haproxy_process_limit_http_comp

  • haproxy_process_limit_session_rate

  • haproxy_process_limit_ssl_rate

  • haproxy_process_listeners

  • haproxy_process_max_backend_ssl_key_rate

  • haproxy_process_max_connection_rate

  • haproxy_process_max_connections

  • haproxy_process_max_fds

  • haproxy_process_max_frontend_ssl_key_rate

  • haproxy_process_max_memory_bytes

  • haproxy_process_max_pipes

  • haproxy_process_max_session_rate

  • haproxy_process_max_sockets

  • haproxy_process_max_ssl_connections

  • haproxy_process_max_ssl_rate

  • haproxy_process_max_zlib_memory

  • haproxy_process_nbproc

  • haproxy_process_nbthread

  • haproxy_process_pipes_free_total

  • haproxy_process_pipes_used_total

  • haproxy_process_pool_allocated_bytes

  • haproxy_process_pool_failures_total

  • haproxy_process_pool_used_bytes

  • haproxy_process_recv_logs_total

  • haproxy_process_relative_process_id

  • haproxy_process_requests_total

  • haproxy_process_spliced_bytes_out_total

  • haproxy_process_ssl_cache_lookups_total

  • haproxy_process_ssl_cache_misses_total

  • haproxy_process_ssl_connections_total

  • haproxy_process_start_time_seconds

  • haproxy_process_stopping

  • haproxy_process_unstoppable_jobs

  • haproxy_process_uptime_seconds

Frontend metrics

  • haproxy_frontend_bytes_in_total

  • haproxy_frontend_bytes_out_total

  • haproxy_frontend_connections_rate_max

  • haproxy_frontend_connections_total

  • haproxy_frontend_current_sessions

  • haproxy_frontend_denied_connections_total

  • haproxy_frontend_denied_sessions_total

  • haproxy_frontend_failed_header_rewriting_total

  • haproxy_frontend_http_cache_hits_total

  • haproxy_frontend_http_cache_lookups_total

  • haproxy_frontend_http_comp_bytes_bypassed_total

  • haproxy_frontend_http_comp_bytes_in_total

  • haproxy_frontend_http_comp_bytes_out_total

  • haproxy_frontend_http_comp_responses_total

  • haproxy_frontend_http_requests_rate_max

  • haproxy_frontend_http_requests_total

  • haproxy_frontend_http_responses_total

  • haproxy_frontend_intercepted_requests_total

  • haproxy_frontend_internal_errors_total

  • haproxy_frontend_limit_session_rate

  • haproxy_frontend_limit_sessions

  • haproxy_frontend_max_session_rate

  • haproxy_frontend_max_sessions

  • haproxy_frontend_request_errors_total

  • haproxy_frontend_requests_denied_total

  • haproxy_frontend_responses_denied_total

  • haproxy_frontend_sessions_total

  • haproxy_frontend_status

Listener metrics

  • haproxy_listener_bytes_in_total

  • haproxy_listener_bytes_out_total

  • haproxy_listener_current_sessions

  • haproxy_listener_denied_connections_total

  • haproxy_listener_denied_sessions_total

  • haproxy_listener_failed_header_rewriting_total

  • haproxy_listener_internal_errors_total

  • haproxy_listener_limit_sessions

  • haproxy_listener_max_sessions

  • haproxy_listener_request_errors_total

  • haproxy_listener_requests_denied_total

  • haproxy_listener_responses_denied_total

  • haproxy_listener_sessions_total

  • haproxy_listener_status

Backend metrics

  • haproxy_backend_active_servers

  • haproxy_backend_agg_check_status

  • haproxy_backend_agg_server_status

  • haproxy_backend_backup_servers

  • haproxy_backend_bytes_in_total

  • haproxy_backend_bytes_out_total

  • haproxy_backend_check_last_change_seconds

  • haproxy_backend_check_up_down_total

  • haproxy_backend_client_aborts_total

  • haproxy_backend_connect_time_average_seconds

  • haproxy_backend_connection_attempts_total

  • haproxy_backend_connection_errors_total

  • haproxy_backend_connection_reuses_total

  • haproxy_backend_current_queue

  • haproxy_backend_current_sessions

  • haproxy_backend_downtime_seconds_total

  • haproxy_backend_failed_header_rewriting_total

  • haproxy_backend_http_cache_hits_total

  • haproxy_backend_http_cache_lookups_total

  • haproxy_backend_http_comp_bytes_bypassed_total

  • haproxy_backend_http_comp_bytes_in_total

  • haproxy_backend_http_comp_bytes_out_total

  • haproxy_backend_http_comp_responses_total

  • haproxy_backend_http_requests_total

  • haproxy_backend_http_responses_total

  • haproxy_backend_internal_errors_total

  • haproxy_backend_last_session_seconds

  • haproxy_backend_limit_sessions

  • haproxy_backend_loadbalanced_total

  • haproxy_backend_max_connect_time_seconds

  • haproxy_backend_max_queue

  • haproxy_backend_max_queue_time_seconds

  • haproxy_backend_max_response_time_seconds

  • haproxy_backend_max_session_rate

  • haproxy_backend_max_sessions

  • haproxy_backend_max_total_time_seconds

  • haproxy_backend_queue_time_average_seconds

  • haproxy_backend_redispatch_warnings_total

  • haproxy_backend_requests_denied_total

  • haproxy_backend_response_errors_total

  • haproxy_backend_response_time_average_seconds

  • haproxy_backend_responses_denied_total

  • haproxy_backend_retry_warnings_total

  • haproxy_backend_server_aborts_total

  • haproxy_backend_sessions_total

  • haproxy_backend_status

  • haproxy_backend_total_time_average_seconds

  • haproxy_backend_uweight

  • haproxy_backend_weight

Server metrics

  • haproxy_server_bytes_in_total

  • haproxy_server_bytes_out_total

  • haproxy_server_check_code

  • haproxy_server_check_duration_seconds

  • haproxy_server_check_failures_total

  • haproxy_server_check_last_change_seconds

  • haproxy_server_check_status

  • haproxy_server_check_up_down_total

  • haproxy_server_client_aborts_total

  • haproxy_server_connect_time_average_seconds

  • haproxy_server_connection_attempts_total

  • haproxy_server_connection_errors_total

  • haproxy_server_connection_reuses_total

  • haproxy_server_current_queue

  • haproxy_server_current_sessions

  • haproxy_server_current_throttle

  • haproxy_server_downtime_seconds_total

  • haproxy_server_failed_header_rewriting_total

  • haproxy_server_http_responses_total

  • haproxy_server_idle_connections_current

  • haproxy_server_idle_connections_limit

  • haproxy_server_internal_errors_total

  • haproxy_server_last_session_seconds

  • haproxy_server_limit_sessions

  • haproxy_server_loadbalanced_total

  • haproxy_server_max_connect_time_seconds

  • haproxy_server_max_queue

  • haproxy_server_max_queue_time_seconds

  • haproxy_server_max_response_time_seconds

  • haproxy_server_max_session_rate

  • haproxy_server_max_sessions

  • haproxy_server_max_total_time_seconds

  • haproxy_server_need_connections_current

  • haproxy_server_queue_limit

  • haproxy_server_queue_time_average_seconds

  • haproxy_server_redispatch_warnings_total

  • haproxy_server_response_errors_total

  • haproxy_server_response_time_average_seconds

  • haproxy_server_responses_denied_total

  • haproxy_server_retry_warnings_total

  • haproxy_server_safe_idle_connections_current

  • haproxy_server_server_aborts_total

  • haproxy_server_sessions_total

  • haproxy_server_status

  • haproxy_server_total_time_average_seconds

  • haproxy_server_unsafe_idle_connections_current

  • haproxy_server_used_connections_current

  • haproxy_server_uweight

  • haproxy_server_weight

Stick table metrics

  • haproxy_sticktable_size

  • haproxy_sticktable_used

Next up

Send Metrics