HAProxy Enterprise Documentation 2.5r1

Using the Dashboard

You configure the Real-time Dashboard in Settings at the top right of the header bar.

Real-time Dashboard basic settings

Real-time Dashboard basic settings

Real-time Dashboard saves these settings automatically in the browser's local storage and they persist throughout page reloads and browser restarts.

  1. You can modify the following settings:

    Basic setting

    Description

    Data Refresh Interval

    Data refresh period.

    Theme

    GUI theme. Currently available themes are Dark (default) and Light.

    Data Source

    Real-time Dashboard module (default) or Real-time Dashboard Gateway.

    Data Source Mode

    • Single Node - Real-time Dashboard requests and displays only the data from the module node it connects to, or the addressed node when the data source is Gateway

    • Cluster Aggregation - Real-time Dashboard requests and displays data from all HAProxy Enterprise nodes from which the module or Gateway was configured to aggregate data.

    Username

    Username for connecting to the Module or Gateway.

    Password

    Password for connecting to the Module or Gateway.

    Real-time Dashboard Gateway Address

    URL of the Real-time Dashboard Gateway (applicable when Data Source is set to Real-time Dashboard Gateway).

    Advanced setting

    Description

    Number Format

    Select between three different formats to display numerical values in Real-time Dashboard:

    • Human Readable - formats values and adds SI order-of-magnitude suffixes (i.e. 1.26 kb/s)

    • Formatted Values - inserts commas in the value and limits the number of decimal places (i.e. 1,261.00 b/s)

    • Raw Values - displays the raw value from the statistics, except when Real-time Dashboard calculates the value, in which case it also limits the number of decimal places (i.e. 1261.00 b/s)

    Calculated Values Source

    • Frontend Values - The values for graphs and Info Bar are summarized from the frontend block stats, errors are request errors

    • Backend Values - The values for graphs and Info Bar are summarized from the backend block stats, errors are response errors

    Cluster Overview Node Display

    Determines the Node display format on Cluster Overview pane:

    • Node #Process ID - i.e. my_server #2

    • Host:Port - i.e. 127.0.0.1:9022

    Warning Threshold

    For values on the Overview Pane with configured limits, this determines the percentage of the limit acting as a threshold over which values show a warning color (yellow). Default is 70%.

    Critical Threshold

    For values on the Overview Pane with configured limits, this determines the percentage of the limit acting as a threshold over which values show a critical color (red). Default is 90%.

    Aggregated Stick Table Suffix

    The suffix that the Stick Table Aggregator takes on to denote the aggregated (i.e. to) stick tables. Default is .aggr

    Local Stick Table Suffix

    The suffix that the Stick Table Aggregator takes on to denote the source (i.e. from) stick tables. Default is .local

    Module HTTP Port

    Port number that Real-time Dashboard listens on by default for non-TLS HTTP connections. If there is no other value specified in the Header URL Box, this is the port number that Real-time Dashboard connects to. Default: 9022.

    Module HTTPS Port

    Port number where the Real-time Dashboard listens on by default for HTTPS connections (when the URL Box is in locked state lock_icon). Default: 9023.

    Gateway TCP Port

    Port number where the HAProxy Enterprise Admin Socket listens on by default for non-TLS socket connections. This value gets relayed to the Dashboard Gateway if there is no other value specified in the URL Box. Default: 9024.

    Gateway SSL Port

    Port number where the HAProxy Enterprise Admin Socket listens on by default for TLS-encrypted socket connections (when the URL Box is in locked state lock_icon. If there is no other port value given, this value gets relayed to the Dashboard Gateway). Default: 9025.

  2. Click Close to modify your settings.

  3. Click Load defaults if you need to return to default settings.

Read metrics in Real-time Dashboard

Overview

The Overview window is the initial view of the application. It displays the node-level (or cluster-level) summarized information, as well as key metric charts.

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/overview2-1e16a1198cc22559a4cd04996560702514fa44850aaf01a7fad5b237a49f3651.png

It contains the following elements:

HTTP Request Rate Chart (1)

Charts the total HTTP request rate across the node (or across all nodes in Cluster Mode).

HTTP Error Rate Chart (2)

Charts the total HTTP error rate across the node (or across all nodes in Cluster Mode).

Average Response Time Chart (3)

Charts the average response time across the node (or across all nodes in Cluster Mode).

HTTP Requests Panel (4)

Displays the node-level (or aggregate) HTTP Request related stats and limits.

SSL Connections Panel (5)

Displays the node-level (or aggregate) SSL Connection related stats and limits.

TCP Connections Panel (6)

Displays the node-level (or aggregate) TCP Connection related stats and limits.

Overview Charts

The Overview graphs display information for HTTP traffic, average response time, and SSL/TCP connections.

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/overviewcharts-82aa993c59b0607470ebc52c9511054875ba80e1e37eb9ac5ba49b8653da66ee.png
Zoom Level Selector (1)

Sets five different zoom levels for the Main Chart Display (30 seconds, 1 minute, 3 minutes, 10 minutes, and 30 minutes) The zoom level is synchronized across all three Overview Graphs.

Main Chart Display (2)

A tool tip appears when hovering over any chart point to show the detailed data point (time and value). This data point is synchronized across all three Overview Charts.

Time Axis (3)

Horizontal axis is the data point event time axis

Value Axis (4)

Vertical axis is the data-point values axis

Chart Navigator (5)

Displays the local data history in a miniature view with its own time axis.

Navigator Handles (6)

Focuses on a time period in the local history when you drag the handles. When you release the handle, the selected zoom level/period is synchronized across the three Overview Charts.

Read statistics in Real-time Dashboard

Stats Pane

The Stats Pane presents individual Proxy (frontend, backend or listen block) statistics, as well as individual Server statistics in a table view.

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/statspane-350d5d828569b8c68f70368f8e32b7729a13cb93e83f630aec34b68dd450ec24.png

The Stats Pane consists of the following major sections:

Filter Block (1)

See Set filter criteria

Quick Navigation buttons (2)

Gives quick access to the selected Proxy on the screen.

Stats Tables (3)

Proxy and Server statistics, see Stats Pane under Data Fields for more detailed data on statistics values.

Quick Action Buttons (4)

See below

Stats Table

The tables operate slightly differently depending on the Data Source Mode:

  • Single Node: Each server appears on one line, and its statistics and status come from the node to which Real-time Dashboard is connected.

  • Cluster Aggregation: Each server appears as a header line in gray with statistics and status aggregated across all the nodes that it contains.

Depending on the mode, Stats tables display the following information:

Proxy Name

Real-time Dashboard infers from available server lines whether the block is a frontend, backend, or listen (or FE/BE pair with same name, which it treats as a listen block).

Server Names

Name of the server

Statistics Columns

Hover over each column heading to display additional information. If the server state is not up, the entire row takes on the Server State color, as used for Filter Block labels. See Stats Pane in the section Understanding data from Real-time Dashboard for more detailed data on statistics values.

Drill down for more information

  • Click the add_icon icons to display statistics for each node (when in cluster mode)

Perform an administrative task

  1. Select a server or all servers in the proxy.

  2. Choose an action from the drop-down list.

  3. Click Apply.

Enable or disable proxy

The Enable/Disable button on the right of the screen allows you to turn off the server for maintenance or turn it back on if it is down.

Read proxy and server states

The Cluster Overview displays with a matrix/"spreadsheet" overview of states of Proxies and Servers for each connected HAProxy Enterprise Node (see Dashboard Aggregation Cluster). The columns in the matrix represent the nodes (which can be HAProxy Enterprise nodes or processes).

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/clusterpane-f50043d9e5357456913f3c631bb549405c1fbf065c2b38f285d118082b06c698.png

The key elements on the pane are:

Quick navigation buttons (1)

Allows quick access to a server or proxy.

Cluster nodes (2)

The rows in the matrix represent HAProxy Enterprise Nodes. The check boxes allow you to select a server or proxy on which to perform an administrative task.

Cluster Overview Matrix (3)

Displays an overview of statuses for Proxies and Servers across Nodes. Servers that do not exist on specific nodes display an empty space (background color) in the matrix.

Proxy Rows (4)

The rows are organized into Proxies containing Servers. The check boxes allow you to select a server or proxy on which to perform an administrative task.

Server Rows (5)

The status color of the header column is medium-gray when the status for the server varies across nodes

Server Columns (6)

Each cell (column) represents the Server status on that column's node.

Set filter criteria

Filtering allows you to display or to perform administrative tasks on selected proxies (i.e. frontend, backend and/or listen blocks) in both Stats and Cluster panes.

The filter criteria persist when you switch between Stats and Cluster panes.

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/filterblock-2e110f6197a41bae9837d297e0ffd1022ef0c8642b9f6059d1c05ae926080f6b.png

Select filter

You can set filters in the following ways:

  • Text - Allows filtering by Frontend or Backend Name (i.e. Proxy name) or by Server Name. There are three modes of filtering:

    • Partial String Matching: For example, when you enter ss, it matches up with blessing, pass, or sserver.

    • Multi-String Partial Matching: For example, when you enter we se it matches up with webserver, awesome_service and we_base.

    • RegEx Matching: For example, when you enter a string that begins and ends with a forward slash (/) it matches only those strings that a regular expression between those two slashes would match. The precise regular expression syntax used is JavaScript Regular Expressions.

  • Server Status - Allows filtering by server status. A click on any server status toggles its state between selected and unselected.

Use filter for administrative tasks

For a list of administrative tasks, see the section on how to Manage servers.

Reset filter

  • Click reset-filter to restore the filter to default state (all Proxies/Servers visible); this function is enabled only when filter is active.

Hide filter

  • Click hide-filter to collapse the filter block (to free up vertical screen space).

Retrieve Stick Table contents

The Stick Table Explorer enables the user to retrieve the contents of stick tables from the HAProxy Enterprise node, and to sort them and filter them to gain insight into the data contained within, and even export them into Excel-friendly CSV format.

https://cdn.haproxy.com/documentation/hapee/2-5r1/assets/rtd-graphics/rtd_1.1_st_explorer_elements_800-fb6ef15f610397c22cc8eb127a26af72442fd54f1e96cb48078da9b448b935c4.png

The Real-time Dashboard works with the Stick Table Explorer to enable you to explore the contents of HAProxy Enterprise Stick Tables in your node(s).

The aggregation cluster mode assumes that the Stick Table Aggregator (hapee-stktagg) aggregates stick tables across the same set of nodes, and that the primary node uses aggregated stick tables from the Stick Table Aggregator.

Local tables used for aggregation have the suffix .local, and resulting aggregated tables have the suffix .aggr (modifiable in General Settings).

Apart from setting up the aggregation cluster and editing table suffixes as necessary, there is no additional setting to configure in Real-time Dashboard.

Retrieve a stick table

  1. Click on stick_icon in the header bar to display the Stick Table Explorer.

  2. Select a table from a drop-down list (1).

  3. Filter out any unwanted data by clicking filter_icon (2) and specifying the filter criteria. Toggling this button shows or hides the filter.

  4. Click get_icon (3) to retrieve the following stick table information (4):

    • Type - type of key

    • Length - number of table columns

    • Expires - how long a row is kept in the table before it expires

    • Purge - whether the table purges old elements

    • Size - maximum number of rows

    • Used - current number of rows

Apply filters

You can filter table data in two stages:

  • Globally, prior to node retrieval (5): Real-time Dashboard retrieves data on the HAProxy Enterprise node itself according to the filtering criteria (up to four maximum).

  • Locally, in the browser (6): In contrast to the global filter, which occurs on the HAProxy Enterprise node itself, the local filter happens within the browser. When you remove a criterion, the table refreshes accordingly; however, it has no effect on the amount of data retrieved from the node.

You can combine the two filters to limit the data retrieved and refine the data to display within the browser.

Set a filter using the interface

  1. Enter a key (string enclosed in quotes "").

  2. Select an operator from the drop-down list and enter a value.

Set a filter using an expression

  1. In the filter box, click on pencil_icon to turn it into a text input box.

  2. Enter an expression in the following format: [data_name operator value] AND [data_name operator value] ... AND [data_name operator value]. The logical AND is implicit in the filter and no other logical operator is accepted.

    where:

    • data_name is one of: key, server_id, gpc0, gpc0_rate, gpc1, gpc1_rate, conn_cnt, conn_cur, conn_rate, sess_cnt, sess_rate, http_req_cnt, http_req_rate, http_err_cnt, http_err_rate, bytes_in_cnt, bytes_in_rate, bytes_out_cnt, bytes_out_rate. When you hover over the table header, a tool tip displays its data point name.

    • operator is any of =, !=, <, >, <= and =>

    • value is numerical value used for comparison in the filter.

Export CSV

  • Click csv_icon (7) to download the entire table in CSV format.

Sort data

  • Click on the up or down arrow next to the column header to sort data by ascending or descending order.

Understanding data from Real-time Dashboard

The Real-time Dashboard interface contains a number of input and output values. The following list documents their meaning, syntax, and, if applicable, the current method of value calculation.

Overview Pane

The Overview pane displays general and summarized real-time information about the HAProxy Enterprise node.

Overview

Description

Overview - Throughput

Real time throughput rates for the HAProxy Enterprise process, containing 3 values:

  • In - Rate of incoming data, in bits per second.

  • Out - Rate of outgoing data, in bits per second.

  • Error Rate - HTTP Error Rate, in errors per second.

Overview - HTTP Request Rate

A graph representation of HTTP request rate, in requests per second.

Overview - HTTP Error Rate

A graph representation of HTTP error rate, in errors per second.

Overview - Average Response Time

A graph representation of average HTTP response time, in milliseconds.

Overview - HAProxy Enterprise Information

HAProxy Enterprise process information, containing:

  • Instance name - HAProxy Enterprise info field Name

  • Instance version - HAProxy Enterprise info field Version

  • Node name - HAProxy Enterprise info field node

  • Process ID - HAProxy Enterprise info field Pid

  • Instance description - HAProxy Enterprise info field Description

  • Uptime - HAProxy Enterprise info field Uptime

  • Idle% - HAProxy Enterprise info field Idle_pct

HTTP Requests

Description

HTTP Requests - Rate

Data block containing two values:

  • Current HTTP request rate, in requests per second.

  • Maximum allowed HTTP request rate, in requests per second.

HTTP Requests - Total

Data block containing two values:

  • Total HTTP requests since the HAProxy Enterprise node was started or since the counters were reset.

  • Highest HTTP request rate seen, in requests per second, since the HAProxy Enterprise node was started or since the counters were reset.

HTTP Requests - Error Rate

Data block containing two values:

  • Current HTTP error rate, in errors per second.

  • Highest HTTP error rate seen, in requests per second, since the HAProxy Enterprise node was started or since the counters were reset.

SSL Connections

Description

SSL Connections - Rate

Data block containing two values:

  • Current SSL connection rate, in requests per second.

  • Maximum allowed SSL connection rate, in requests per second.

SSL Connections - #Conn

Data block containing two values:

  • Current number of active SSL connections.

  • Maximum allowed number of active SSL connections.

SSL Connections - Total

Data block containing two values:

  • Total SSL connections since the HAProxy Enterprise node was started or since the counters were reset.

  • Highest SSL connection rate seen, in requests per second, since the HAProxy Enterprise node was started or since the counters were reset.

TCP Connections

Description

TCP Connections - Rate

Data block containing two values:

  • Current TCP connection rate, in requests per second.

  • Maximum allowed TCP connection rate, in requests per second.

TCP Connections - #Conn

Data block containing two values:

  • Current number of active TCP connections.

  • Maximum allowed number of active TCP connections.

TCP Connections - Total

Data block containing two values:

  • Total TCP connections since the HAProxy Enterprise node was started or since the counters were reset.

  • Highest TCP connection rate seen, in requests per second, since the HAProxy Enterprise node was started or since the counters were reset.

HAProxy Enterprise Real-time Dashboard Copyright and Version

Page footer containing the version of the HAProxy Enterprise Real-time Dashboard. The most recent released version is v1.0.

Stats Pane

The Stats pane displays detailed real time information about the HAProxy Enterprise frontends and backends, and allows you to perform management actions on servers.

Filter

Description

Filter - Frontend/Backend Name

Filters the list and only displays frontend and backend sections whose name matches filter criteria.

Filter - Server Name Full or partial server name

Filters the list and only displays servers (and their corresponding frontend or backend sections) whose name matches the specified filter.

Filter - Server State List of server states

Filters the list and only displays servers (and their corresponding frontend or backend sections) whose state matches one of the selected states. Holding Ctrl while clicking on a particular state will deselect or select all other states for convenience.

Filter - Action to perform

Select the action to perform on all servers visible after filtering (see Manage servers)

Statistics

Description

Frontend or Backend Name

Name of the Frontend/Backend/Listen block (pxname). The checkbox left to the name selects all servers in the block for the Server Management ------ Actions at the bottom. For each server line:

Server name

Name of the server (svname)

Queue - Cur

Current number of requests in server's queue

Queue - Max

Maximum value of current queued requests

Queue - Limit

Configured maximum queue for the server

Session Rate - Cur

Number of sessions per second over last elapsed second

Session Rate - Max

Maximum number of new sessions per second

Session Rate - Limit

Configured limit on new sessions per second

Sessions - Cur

Current number of sessions

Sessions - Max

Maximum number of sessions

Sessions - Limit

Configured session limit

Sessions - Total

Cumulative number of sessions

Sessions - LbTot

Total number of times the server was selected

Bytes - In

Cumulative bytes in to server

Bytes - Out

Cumulative bytes out from server

Denied - Req

Requests denied due to security concerns

Denied - Resp

Responses denied due to security concerns

Error - Req

Request errors

Error - Conn

Number of requests that encountered an error while trying to connect to a backend server

Error - Resp

Response errors

Warnings - Retr

Number of times a connection to a server was retried

Warnings - Redis

Number of times a request was redispatched to another server

Server - Status

Current status and time elapsed since last UP <-> DOWN transition

Server - LastChk

Status of last health check

Server - Wght

Total weight (backend), or server weight (server)

Server - Act

Number of active servers (backend)

Server - Bck

Number of backup servers (backend

Server - Chk

Failed Health Checks

Server - Dwn

Number of UP -> DOWN transitions

Server - Dwntime

Total downtime (backend)

Server - Throttle

Current throttle percentage for the server

Actions

Action on server (see Manage servers)

  • Enable - Set server to READY

  • Disable - Set server to MAINT

Manage servers

You can perform specific management tasks on selected frontends, backends and listen blocks.

Perform an action on server(s)

  1. Select a server by clicking on the checkbox next to its name. To select all servers in a block, click the checkbox next to the block name.

  2. Select a task from the drop-down list (see below).

  3. Click Apply.

Statistics

Description

Set state to READY

Set server state to READY (Enable Server)

Set state to DRAIN

Set server state to DRAIN (remove from load-balancing, but remain available for health checks and persistent connections)

Set state to MAINT

Set server state to MAINT (Disable Server)

Health: disable checks

Mark primary server health checks as temporarily stopped

Health: enable checks

Enable primary server health checks

Health: force UP

Force server health to UP (regardless of pending checks)

Health: force NOLB

Force server health to NOLB state (regardless of slow checks)

Health: force DOWN

Force server health to DOWN (regardless of pending checks)

Agent: disable checks

Mark auxiliary agent checks as temporarily stopped

Agent: enable checks

Enable auxiliary agent checks

Agent: force UP

Force server agent state to UP

Agent: force DOWN

Force server agent state to DOWN

Kill Sessions

Immediately terminate all server sessions


Next up

Process Manager