Administration
Manage HAProxy Enterprise logs
HAProxy Enterprise generates two types of logs: access logs, also called traffic logs, and administrative logs.
Enable access logs Jump to heading
Access logs, also called traffic logs, record information about client connections and requests. The logs are stored in the /var/log/hapee-<VERSION> directory and have file names prefixed with lb-access- plus a timestamp.
To enable access logs:
-
Add the following
logdirective in theglobalsection of the HAProxy Enterprise configuration file, if one does not already exist:haproxygloballog 127.0.0.1 local0haproxygloballog 127.0.0.1 local0This definition, which does not set a severity level for events, captures all events and tags them with the
local0syslog facility code. An rsyslog configuration file at/etc/rsyslog.d/hapee-<VERSION>-lb.confroutes these events to the access log file.You can change the IP address to send logs to a remote rsyslog server over UDP.
-
In your
frontendordefaultssection, add alog globaldirective, which applies thelogdirective from theglobalsection to the frontend.haproxyfrontend fe_mainlog globalhaproxyfrontend fe_mainlog global
Enable administrative logs Jump to heading
Administrative logs record information about load balancer process events such as when the load balancer starts or stops. The logs are stored in the /var/log/hapee-<VERSION> directory and have file names prefixed with lb-admin- plus a timestamp.
To enable administrative logs:
-
Add the following
logdirective in theglobalsection of the HAProxy Enterprise configuration file, if one does not already exist:haproxygloballog 127.0.0.1 local1 noticehaproxygloballog 127.0.0.1 local1 noticeThis definition, which sets its severity level to
notice, captures most log events but omits the more verbose traffic events captured by the access logs. It tags those events with the syslog facility codelocal1. An rsyslog configuration file at/etc/rsyslog.d/hapee-<VERSION>-lb.confroutes these events to the admin log file. Basically, they have the same contents as access logs but without theinfolevel details about connection and request events.You can change the IP address to send logs to a remote rsyslog server over UDP.
-
In your
frontendordefaultssection, add alog globaldirective, which applies thelogdirective from theglobalsection to the frontend.haproxyfrontend fe_mainlog globalhaproxyfrontend fe_mainlog global
Set logging levels Jump to heading
Log levels are the same as the severity levels used by the syslog service. Below, we set the level to notice, which captures all events at that level and above (up to emerg):
haproxy
haproxy
HAProxy Enterprise tags each loggable event with a severity level. For example, it categorizes log messages related to connections and HTTP requests with the info severity level. Other events are categorized using one of the other, less verbose levels. From most to least verbose, the severity levels are:
| Level | Events at this level |
|---|---|
| emerg | Errors such as running out of operating system file descriptors. |
| alert | Some rare cases where something unexpected has happened, such as being unable to cache a response. |
| crit | Not used. |
| err | Errors such as being unable to parse a map file, being unable to parse the HAProxy configuration file, and when an operation on a stick table fails. |
| warning | Certain important, but non-critical, errors such as failing to set a request header or failing to connect to a DNS nameserver. |
| notice | Changes to a server’s state, such as being UP or DOWN or when a server is disabled. Other events at startup, such as starting proxies and loading modules, are also included. Health check logging, if enabled, also uses this level. |
| info | TCP connection and HTTP request details and errors. |
| debug | You may write custom Lua code that logs at this level. |
Set conditional log levels Jump to heading
Sometimes, you’ll want to capture a more verbose log entry for a particular request or response, such as when the server returned an error. To change the log level for individual requests or responses that match specified criteria, use one of these directives, depending on whether you’re targeting the request phase or response phase, and whether your’re in HTTP or TCP mode:
http-request set-log-level <level> [ { if | unless } <condition> ]http-response set-log-level <level> [ { if | unless } <condition> ]tcp-request content set-log-level <level> [ { if | unless } <condition> ]tcp-response content set-log-level <level> [ { if | unless } <condition> ]
Using these directives, you can set the log level to any of the eight syslog levels or to the special level silent, which disables logging for this request/response. This rule is not final so the last matching rule wins.
Specify a condition using the ACL syntax:
haproxy
haproxy
Log ASAP Jump to heading
By default, logs are emitted when the session terminates. That is, when all log values are finalized. Capturing some log values, such as total transfer time and message size, can delay generation of the log content, particularly for a large file transfer.
To avoid such delays, you can choose to log available values as quickly as possible by specifying the logasap option. With this option, the load balancer creates the log message as soon as the server connection has been established in mode tcp or as soon as the server sends the complete headers in mode http.
Values not finalized at this time are total time and total size metrics, and they are prefixed with a plus sign “+” in the log to indicate that they are not true, final values.
Enable early logging by adding option logasap to the defaults or frontend section, as shown below. In this example, we also capture the Content-Length header in the logs so that the logs at least indicate how many bytes are expected to be transferred.
haproxy
haproxy
outputtext
outputtext
Set the log format Jump to heading
By default, HAProxy Enterprise logs are sparse, capturing minimal information about each TCP connection in the access logs:
text
text
The information includes:
| Field | Example value |
|---|---|
| the PID of the HAProxy Enterprise process | hapee-lb[2901] |
| the text “Connect from” followed by the client’s source IP address and port | Connect from 192.168.50.1:61459 |
| the text “to” followed by the destination IP address and port | to 192.168.50.25:80 |
| the name of the frontend that received the request and the mode (e.g. HTTP) | (fe_main/HTTP) |
To get a more detailed log, use one of the directives describes in the following sections.
TCP log format Jump to heading
The option tcplog directive is suitable for frontend sections that either specify mode tcp or don’t set mode at all.
haproxy
haproxy
After reloading your configuration, you’ll see the following log format:
outputtext
outputtext
The information included is detailed in the TCP log format section of the reference guide.
The tcplog log format is equivalent to the following log-format definition, if you were to set it yourself:
haproxy
haproxy
HTTP log format Jump to heading
The option httplog directive is suitable for frontend sections that specify mode http. This option provides the same information as the tcplog option, but with additional HTTP details.
haproxy
haproxy
After reloading your configuration, you’ll see the following log format:
outputtext
outputtext
The information included is detailed in the HTTP log format section of the reference guide.
The httplog log format is equivalent to the following log-format definition, if you were to set it yourself:
haproxy
haproxy
CLF log format Jump to heading
If you intend to pass the HTTP log to a log analyzer that supports the CLF format, you can specify the optional clf argument:
haproxy
haproxy
The CLF format provides the same details as the HTTP log format, but in the order required for CLF parsing. It is equivalent to the following log-format definition:
haproxy
haproxy
HTTPS log format Jump to heading
Available since
- HAProxy Enterprise 2.5r1
The option httpslog directive is suitable for HTTP over TLS connections. This option provides the same information as the option httplog directive, but with additional TLS details.
haproxy
haproxy
After reloading your configuration, you’ll see the following log format:
outputtext
outputtext
The information included is detailed in the HTTPS log format section of the reference guide.
The httpslog log format is equivalent to the following log-format definition:
haproxy
haproxy
Custom log format Jump to heading
You can define your own log format and record a custom set of information about connections or HTTP requests. Your log format can include variables and values from fetch methods.
Use the log-format directive to create a new log format in your defaults or frontend section. It specifies a string that contains variables referring to the fields that you’d like to capture.
Often, you will want to keep the existing information collected by the premade log format rules and append new information to them. From version 2.7r1 onward, to make that easier, you can use one of the following environment variables:
HAPROXY_TCP_LOG_FMTHAPROXY_HTTP_LOG_FMTHAPROXY_HTTPS_LOG_FMT
In versions prior to 2.7r1, define the environment variables yourself:
haproxy
haproxy
Reference the environment variable in your log-format line. Below, we start with the HTTP log format and then append a variable named txn.myvariable to the end of the original HTTP log format:
haproxy
haproxy
A log format variable is a string prefixed by the character %. You can use them to create custom log formats. A common way to use this is to record a variable in the logs by using the var fetch method as shown below:
haproxy
haproxy
Avoid spaces in the variable name. A space is considered a separator. If a variable is between square brackets ([ … ]), then it can be an expression, such as to call a fetch method to get a value. Below, we get the value from the res.cache_hit fetch method, which indicates whether the load balancer’s cache was hit:
haproxy
haproxy
HTTP response variables, those with a res. prefix, are easy because they’re certain to be available. However, variables set during the request phase are not available to the log-format directive, which executes during the response phase. To use them, you must store them in a transaction-level variable, as shown below:
haproxy
haproxy
You can prefix a variable with a plus or minus sign and a flag. A plus adds the flag, while a minus removes it. There are three flags available:
Q: adds double quotes around the valueX: represents the value as hexadecimalE: escapes characters with a backslash, such as double quotes and backslashes
In the example below, we add double quotes around the Host header value:
haproxy
haproxy
Use the special variable %o to propagate its flags to all following variables in the same format string.
To emit a verbatim %, it must be preceded by another % to result in %%. HAProxy Enterprise automatically merges consecutive separators.
Log with Docker Jump to heading
When HAProxy Enterprise runs as a Docker container, you can collect logs in the following ways:
- Forward logs to standard out, allowing you to capture them using a log aggregation tool.
- Forward logs to a remote Syslog server, which can run in a separate container.
Log to standard out Jump to heading
This method publishes logs to standard out, allowing you to offload the collection of logs to an external tool, such as logspout.
-
Place a
logdirective into theglobalsection of your configuration file:- use
stdoutas the address - use
format rawfor a shortened log format compatible with a variety of tools - use
local0as the facility code
haproxygloballog stdout format raw local0haproxygloballog stdout format raw local0 - use
-
Add a
log globaldirective to adefaultssection to enable the global logging rule in all subsequentlisten,frontend, andbackendsections:haproxydefaultslog globalhaproxydefaultslog global -
View logs by querying the HAProxy Enterprise container using the
docker logscommand:nixsudo docker logs hapeenixsudo docker logs hapee
Log to a syslog container Jump to heading
This method allows you to forward logs to a container running a Syslog server, such as Rsyslog.
-
Place a
logdirective into theglobalsection of your configuration file:- use the IP address or name of your Syslog container, with an optional port number
- use
local0as the facility code
haproxygloballog rsyslog:514 local0haproxygloballog rsyslog:514 local0 -
Add a
log globaldirective to adefaultssection to enable the global logging rule in all subsequentlisten,frontend, andbackendsections:haproxydefaultslog globalhaproxydefaultslog globalThe full configuration might look like this:
haproxygloballog rsyslog:514 local0defaultslog globalfrontend fe_mainbind :80default_backend be_mainbackend be_mainserver web1 web1:8080 checkhaproxygloballog rsyslog:514 local0defaultslog globalfrontend fe_mainbind :80default_backend be_mainbackend be_mainserver web1 web1:8080 check -
Launch containers:
- Create the Docker network:
nixsudo docker network create -d bridge my-networknixsudo docker network create -d bridge my-network- Run the web server container:
nixsudo docker run -d --network my-network --name web1 \--restart unless-stopped jmalloc/echo-servernixsudo docker run -d --network my-network --name web1 \--restart unless-stopped jmalloc/echo-server- Run the Rsyslog container:
nixsudo docker run -d --network my-network --name rsyslog \--restart unless-stopped jumanjiman/rsyslognixsudo docker run -d --network my-network --name rsyslog \--restart unless-stopped jumanjiman/rsyslog- Run the HAProxy Enterprise container:
nixsudo docker run -d --network my-network --name hapee \-p 80:80 -p 443:443 -p 5555:5555 --restart unless-stopped \-v (pwd):/etc/hapee-2.8 \hapee-registry.haproxy.com/haproxy-enterprise:2.8r1nixsudo docker run -d --network my-network --name hapee \-p 80:80 -p 443:443 -p 5555:5555 --restart unless-stopped \-v (pwd):/etc/hapee-2.8 \hapee-registry.haproxy.com/haproxy-enterprise:2.8r1 -
View logs by querying the rsyslog container using the
docker logscommand:nixsudo docker logs rsyslognixsudo docker logs rsyslog
See also Jump to heading
Do you have any suggestions on how we can improve the content of this page?