Log format for access logs
HAProxy Enterprise supports several extended log formats that you can use instead of the default access log format. You can also develop your own custom format for access logs.
Access logs, also called traffic logs, have the prefix lb-access and describe connections and HTTP requests/responses.
Default access log format
By default, HAProxy Enterprise logs a predefined set of information about each TCP connection in the access logs.
>>> Jan 27 21:15:06 localhost hapee-lb[2901]: Connect from 192.168.50.1:61459 to 192.168.50.25:80 \
(fe_main/HTTP)
The information includes:
Field | Example |
---|---|
the PID of the HAProxy Enterprise process | hapee-lb[2901] |
the text "Connect from" followed by the client's source IP address and port | Connect from 192.168.50.1:61459 |
the text "to" followed by the destination IP address and port | to 192.168.50.25:80 |
the name of the frontend that received the request and the mode (e.g. HTTP) | (fe_main/HTTP) |
TCP access log format
The tcplog
option is suitable for pure TCP proxies.
If you are using mode tcp
in your frontend
and backend
sections, then you can collect more detailed logs by adding option tcplog
to the frontend
or defaults
section, as shown below:
defaults
mode tcp
option tcplog
Restart the load balancer to enable the option.
>>> Jan 27 21:22:53 localhost hapee-lb[3042]: 192.168.50.1:61535 [27/Jan/2021:21:22:10.944] \
fe_main be_servers/s1 1/0/42404 2895 cD 1/1/0/0/0 0/0
The information includes:
Field | Example |
---|---|
the PID of the HAProxy Enterprise process | hapee-lb[3042] |
the client's source IP address and port | 192.168.50.1:61535 |
the date that the connection was accepted | [27/Jan/2021:21:22:10.944] |
the name of the frontend that received the request | fe_main |
the name of the backend that received the request, followed by a slash and the name of the server | be_servers/s1 |
three numbers separated by slashes:
| 1/0/42404 |
the number of bytes read (plus sign " | 2895 |
the termination state, which is a two-digit letter code that describes how the connection closed | cD |
five numbers separated by slashes:
| 1/1/0/0/0 |
two numbers separated by a slash:
| 0/0 |
The tcplog
log format is equivalent to the following log-format
definition:
log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"
HTTP access log format
The httplog
option is suitable for HTTP proxies. This option provides the same information as the tcplog
option with HTTP details added.
If you are using mode http
in your frontend
and backend
sections, you can collect more detailed logs by adding option httplog
to the frontend
or defaults
section, as shown below:
defaults
mode http
option httplog
Restart the load balancer to enable the option.
>>> Jan 27 21:52:56 localhost hapee-lb[3098]: 192.168.50.1:61818 [27/Jan/2021:21:52:56.086] \
fe_main be_servers/s1 0/0/1/1/2 200 517 - - ---- 1/1/0/0/0 0/0 {1wt.eu} {} "GET / HTTP/1.1"
The information includes:
Field | Example |
---|---|
the PID of the HAProxy Enterprise process | hapee-lb[3098] |
the client's source IP address and port | 192.168.50.1:61818 |
the date that the connection was accepted | [27/Jan/2021:21:52:56.086] |
the name of the frontend that received the request | fe_main |
the name of the backend that received the request, followed by a slash and the name of the server | be_servers/s1 |
five numbers separated by slashes:
| 0/0/1/1/2 |
HTTP status code | 200 |
the number of bytes read (plus sign " | 517 |
captured request cookie values (requires using the older | - |
captured response cookie values (requires using the older | - |
the termination state, which is a four-digit letter code that describes how the connection closed | ---- |
five numbers separated by slashes:
| 1/1/0/0/0 |
two numbers separated by a slash:
| 0/0 |
captured request headers (absent if capture is disabled) | {1wt.eu} |
captured response headers (absent if capture is disabled) | {} |
the HTTP request line | "GET / HTTP/1.1" |
The httplog
log format is equivalent to the following log-format
definition:
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
CLF option
If you intend to pass the HTTP log to a log analyzer that supports the CLF format, you can specify the optional clf
argument, like so:
defaults
mode http
option httplog clf
The CLF format provides the same details as the HTTP log format but in the order required for CLF parsing. The httplog clf
log format is equivalent to the following log-format
definition:
log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B \"\" \"\" %cp %ms %ft %b %s %TR %Tw %Tc %Tr %Ta %tsc %ac %fc %bc %sc %rc %sq %bq %CC %CS %hrl %hsl"
HTTPS access log format
The httpslog
option is suitable for HTTP over SSL connections. This option provides the same information as the httplog
option with SSL details added.
If you are using mode http
in your frontend
and backend
sections, you can collect more detailed logs by adding option httpslog
to the frontend
or defaults
section, as shown below:
defaults
mode http
option httpslog
Restart the load balancer to enable the option.
>>> Feb 6 12:14:14 localhost hapee-lb[3098]: 192.168.50.1:61818 [27/Jan/2021:21:52:56.086] \
fe_main be_servers/s1 0/0/1/1/2 200 517 - - ---- 1/1/0/0/0 0/0 {1wt.eu} {} \
"GET / HTTP/1.1" 0/0/0/0/0 1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384
The information includes:
Field | Example |
---|---|
the PID of the HAProxy Enterprise process | hapee-lb[3098] |
the client's source IP address and port | 192.168.50.1:61818 |
the date that the connection was accepted | [27/Jan/2021:21:52:56.086] |
the name of the frontend that received the request | fe_main |
the name of the backend that received the request, followed by a slash and the name of the server | be_servers/s1 |
five numbers separated by slashes:
| 0/0/1/1/2 |
HTTP status code | 200 |
the number of bytes read (plus sign " | 517 |
captured request cookie values (requires using the older | - |
captured response cookie values (requires using the older | - |
the termination state, which is a four-digit letter code that describes how the connection closed | ---- |
five numbers separated by slashes:
| 1/1/0/0/0 |
two numbers separated by a slash:
| 0/0 |
captured request headers (absent if capture is disabled) | {1wt.eu} |
captured response headers (absent if capture is disabled) | {} |
the HTTP request line | "GET / HTTP/1.1" |
five numbers separated by slashes:
| 0/0/0/0/0 |
Server Name Indication/SSL version/SSL cipher | 1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384 |
The httpslog
log format is equivalent to the following log-format
definition:
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %[fc_err]/%[ssl_fc_err,hex]/%[ssl_c_err]/%[ssl_c_ca_err]/%[ssl_fc_is_resumed] %[ssl_fc_sni]/%sslv/%sslc"
Custom log formats
You can define your own format and record a custom set of information about connections or HTTP requests. To do so, you will define log format variables, or use predefined log format variables, and arrange them in a new log format rule. Some reasons you may want to customize the fields captured by the HAProxy Enterprise logs:
The default log format is giving you more information than needed
Important pieces of information with the default log format are missing
You need to structure the fields in a way that external tools can read them
You rely on a specific standard log format
Log format variables
A log format variable is a string prefixed by the character %
:
%variable
You can use them to create custom log formats that record custom data in your HAProxy Enterprise logs.
Avoid spaces in the variable name. A space is considered as a separator. If a variable is between square brackets ([
.. ]
), then it can be an expression, such as to call a fetch method to get a value. Below, we get the value of the Host HTTP header by using the req.hdr
fetch method:
%[req.hdr(Host)]
A common way to use this is to record a variable in the logs by using the var
fetch method as shown below:
frontend www
bind :80
# store the 'path' in a variable named 'txn.mypath'
http-request set-var(txn.mypath) path
# logs will show the path value
log-format %[var(txn.mypath)]
You can prefix a variable with a plus or minus sign and a flag. A plus adds the flag, while a minus removes it. There are three flags available:
Q: adds double quotes around the value
X: represents the value as hexadecimal
E: escapes characters with a backslash, such as double quotes and backslashes
In the example below, we add double quotes around the Host header value:
%{+Q}[req.hdr(Host)]
You can use a special variable %o
to propagate its flags to all following variables in the same format string.
Check the list of predefined log format variables in the Reference Guide.
Define a new log format
You will use the log-format
directive to create a new log format. This line accepts log format variables. The log-format
directive goes into your defaults
, frontend
or listen
section. It specifies a string that contains variables referring to the fields that you'd like to capture.
If you were to replicate the format you get with option tcplog
, it would look like this:
defaults
log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"
The option httplog
format would look like this:
defaults
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
You can define your own format. For example, below, we collect only the client's IP address and port, the server's IP address and port, the URL path, and the response status code:
defaults
log-format "%ci:%cp %si:%sp %HU %ST"
# logs: 192.168.50.1:56428 127.0.0.1:8080 / 304
Get the log format from an environment variable
Often, you will want to keep the existing information collected by the premade log format rules and append new information to them. To make that easier, you can read one of the built-in environment variables and then add onto it in your log-format
line.
Use one of the following environment variables:
HAPROXY_TCP_LOG_FMT
HAPROXY_HTTP_LOG_FMT
HAPROXY_HTTPS_LOG_FMT
Reference the environment variable in your log-format
line.
We append a variable named txn.myvariable
to the end of the original HTTP log format:
defaults
log-format "$HAPROXY_HTTP_LOG_FMT %[var(txn.myvariable)]"
More information
Next up
Log with Docker