HAProxy Enterprise Documentation 2.5r1

Log format for access logs

HAProxy Enterprise supports several extended log formats that you can use instead of the default access log format. You can also develop your own custom format for access logs.

Access logs, also called traffic logs, have the prefix lb-access and describe connections and HTTP requests/responses.

Default access log format

By default, HAProxy Enterprise logs a predefined set of information about each TCP connection in the access logs.

>>> Jan 27 21:15:06 localhost hapee-lb[2901]: Connect from 192.168.50.1:61459 to 192.168.50.25:80 \
      (fe_main/HTTP)

The information includes:

Field	Example
the PID of the HAProxy Enterprise process	hapee-lb[2901]
the text "Connect from" followed by the client's source IP address and port	Connect from 192.168.50.1:61459
the text "to" followed by the destination IP address and port	to 192.168.50.25:80
the name of the frontend that received the request and the mode (e.g. HTTP)	(fe_main/HTTP)

TCP access log format

The tcplog option is suitable for pure TCP proxies.

If you are using mode tcp in your frontend and backend sections, then you can collect more detailed logs by adding option tcplog to the frontend or defaults section, as shown below:

defaults
   mode tcp
   option tcplog

Restart the load balancer to enable the option.

>>> Jan 27 21:22:53 localhost hapee-lb[3042]: 192.168.50.1:61535 [27/Jan/2021:21:22:10.944] \
      fe_main be_servers/s1 1/0/42404 2895 cD 1/1/0/0/0 0/0

The information includes:

Field	Example
the PID of the HAProxy Enterprise process	hapee-lb[3042]
the client's source IP address and port	192.168.50.1:61535
the date that the connection was accepted	[27/Jan/2021:21:22:10.944]
the name of the frontend that received the request	fe_main
the name of the backend that received the request, followed by a slash and the name of the server	be_servers/s1
three numbers separated by slashes: milliseconds the connection was queued milliseconds waiting to connect to the backend server milliseconds between accepting the connection and closing it (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	1/0/42404
the number of bytes read (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	2895
the termination state, which is a two-digit letter code that describes how the connection closed	cD
five numbers separated by slashes: number of concurrent connections when the session was logged number of concurrent connections on the frontend when the session was logged number of concurrent connections on the backend when the session was logged number of concurrent connections active on the server when the session was logged number of connection retries to the server	1/1/0/0/0
two numbers separated by a slash: number of requests processed before this one in the server's queue (if using session persistence) number of requests processed before this one in the backend's queue	0/0

The tcplog log format is equivalent to the following log-format definition:

log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts \
            %ac/%fc/%bc/%sc/%rc %sq/%bq"

HTTP access log format

The httplog option is suitable for HTTP proxies. This option provides the same information as the tcplog option with HTTP details added.

If you are using mode http in your frontend and backend sections, you can collect more detailed logs by adding option httplog to the frontend or defaults section, as shown below:

defaults
   mode http
   option httplog

Restart the load balancer to enable the option.

>>> Jan 27 21:52:56 localhost hapee-lb[3098]: 192.168.50.1:61818 [27/Jan/2021:21:52:56.086] \
      fe_main be_servers/s1 0/0/1/1/2 200 517 - - ---- 1/1/0/0/0 0/0 {1wt.eu} {} "GET / HTTP/1.1"

The information includes:

Field	Example
the PID of the HAProxy Enterprise process	hapee-lb[3098]
the client's source IP address and port	192.168.50.1:61818
the date that the connection was accepted	[27/Jan/2021:21:52:56.086]
the name of the frontend that received the request	fe_main
the name of the backend that received the request, followed by a slash and the name of the server	be_servers/s1
five numbers separated by slashes: milliseconds waiting for the full HTTP request to arrive from the client milliseconds the connection was queued milliseconds waiting to connect to the backend server milliseconds waiting for the server to return a response milliseconds the request was active in HAProxy Enterprise (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	0/0/1/1/2
HTTP status code	200
the number of bytes read (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	517
captured request cookie values (requires using the older `http-request capture cookie` directive)	-
captured response cookie values (requires using the older `http-response capture cookie` directive)	-
the termination state, which is a four-digit letter code that describes how the connection closed	----
five numbers separated by slashes: number of concurrent connections when the session was logged number of concurrent connections on the frontend when the session was logged number of concurrent connections on the backend when the session was logged number of concurrent connections active on the server when the session was logged number of connection retries to the server	1/1/0/0/0
two numbers separated by a slash: number of requests processed before this one in the server's queue (if using session persistence) number of requests processed before this one in the backend's queue	0/0
captured request headers (absent if capture is disabled)	{1wt.eu}
captured response headers (absent if capture is disabled)	{}
the HTTP request line	"GET / HTTP/1.1"

The httplog log format is equivalent to the following log-format definition:

log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
            %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

CLF option

If you intend to pass the HTTP log to a log analyzer that supports the CLF format, you can specify the optional clf argument, like so:

defaults
   mode http
   option httplog clf

The CLF format provides the same details as the HTTP log format but in the order required for CLF parsing. The httplog clf log format is equivalent to the following log-format definition:

log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B \"\" \"\" %cp \
            %ms %ft %b %s %TR %Tw %Tc %Tr %Ta %tsc %ac %fc \
            %bc %sc %rc %sq %bq %CC %CS %hrl %hsl"

HTTPS access log format

The httpslog option is suitable for HTTP over SSL connections. This option provides the same information as the httplog option with SSL details added.

If you are using mode http in your frontend and backend sections, you can collect more detailed logs by adding option httpslog to the frontend or defaults section, as shown below:

defaults
   mode http
   option httpslog

Restart the load balancer to enable the option.

>>> Feb  6 12:14:14 localhost hapee-lb[3098]: 192.168.50.1:61818 [27/Jan/2021:21:52:56.086] \
      fe_main be_servers/s1 0/0/1/1/2 200 517 - - ---- 1/1/0/0/0 0/0 {1wt.eu} {} \
      "GET / HTTP/1.1" 0/0/0/0/0 1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384

The information includes:

Field	Example
the PID of the HAProxy Enterprise process	hapee-lb[3098]
the client's source IP address and port	192.168.50.1:61818
the date that the connection was accepted	[27/Jan/2021:21:52:56.086]
the name of the frontend that received the request	fe_main
the name of the backend that received the request, followed by a slash and the name of the server	be_servers/s1
five numbers separated by slashes: milliseconds waiting for the full HTTP request to arrive from the client milliseconds the connection was queued milliseconds waiting to connect to the backend server milliseconds waiting for the server to return a response milliseconds the request was active in HAProxy Enterprise (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	0/0/1/1/2
HTTP status code	200
the number of bytes read (plus sign "`+`" prefix indicates value is not final due to `logasap` option)	517
captured request cookie values (requires using the older `http-request capture cookie` directive)	-
captured response cookie values (requires using the older `http-response capture cookie` directive)	-
the termination state, which is a four-digit letter code that describes how the connection closed	----
five numbers separated by slashes: number of concurrent connections when the session was logged number of concurrent connections on the frontend when the session was logged number of concurrent connections on the backend when the session was logged number of concurrent connections active on the server when the session was logged number of connection retries to the server	1/1/0/0/0
two numbers separated by a slash: number of requests processed before this one in the server's queue (if using session persistence) number of requests processed before this one in the backend's queue	0/0
captured request headers (absent if capture is disabled)	{1wt.eu}
captured response headers (absent if capture is disabled)	{}
the HTTP request line	"GET / HTTP/1.1"
five numbers separated by slashes: status of connection on frontend last error of the first SSL error stack status of the client's certificate verification process status of the client's certificate chain verification process true if the incoming TLS session was resumed with the stateful cache or a stateless ticket	0/0/0/0/0
Server Name Indication/SSL version/SSL cipher	1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384

The httpslog log format is equivalent to the following log-format definition:

log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
            %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r \
            %[fc_err]/%[ssl_fc_err,hex]/%[ssl_c_err]/\
            %[ssl_c_ca_err]/%[ssl_fc_is_resumed] %[ssl_fc_sni]/%sslv/%sslc"

Custom log formats

You can define your own format and record a custom set of information about connections or HTTP requests. To do so, you will define log format variables, or use predefined log format variables, and arrange them in a new log format rule. Some reasons you may want to customize the fields captured by the HAProxy Enterprise logs:

The default log format is giving you more information than needed

Important pieces of information with the default log format are missing

You need to structure the fields in a way that external tools can read them

You rely on a specific standard log format

Log format variables

A log format variable is a string prefixed by the character %:

%variable

You can use them to create custom log formats that record custom data in your HAProxy Enterprise logs.

Avoid spaces in the variable name. A space is considered as a separator. If a variable is between square brackets ([ .. ]), then it can be an expression, such as to call a fetch method to get a value. Below, we get the value of the Host HTTP header by using the req.hdr fetch method:

%[req.hdr(Host)]

A common way to use this is to record a variable in the logs by using the var fetch method as shown below:

frontend www
   bind :80

   # store the 'path' in a variable named 'txn.mypath'
   http-request set-var(txn.mypath) path

   # logs will show the path value
   log-format %[var(txn.mypath)]

You can prefix a variable with a plus or minus sign and a flag. A plus adds the flag, while a minus removes it. There are three flags available:

Q: adds double quotes around the value
X: represents the value as hexadecimal
E: escapes characters with a backslash, such as double quotes and backslashes

In the example below, we add double quotes around the Host header value:

%{+Q}[req.hdr(Host)]

You can use a special variable %o to propagate its flags to all following variables in the same format string.

Check the list of predefined log format variables in the Reference Guide.

Define a new log format

You will use the log-format directive to create a new log format. This line accepts log format variables. The log-format directive goes into your defaults, frontend or listen section. It specifies a string that contains variables referring to the fields that you'd like to capture.

If you were to replicate the format you get with option tcplog, it would look like this:

defaults
   log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"

The option httplog format would look like this:

defaults
   log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

You can define your own format. For example, below, we collect only the client's IP address and port, the server's IP address and port, the URL path, and the response status code:

defaults
   log-format "%ci:%cp %si:%sp %HU %ST"
   # logs: 192.168.50.1:56428 127.0.0.1:8080 / 304

Store the log format in an environment variable

Often, you will want to keep the existing information collected by the premade log format rules and append new information to them. To make that easier, you can store the original format in an environment variable and then reference it in your log-format line.

Add the premade log format to an environment variable by using setenv in the global section. Below, we store the HTTP log format in a variable named HAPROXY_HTTP_LOG_FMT:
```
global
   setenv HAPROXY_HTTP_LOG_FMT "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
```
Reference the environment variable in your log-format line.
We append a variable named txn.myvariable to the end of the original HTTP log format:
```
defaults
   log-format "$HAPROXY_HTTP_LOG_FMT %[var(txn.myvariable)]"
```

More information

Next up

Log with Docker

Log format for access logs

Default access log format

TCP access log format

HTTP access log format

CLF option

HTTPS access log format

Custom log formats

Log format variables

Define a new log format

Store the log format in an environment variable

Your feedback is important to us!