Synopsis

One of the strength of HAProxy is its logging system. It is very verbose and provides many information.
HAProxy HTTP log line is briefly explained in an HAProxy Technologies memo. It’s a must have document when you have to analyze HAProxy‘s log lines to troubleshoot an issue.
An other interesting tool is HALog. It is available in HAProxy‘s sources, in the contrib directory. I’ll write later an article about it. In order to have an idea on how to use it, just have a look at HAProxy Technologies howto related to halog and HTTP analyze.

Why customizing HAProxy’s logs ???


There may be several reasons why one want to customize HAProxy’s logs:

  • the default log format is too much complicated
  • there are too many information in the default log format
  • there is not enough information in the default log format
  • third party log anaylzer can hardly understand default HAProxy log format
  • logs generated by HAPorxy must be compliant to an existing format from an existing appliance in the architecture
  • … add your own reason here …

That’s why, at HAProxy Technologies, we felt the need of letting our users to create their own HAProxy log-format.
As for compression in HAProxy, the job was done by Wlallemand.

HAProxy log format customization

Configuration directive

The name of the directive which allows you to generate a home made log format is simply called log-format.

Variables

The log-format directive understand variables.
A variable follows the rules below:

  • it is preceded by a percent character: ‘%
  • it can take arguments in braces ‘{}‘.
  • If multiple arguments, then they are separated by commas ‘,‘ within the braces.
  • Flags may be added or removed by prefixing them with a ‘+‘ or ‘‘ sign.
  • spaces ‘ ‘ must be escaped (It is considered as a separator)

Currently available flags:

  • Q: quote a string
  • X: hexadecimal representation (IPs, Ports, %Ts, %rt, %pid)

Currently available variables:
[sourcecode]
+—+——+———————————————–+————-+
| R | var | field name (8.2.2 and 8.2.3 for description) | type |
+—+——+———————————————–+————-+
| | %o | special variable, apply flags on all next var | |
+—+——+———————————————–+————-+
| | %B | bytes_read | numeric |
| | %Ci | client_ip | IP |
| | %Cp | client_port | numeric |
| | %Bi | backend_source_ip | IP |
| | %Bp | backend_source_port | numeric |
| | %Fi | frontend_ip | IP |
| | %Fp | frontend_port | numeric |
| | %H | hostname | string |
| | %ID | unique-id | string |
| | %Si | server_IP | IP |
| | %Sp | server_port | numeric |
| | %T | gmt_date_time | date |
| | %Tc | Tc | numeric |
| H | %Tq | Tq | numeric |
| H | %Tr | Tr | numeric |
| | %Ts | timestamp | numeric |
| | %Tt | Tt | numeric |
| | %Tw | Tw | numeric |
| | %ac | actconn | numeric |
| | %b | backend_name | string |
| | %bc | beconn | numeric |
| | %bq | backend_queue | numeric |
| H | %cc | captured_request_cookie | string |
| H | %rt | http_request_counter | numeric |
| H | %cs | captured_response_cookie | string |
| | %f | frontend_name | string |
| | %ft | frontend_name_transport (‘~’ suffix for SSL) | string |
| | %fc | feconn | numeric |
| H | %hr | captured_request_headers default style | string |
| H | %hrl | captured_request_headers CLF style | string list |
| H | %hs | captured_response_headers default style | string |
| H | %hsl | captured_response_headers CLF style | string list |
| | %ms | accept date milliseconds | numeric |
| | %pid | PID | numeric |
| H | %r | http_request | string |
| | %rc | retries | numeric |
| | %s | server_name | string |
| | %sc | srv_conn | numeric |
| | %sq | srv_queue | numeric |
| S | %sslc| ssl_ciphers (ex: AES-SHA) | string |
| S | %sslv| ssl_version (ex: TLSv1) | string |
| H | %st | status_code | numeric |
| | %t | date_time | date |
| | %ts | termination_state | string |
| H | %tsc | termination_state with cookie status | string |
+—+——+———————————————–+————-+

R = Restrictions : H = mode http only ; S = SSL only
[/sourcecode]

Log format examples

Default log format

  • TCP log format
    [sourcecode language=”text”]log-format %Ci:%Cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts
    %ac/%fc/%bc/%sc/%rc %sq/%bq
    [/sourcecode]
  • HTTP log format
    [sourcecode language=”text”]log-format %Ci:%Cp [%t] %ft %b/%s %Tq/%Tw/%Tc/%Tr/%Tt %st %B %cc
    %cs %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r
    [/sourcecode]
  • CLF log format
    [sourcecode language=”text”]log-format %{+Q}o %{-Q}Ci – – [%T] %r %st %B "" "" %Cp
    %ms %ft %b %s %Tq %Tw %Tc %Tr %Tt %tsc %ac %fc
    %bc %sc %rc %sq %bq %cc %cs %hrl %hsl
    [/sourcecode]

Home made formats

  • Logging HTTP Host header, the URL, the status code, number of bytes read from server and the server response time
    [sourcecode language=”text”]capture request header Host len 32
    log-format %hr %r %st %B %Tr
    [/sourcecode]
  • SSL log format with: HAProxy path (frontend, backend and server name), client information (source IP and port), SSL information (protocol version and negotiated cypher), connection termination state, including a few strings:
    [sourcecode language=”text”]log-format frontend:%f %b/%s client_ip:%Ci client_port:%Cp SSL_version:%sslv SSL_cypher:%sslc %ts[/sourcecode]

Links