One of HAProxy’s greatest strengths has got to be its logging system. The amount of information it provides can be invaluable when diagnosing unexpected behavior, studying your website’s traffic, or just getting a feel for how HAProxy works.

HAProxy comes with a few standard log formats that define which fields will be captured. Common wisdom says that you should add the option httplog configuration directive to your frontend or defaults section when using HAProxy as a Layer 7 proxy. Or, use option tcplog when using it as a Layer 4 proxy. Either one will give your logs an instant power-up, making them more comprehensive.

Sometimes, though, these premade settings are not exactly what you need. In that case, you can configure a custom log format. In this blog post, you’ll see how.

Customizing the HAProxy Logs

There are plenty of reasons why you might want to customize the fields captured by the HAProxy logs. For example:

• the default log format is giving you more information that you need
• you’re missing an important piece of information with the default log format
• you need to structure the fields in a way that an external tool can read them
• you rely on a standard log format and HAProxy must also comply

Luckily, it’s easy to add a template that says which fields to log and in which order. You just need to use the log-format directive. Let’s dive into its structure.

Adding a New HAProxy Log Format

The log-format directive goes into your defaults, frontend or listen section. It specifies a string that contains variables referring to the fields that you’d like to capture.

When you place this into your defaults section, it affects all of the proxy sections (frontend and listen) that follow. If you need a different format for a particular proxy, adding another log-format to that section will override the default.

In the next section, you’ll learn about the variables that are available.

Variables

A large part of your custom log format will likely be made up of variables that are pre-defined by HAProxy. Others you will define yourself.

In the broadest terms, all variables follows the rules below when added to a format string:

• It is preceded by a percent character: %
• It can take flags in braces {}
• If multiple flags, then they are separated by commas within the braces
• Flags may be added or removed by prefixing them with a + or a – sign
• Spaces, other than those between variables, must be escaped by a backslash

Currently, there are three flags:

• Q: quotes a string
• X: hexadecimal representation
• E: escapes characters (“, \, ]) with backslashes

Here are a few examples:

The pre-defined variables (also available in the documentation) are:

The default log formats use these variables in the following ways:

TCP log format:

HTTP log format:

CLF log format:

You can create your own format by using any of the pre-defined variables in the order that you wish. The following logs the client’s IP address and port, the server’s IP address and port, the path, and the response status:

If you prefer, you can add text to give the fields more context. Placing %{+Q}o at the beginning will apply the flag, in this case quotes, to all of the following strings.

You can also capture specific pieces of data not represented by the pre-defined variables. For example, you can log request headers. You’ll need to capture them first, though, and then refer to them by number. The numbers are incremented in the order in which the field was captured. Here are a few examples:

Custom variables can be logged by using the var fetch method. Note that you must prefix your variable names with txn.

Fetch methods can be included by placing them inside square brackets. In this example, the http_first_req fetch shows a 1 if this is the first request from the client, or a 0 if not.

Conclusion

In this blog post, you learned how to define a custom log format for your HAProxy logs. There are a number of pre-defined variables that you can use to build your messages. Or, you can add other data such as request headers, custom variables and explanatory text.

Want to keep up to date on similar topics? Subscribe to this blog. Also, follow us on Twitter and join the conversation on Slack! HAProxy Enterprise combines HAProxy with enterprise-class features and premium support. Contact us to learn more or sign up for a free trial today!