Version 2.4 improves its e2e tests, revamps how logging in the HAProxy Data Plane API works, adds support for namespace filtering in Consul Service Discovery, improves runtime capabilities for maps and ACLs, adds server-template support and adds log_targets to global and defaults sections.

 

HAProxy Data Plane API version 2.4 continues efforts from the previous version to reduce complexity for users and add freedom to configure the Data Plane API logs to suit your environment, with an easy to understand API configuration. Our main goal is to fully support all of HAProxy’s configuration options, which we’ve made progress on by adding log_targets to global and default sections. Also, while the previous release added support for the resolvers section, this version adds support for the server-template line, which works with resolvers to enable DNS-based service discovery and scaling.

To help make using the API less complex, version 2.4 introduces the ability to add multiple map and ACL file entries as a single atomic command, if used with HAProxy 2.4 or greater. It leverages the prepare, add and commit commands in the HAProxy Runtime API.

Contributors also worked to bring the overall quality of the Data Plane API to a higher level, and that can be seen in a big overhaul of the project’s end-to-end tests and some tweaks to its continuous-delivery processes.

All of these changes improve usability and provide more freedom in configuring logging, while still continuing to progress closer to covering the full feature set of HAProxy. This wouldn’t be possible without the great effort of our community with some great contributions in code, bug reports and feature requests!

Logging rework

Before this version, you would configure logging with the --log CLI options or similar directives in the API’s configuration file. These methods let you record the API’s log, but only to a single location. It looked something like this:

That could be a little cumbersome, and not to mention that if you wanted to forward those logs to somewhere else you had to work your way around this on your own. Starting from 2.4, you have the ability to configure multiple log targets for the Data Plane API in the configuration file. For example, this configuration matches the previous CLI options:

— yaml—

—hcl—

Suppose you want to configure multiple log targets. Let’s say you want to see all of your debug-level logs on the terminal’s stdout, store info-level application logs in a JSON file, and send Apache formatted access logs to a dedicated syslog instance. You could do something like this:

—yaml—

—hcl—

See our HCL formatted example and YAML formatted example to learn how to configure logging to suit your needs.

What also changed is that there is now a distinction between the two types of log sources: application and access logs. Access logs were recorded on two lines that were a bit hard to follow when multiple API calls were made, one line representing the start of a request and one for completing it:

These access logs are now replaced with a single Apache Common Log formatted log line:

The default format uses the following HAProxy log variables: %h %l %u %t "%r %>s %b "%{Referer}i" "%{User-agent}i" %{us}T. However, you can customize it using the acl_format setting to better suit your needs.

All other logs, such as events recorded when the API starts, are considered application logs, and with the log_type you can define what type of logs you want to send to the configured log target.

Deprecation warning

The CLI options for logging and the older log section in the configuration file remain in place, but they are deprecated, meaning that in version 2.5 they will be removed. Currently, they are only used when no log_targets section is configured in the configuration file to retain users‘ environments and expected behavior. Once there are log_targets configured, only those are used. The CLI options and old log section is ignored in that case.

Extended HAProxy configuration keywords support

The HAProxy Data Plane API is continuously improving its coverage of supported HAProxy keywords with the goal of becoming simpler and easier to use. In that area, 2.4 brings a couple of improvements.

Support for server-templates

We’ve added a new API resource, /v2/services/haproxy/configuration/server_templates, to support the server-template keyword in the HAProxy configuration backend sections. This is a feature that has been requested multiple times and finally lands in the HAProxy Data Plane API!

Global and defaults log_targets

Previously, you could add /v2/services/haproxy/configuration/log_targets only to backends and frontends sections, which you specified using parent_type and parent_name query string parameters. With 2.4 you can add them to global or defaults sections too using parent_type=global or parent_type=defaults query string parameters. Note that you don’t need to specify parent_name in this case.

Minor changes

Version 2.4 adds several fields to the following resources:

/v2/services/haproxy/configuration/global:

  • Added hard-stop-after
  • Added localpeer
  • Added server_state_base
  • Added server_state_file

/v2/services/haproxy/configuration/defaults:

  • Added client_fin_timeout
  • Added server_fin_timeout

/v2/services/haproxy/configuration/backends:

  • Added uri_path_only to balance field

Extended HAProxy Runtime API commands support

In the HAProxy 2.4 Runtime API there is an ability to add multiple map and ACL file entries in an atomic way using a prepare, add and commit workflow. This feature allows the Data Plane API to add multiple entries to map_file and acl_file resources in one API call atomically without worrying about partial data inserts in cases of failure.

You can do that by using the PUT method on /v2/services/haproxy/runtime/acl_file_entries and /v2/services/haproxy/runtime/maps/{name} resources respectively.

Minor quality of life improvements

HAProxy Data Plane API 2.4 comes out with a batch of quality of life improvements for users and the contributing developers. It includes a series of bug fixes that have been backported to v2.3.6 already, the Consul Service Discovery now has the ability to filter services by namespace, and the Configuration-Version header is now returned even in the case of an error response.

To get to higher quality code and attract more contributors, the Data Plane API now has a more readable end-to-end testing suite, which will make first-time contributions easier and safer.

Contributors

Contributor Area
Amel Husic FEATURE CLEANUP BUG BUILD
Dario Tranchitella FEATURE BUG TEST
Georgi Dimitrov BUG
Goran Galinec FEATURE CLEANUP DOC
Marko Juraga FEATURE BUG BUILD DOC REORG TEST
Piotr Olchawa BUG
Robert Maticevic FEATURE
Zlatko Bratkovic FEATURE BUG BUILD CLEANUP

Conclusion

The HAProxy Data Plane API version 2.4 focuses on quality of life improvements and general usability. With the new logging system you can better monitor what is happening with the API, and easily include it in your logging systems. It further expands to cover more features from both the configuration file and the Runtime API, and shows an effort to be more maintainable and easier to contribute to with the revamped end-to-end testing suite. The API continues to get better thanks to the active community surrounding it. If you’d like to get involved, check out our GitHub repository to learn more!

SHARE THIS ARTICLE