Join us for our live webinar on October 15th to explore the exciting new features of our HAProxy Data Plane API 3.0 release!
HAProxy Data Plane API 3.0 is now available! The latest version is hosted on our GitHub releases page.
This release follows the recent HAProxy 3.0 release and incorporates its changes, along with some improvements and changes specific to the API.
HAProxy Data Plane 3.0 adds multiple breaking changes. We'll cover the impacts of these changes in detail to highlight how your implementation and usage of Data Plane API may be affected. Despite some initial growing pains, v3 provides a stronger foundation for future development as we continue to modernize and simplify the API to meet users' needs. We recommend reading carefully through this upcoming section.
This release also makes key improvements to the API, which we'll also expand upon later:
Users can now fetch a parent resource with all children embedded.
We've added a PUT operation on index-based resources.
Timeouts are now handled with your preferred time suffix.
The API's state data is removed from the configuration file. User-defined settings are now the configuration's sole focus.
Support has been added for every new keyword introduced in HAProxy 3.0.
But that only scratches the surface. We're excited to show you the core functionality updates we've made in v3 and why they matter.
Breaking changes
This version introduces breaking changes described in this section.
Removed _version and data wrapper fields in responses
Each time the Data Plane API changes the load balancer configuration it increments the version of the file. This enables optimistic locking of the configuration to prevent a user from unintentionally overwriting another user's changes. In the initial design, each API endpoint returned JSON data with _version
and data
fields as top keys in the response object. Later, we also added a Configuration-Version
HTTP header, making the configuration version available in both the wrapped response and the header.
In v3, we've removed the redundant _version
and data
fields from the response and moved the JSON previously under data to the top—making it easier to parse. This simplifies the process of changing an existing resource. Users can leverage this flat structure to fetch the resource, change a few fields, and send back the same object without needing to unpack it from the data
field.
Renamed the defaults resource
In the early stages of the Data Plane API, we intended to generate and manage only one defaults
section within the HAProxy configuration by using the /v2/services/haproxy/configuration/defaults
endpoint. This decision to manage just one was based on the fact that support for assigning defaults
sections names—by which you could indicate which to inherit settings from—didn't originally exist. Assigning a name to a defaults
section, although valid syntactically, had no functional meaning in HAProxy.
However, HAProxy 2.4 introduced keywords to the frontend
and backend
sections, giving meaning to these names and allowing users to create multiple, named defaults
that frontends and backends could then inherit settings from. Since the defaults
resource initially returned an object instead of a list, we introduced a new resource called named_defaults
to maintain backward compatibility. This resource returned a list of defaults
resources that names and could be referenced in backends
and frontends
sections. We kept the original defaults
resource for backward compatibility as an object.
With v3 of the API, we're migrating named_defaults
to simply defaults
, breaking the old defaults
behavior. The defaults
resource now returns a list versus an object.
Moved child resources as nested resources
In v2, some resources represent lines within a section of HAProxy configuration, effectively making them child resources of the parent section resource. For example, the acl
resource can be a child of a frontend
or backend
, and a server
resource can be a child of a backend
parent.
You could previously fetch or update these resources in the API by identifying them with their index, which was the case for the acl
resource. You could do the same by name for other resources, such as the servers
resource. You'd then specify their parents using the parent_type
and parent_name
query-string parameters.
This process was unfortunately confusing, so we've redesigned child resources as nested resources in the URL. Because a child resource cannot simultaneously exist in multiple parent resources, it makes sense to include this in the parent resource's URL. This also boosts compatibility with many external tools. For example, you can build role-based access control (RBAC) rules more easily atop the new URL hierarchy without relying on query-string parameters.
Here's how the two implementations compare:
In v2, you'd see
/services/haproxy/configuration/acls?parent_type=frontend&parent_name=my-example-frontend
In v3, you'll instead see
/services/haproxy/configuration/frontends/my-example-frontend/acls
Removed explicit index field on index-based child resources
In v2 of the API, all index-based resources—which represent configuration lines that can be repeated within a section and are distinguished by the order of appearance—had an index
field indicating the position of that specific resource. For example, acl
resources get an index to indicate their order within the configuration.
We've removed that index
field in v3, as we found it redundant and complicated to maintain when positions change. You can infer the ordering from the actual position in the returned list.
Removed support for HAProxy in multi-process mode
HAProxy previously had a feature that allowed it to run in multi-process mode. This mode has since been deprecated and removed following the introduction of multi-thread mode. In v2 of the API, we supported all keywords related to multi-process mode. Additionally, we supported multiple runtime APIs (one for each process mode), so all /runtime
resources worked with multiple process support.
We've removed this support altogether in v3 alongside the following fields from the respective sections:
Backend resource:
bind_process
Bind resource:
process
Defaults resource:
bind_process
Frontend resource:
bind_process
Global resource:
nbproc
In addition to removing these fields, /runtime
endpoints have changed.
The stats resource at /v2/services/haproxy/stats/native
used to return an array of arrays. The top-level array held per-process stats, with each array element containing an array of stat lines for each object (server, backend, frontend). Now that multi-process mode support is ending, /v3/services/haproxy/stats/native
returns an object with a single stats array for the server, backend, or frontend object you've requested. Similarly, we removed one layer of arrays that returned process information on a per-process basis. This change affected /v2/services/haproxy/runtime/stick_table
such that the process
field is removed from responses that previously indicated from which process the stick table information was being fetched.
In addition to this, the query string parameter process
was removed from both /v2/services/haproxy/runtime/stick_tables
and /v2/services/haproxy/runtime/stick_table_entries
resources.
Reorganized global resource fields
One thing we set out to do with v3 was reorganize the global
resource fields. The global
resource is our largest if you measure by the number of fields, and it primarily had a flat structure. To improve readability of this resource on the API, we reorganized the fields into nested objects.
While making this change, we tried to adhere to HAProxy's field naming conventions and followed documentation hints when deciding how best to structure the revised fields. We tried to maintain only one level of nested objects to avoid unnecessary complexity. Some fields remain in the root level of the object, and some options are split into debug_options
and performance_options
as specified in the Configuration Manual.
We've also added an ssl_options
nested object to cover all SSL directives. Other objects like tune_ssl_options
, tune_vars_options
, and tune_zlib_options
are all derived from tune.*.*
options. Plus, the tune_options
object covers other tuning global directives and acts as a catch-all. Meanwhile, http_client_options
and fifty_one_degrees_options
are similarly derived from the manual's dot notation.
For the full and detailed specification of fields, please see the global model specification within our GitHub project.
Removed deprecated fields and resources
As a major version release, Data Plane API 3.0 gave us the chance to clean up keywords previously marked as deprecated. Here's more information about the keywords we've removed in v3:
Global section
Removed the deprecated
tune_ssl_default_dh_param
field that can now be found intune_ssl_options
asdefault_dh_param
Backend section
Removed the deprecated
http-check
field. You can now specify multiplehttp_check
resources on/v3/services/haproxy/configuration/backends/{parent_name}/http_checks
.Removed the
http-keep-alive
,http-server-close
, andhttpclose
fields as they're mutually exclusive and configurable using thehttp_connection_mode
field
HTTP request/response rules and TCP Request Rule resource
Removed the deprecated
track_sc<stick_counter>
field. In v2 we only supported 0, 1, and 2 stick counter indexes by hard-coding them in the field names. We've now added thetrack_sc_stick_counter
field (specifying the stick counter index) which can exceed 2 if configured using thetune.stick_counters
keyword in theglobal
section. It defaults to 3 and debuted in HAProxy 2.8.
Service discovery resources
Removed the deprecated
service-whitelist
andservice-blacklist
fields, which we've replaced withservice_allowlist
andservice_denylist
, respectively
Further API Improvements
We'll describe other notable improvements in this section.
Fetching a parent resource with all children embedded
While we've found that our granular approach to creating resources on the API is helpful, we're aware it can also add complexity. As a result, we've added an extra query string parameter full_section
on all the section resources such as frontend and backend resources. This lets you fetch, create, or replace a complete section with all of its child resources.
This query string parameter works on both PUT and POST endpoints, so you can now edit or create the whole parent section with just one API call.
Added a PUT operation to index based resources
Previously, change automation on a list of index-based resources was complex for several reasons. First, you had to reconcile all index fields in each resource. This made it difficult to locate a resource if something was deleted and added, as its index could have changed. Second, large lists with multiple changes forced you to make numerous API calls.
With v3, we added a PUT operation to the list endpoints, letting you automatically replace the entire list of resources.
Handle timeout options with preferred units
We handle timeout options as integers in the API, since we want to represent numeric values as numbers wherever possible. However, since HAProxy supports strings by accepting time unit suffixes, we recalculated the submitted values to the default unit in v2 (usually milliseconds) and stored them as integers in the configuration file. This was challenging for those who still use and read the raw configuration file, as it changed their preferred inputs and made the file less readable.
In v3, we've introduced a new option in the Data Plane API configuration file called preffered_time_suffix
where you can specify one of the options: nearest
, none
, ms
, s
, m
, h
, and d
. Data Plane API will then honor your preferred time suffix, which is nearest
by default, and calculate it to the smallest possible value with the corresponding suffix. When using none
, your timeout keywords in the configuration file are written without the suffix while using the default value for the respective keyword.
For further info, we've added some documentation to our Open API specification by annotating such fields with x-duration: true
and x-default-unit
to indicate the standard time unit for that specific field.
Keep Data Plane API config file unchanged
Since Data Plane API lacks its own storage, it previously used its configuration file to store state—notably for service discovery and user configurations. This proved problematic when using Data Plane API with automation software like Ansible, Puppet, and others.
Data Plane API will no longer write to its own configuration file from v3 onward. We've introduced dataplane_storage_dir
, where Data Plane API's state will be stored in specific JSON files for each feature. We chose JSON for easier readability and debugging.
Upon startup, state data will automatically migrate from the configuration file into JSON files located at /etc/haproxy/dataplane
. For information on where the data goes, view our brief Data Plane API internal storage Readme on GitHub.
Added support for all new keywords added in HAProxy 3.0
Following the latest release of HAProxy, we're extending the configuration keyword support to include many new features in the load balancer. These keywords impact security, persistence, streaming, directives, and more.
To view and understand the complete collection of keywords now available in HAProxy 3.0, check out our Reviewing Every New Feature in HAProxy 3.0 blog post.
Contributions
As always, we'd like to extend a massive thank you to everyone who helped accelerate Data Plane API's development and make v3 possible:
Contributor | Area |
Zlatko Bratkovic | BUG | BUILD | CLEANUP | FEATURE |
Olivier Duclos | BUG | FEATURE | REORG | TEST |
Helene Durand | BUG | BUILD | FEATURE | REORG |
Vincent Gramer | BUG | BUILD |
Libo Huang | BUG |
Andjelko Iharos | FEATURE |
Marko Juraga | BUG | BUILD | CLEANUP | FEATURE | TEST |
Dinko Korunic | FEATURE | OPTIM |
Aurélien Maigret | BUG |
Robert Maticevic | BUG | BUILD | CLEANUP | FEATURE | OPTIM | TEST |
Fabiano Parente | BUG |
Dario Tranchitella | BUG | REORG |
Csaba Tűz | FEATURE |
George Vine | BUG | FEATURE |
Data Plane API 3.0 constitutes a major step forward in functionality and maintainability for Data Plane API. We're excited to bring you even more features and enhancements soon in response to your feedback.
For more information including installation and upgrade instructions, visit our Data Plane API documentation. You can also view our Data Plane API releases section on GitHub for updated release notes and a changelog.
Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.