Release notes

Version 3.1r1 Jump to heading

Key changes in the HAProxy Enterprise 3.1r1 release include:

ADFSPIP module

The new ADFSPIP module enables HAProxy Enterprise to give external clients access to web applications running in a Windows corporate network. HAProxy Enterprise becomes a proxy in front of Microsoft Active Directory Federation Services and web applications running inside the corporate network. It can route external clients to an AD FS sign-in page and to the web applications that they would otherwise not be able to reach.

Captcha module

New Captcha module options give you better control over the cookie that the module creates in the user’s browser. You can set the cookie’s domain, expiration, max age, path, SameSite option, and Secure option.

Global Profiling Engine

This version updates the Global Profiling Engine, which typically runs on a server separate from HAProxy Enterprise, to have new logging capabilities. You can send log messages to a file, a local syslog UNIX domain socket, or a remote syslog server.

UDP module

When configuring the UDP module via its udp-lb configuration section, you can now set the hash-type and hash-balance-factor directives to control how hash-based load balancing behaves with UDP traffic.

Stick table aggregator

For customers who have not migrated to the Global Profiling Engine, HAProxy Enterprise 3.1 introduces the new Stick Table Aggregator package version 2.1. It has the following enhancements:
- show aggrs now supports multiple buffers, ensuring all data is visible instead of just the first chunk.
- Active connections to down peers are deactivated by default, reducing unnecessary overhead. The legacy behavior can still be enabled with the legacy-active-connect argument.
- A new no-ascend option prevents data from being sent to up peers in multilayer environments.
- Multiple from lines are now supported per aggregation, offering greater flexibility in defining aggregation sources.
- The module now properly handles previously unsupported stick table data types.

Logging

The way that HAProxy Enterprise emits its logs is more flexible now with the introduction of log profiles, which let you assign names to your log formats. By defining log formats with names, you can choose the one best suited for each log server and even emit logs to multiple servers at the same time, each with its own format. Log profiles also allow you to write log messages when different events occur, such as when accepting a connection, receiving a request, connecting to a backend server, and receiving a response.
With the new do-log action, you can emit custom log messages throughout the processing of a request or response, allowing you to add debug statements that help you troubleshoot issues. Add the do-log action at various points of your configuration.
The option tcplog directive now allows an optional argument: clf. When enabled, CLF (Common Log Format) sends the same information as the non-CLF option, but in a standardized format that CLF log servers can parse.

set-retries action

The tcp-request content and http-request directives have a new action named set-retries that dynamically changes the number of times HAProxy Enterprise will try to connect to a backend server if it fails to connect initially.

quic-initial directive

The new quic-initial directive, which you can add to frontend, listen, and named defaults sections, gives you a way to deny QUIC (Quick UDP Internet Connections) packets early in the pipeline to waste no resources on unwanted traffic.

SPOE

In your configuration file, set mode to spop in the backend that contains the SPOE agents. This mode is now mandatory and automatically set for backends referenced by SPOEs. Configuring your backend in this way means that you are no longer required to use a separate configuration file for SPOE. This new spop backend mode adds flexibility to SPOE, optimizes traffic distribution among servers, improves performance, and will ultimately make the entire system more reliable, as future changes to the SPOE engine will only affect pieces specific to SPOE.
You can now pass variables from the main stream that’s processing a request to the child stream of a Stream Processing Offload Agent (SPOA). When reading a variable in an spop backend, prefix the variable scope with the letter p for parent stream. For example, instead of req, use preq. This works for these scopes: psess, ptxn, preq, and pres.
The following SPOE parameters were removed in this version and are silently ignored when present in the SPOE configuration:
- maxconnrate
- maxerrrate
- max-waiting-frames
- timeout hello
- timeout idle

Health checks

When defining health checks, add the new init-state argument to a server directive or server-template directive to control how quickly each server can return to handling traffic after restarting, coming out of maintenance mode, or adding the server through service discovery.
The option httpchk directive supports a new parameter for passing a host HTTP header with each health check to backend servers. This avoids having to specify carriage return and newline characters to do it, unlike previous versions.

Address families

When setting an address on a bind line, you can now set these address family prefixes:

Prefix	Description
`abnsz@`	Zero-terminated abstract namespace. This lets you interconnect with software that determines the length of the namespace’s name by the length of the string, terminated by a null byte.
`mptcp@`, `mptcp4@`, `mptcp6@`	MultiPath Transmission Control Protocol (MPTCP). MPTCP is a TCP extension that improves resource utilization, increases throughput, and responds quicker to failures.

Fetch methods

This version introduces the following fetch methods:

Name	Description
`ssl_c_san`	Returns a string of comma-separated Subject Alt Name fields contained in the client certificate.
`ssl_fc_sigalgs_bin`	Returns the content of the `signatures_algorithms` (13) TLS extension presented during the Client Hello.
`ssl_fc_supported_versions_bin`	Returns the content of the `supported_versions` (43) TLS extension presented during the Client Hello.
`last_entity`	Returns the identity of the last entity that was evaluated during stream analysis.
`waiting_entity`	Returns the identity of the entity that was waiting to continue its processing when an error or a timeout was encountered.

Converters

This version introduces the following converters:

Name	Description
`date`	Converts an HTTP date string to a UNIX timestamp.
`rfc7239_nn`	Converts an IPv4 or IPv6 address to a compliant address that you can use in the `from` field of a `Forwarded` header. The `nn` here stands for node name. You can use this converter to build a custom `Forwarded` header.
`rfc7239_np`	Converts an integer into a compliant port that you can use in the `from` field of a `Forwarded` header. The `np` here stands for node port. You can use this converter to build a custom `Forwarded` header.
`when`	Enables you to pass data, such as debugging information, only when a condition is met, such as an error condition.

Runtime API

This version makes the following changes to the Runtime API:

The debug counters command shows all internal counters placed in the code. Primarily aimed at developers, these debug counters provide insight for analyzing glitch counters and counters placed in the code using the new COUNT_IF() macro. Developers can use this macro during development to place arbitrary event counters anywhere in the code and check the counters’ values at runtime using the Runtime API. For example, glitch counters can provide useful information when they are increasing even though no request is instantiated or no log is produced.
The dump ssl cert command will display an SSL certificate directly in PEM format; useful for placing delimiters and saving a certificate when it was updated on the CLI and not on the filesystem yet. You can also dump a transaction by prefixing the filename with an asterisk. This command is restricted and can only be issued on sockets configured for level admin.
The echo command with syntax echo <text> will print what’s contained in <text> to the console output; it’s useful for writing comments in between multiple commands.
This version improves the show dev Runtime API command by printing more information about arguments provided on the command line as well as the Linux capabilities set at process start and the current capabilities (the ability to preserve capabilities was introduced in Version 2.9 and improved in Version 3.0). This information is crucial for engineers troubleshooting the product.
The show env command, which dumps environment variables known to the process, can now show information for a specific environment variable.
The show quic command produces more internal information about the internal state of the congestion control algorithm and other dynamic metrics (such as window size, bytes in flight, and counters).
The show info command will now report the current and total number of streams. It can help to quickly detect if a slowdown is caused on the client side or the server side and facilitate the export of activity metrics.

Troubleshooting

This release includes a number of troubleshooting and debugging improvements:

Starting in version 3.1, traces get a dedicated configuration section named traces, providing a better user experience compared to previous versions. Traces report more information than before, too.
The new when converter adds a condition to the end of a fetch method to only fetch the data when the condition is true, such as an error condition.
As of version 2.5, you can use the sample fetches fc_err_* for frontends and bc_err_* for backends to help determine the cause of an error on the current connection. In this release, these fetches have been enhanced to include connection-level errors that occur during data transfers.
The system may produce a core dump on a fatal error or when the watchdog fires, which detects deadlocks. While crucial to diagnosing issues, sometimes these files are truncated or can be missing information vital to analysis. This release includes an internal post_mortem structure to be included in core dumps, which contains pointers to the most important internal structures. This structure, present in all core dumps, allows developers to more easily navigate the process’s memory, reducing analysis time, and prevents the user from needing to change their settings to produce different debug output. Additionally, more hints have been added to the crash output to help in decoding the core dump. To view this debugging information without producing a core dump, use the improved show dev command.
In previous versions, sometimes stderr outputs of the thread backtraces in core dumps would be missing, or only the last one was present due to the reuse of the same output buffer for each thread. Core dumps now include backtraces for all threads, as each thread’s backtrace is now dumped in its own buffer. Also present in core dumps as of this version are the output messages for each thread, which assists developers in determining the causes of issues even when debug symbols are not present.
This version includes improvements to HAProxy’s watchdog, which detects deadlocks and kills runaway processes. The watchdog will now watch for stuck threads more often, by default every 100ms, and it will emit warnings regarding a stuck thread’s backtrace before killing it. It will stop the thread if, after the first warning, the thread makes no progress for one second. In this way, you should see ten warnings about a stuck thread before the watchdog kills it. Note that you can adjust the time delay after which HAProxy Enterprise will emit a warning for a stuck thread using the global debugging directive warn-blocked-traffic-after. We do not advise that you change this value, but changing it may be necessary during a debugging session.
This version enhances the accuracy of the memory profiler by improving the tracking of the association between memory allocations and releases and by intercepting more calls such as strdup() as well as non-portable calls such as strndup() and memalign(). This improvement in accuracy applies to the per-DSO (dynamic shared object) summary as well, and should fix some rare occurrences where it incorrectly appeared that there was more memory free than allocated. New to this version, a summary is provided per external dependency, which can help to determine if a particular library is leaking memory and where.

Performance improvements

HAProxy Enterprise 3.1r1 adds these performance improvements:

The H2 mux is significantly more performant in this version. This was accomplished by optimizing the H2 mux to wake up only when there are requests ready to process, saving CPU cycles, and resulting in using 30% fewer instructions on average when downloading. The POST upload performance has been increased up to 24x with default settings and it now also avoids head-of-line blocking when downloading from H2 servers. Two new global directives, tune.h2.be.rxbuf and tune.h2.fe.rxbuf allow for further tuning of this behavior. Specify a buffer size in bytes using tune.h2.fe.rxbuf for incoming connections and tune.h2.be.rxbuf for outgoing connections. For both uploads and for downloads, one buffer is granted to each stream and 7/8 of the unused buffers is shared between streams that are uploading / downloading, which is the mechanism that significantly improves performance.
New to this version are two new global directives for tuning QUIC performance:
- tune.quic.cc.cubic.min-losses takes a number that defines a threshold for how many packets must be missed before the Cubic congestion control algorithm determines that a loss has occurred. This setting allows the algorithm to be slightly more tolerant to false losses, though you should exercise caution when changing the value from the default value of 1. A value of 2 may prove to show some performance improvement, though we do not recommend running this way for extended periods of time, only for analysis, and you should avoid providing a value larger than 2.
- tune.quic.frontend.default-max-window-size defines the default maximum window size for the congestion controller of a single QUIC connection, by specifying an integer value between 10k and 4g, with a suffix of “k”, “m” or “g”.
This version sees an efficiency improvement in regards to the QUIC buffer allocator and using this tunable, you are able to vary the size of the memory required per-connection, thus reducing overallocation.
The transmission path for QUIC has been significantly improved in this version so that it will now adapt to the current send window size and will use Generic Send Offload to let the kernel send multiple packets in a single system call. This offloads processing from HAProxy Enterprise and the kernel and places it onto the hardware. This is especially meaningful when used on virtual machines where system calls have potential to be expensive.
Small frames for the QUIC buffer handling now use small buffers. This improves both the memory and CPU usage, as the buffers are now more appropriately sized and do not require realignment. QUIC will always send a NEW_TOKEN frame to new clients for reuse in the next connection. This behavior permits clients to reconnect after being validated without going through the address validation process again on the next connection. In other words, the next established connection will improve network performance when a listener is attacked or when dealing with a lossy network.
To help improve performance in the case of large configurations that consume a lot of CPU on reload, two global configuration directives tune.renice.startup and tune.renice.runtime are new to this version. These global directives take a value between -20 and 19 to apply a scheduling priority to configuration parsing.
TCP logs saw a 56% performance gain in this version thanks to the implementation of the line-by-line parser into the TCP log forwarder.
In regards to log servers, the ring sending mechanism sees improvement in this version, as the load is better balanced across available threads, assigning new server connections to threads with the least load. You can now use the max-reuse directive for TCP connections served by rings. When used for this reason, the sink TCP connection processors will not reuse a server connection more times than the indicated maximum. This means that connections to the servers will be forcefully removed and re-created, which helps to better distribute the load across available threads, thus increasing performance. Make sure that when using this directive that the connections are not closed more than a couple of times per second.
In previous versions, some users may have seen intense CPU usage by the pattern LRU cache when performing lookups with low cardinality. To remedy this, in this version the cache will be skipped for maps or expressions with patterns with low cardinality, that is, less than 5 for regular expressions, less than 20 for others. Depending on your setup, you could see a savings of 5-15% CPU in these cases.
As of this version, configured servers for backends are now properly indexed, which saves time in detecting duplicate servers. As such, the startup time for a configuration with a large number of servers could see a reduction of up to a factor of 4.
Variables have been moved from a list to a tree, resulting in a 67% global performance gain for a configuration including 100 variables.
We saw a performance gain of, on average, 7% regarding arithmetic and string expressions by removing the need for trivial casts for samples and converters of the same types.
The Lua function core.set_map() has doubled its performance in speed by avoiding duplicate lookups.
This version includes a performance gain regarding smoother reloads for large systems, that is, systems requiring a large number of file descriptors and a large number of threads. This gain is due to how file descriptors are handled on boot, shortening initialization time from 1.6s to 10ms for a setup with 2M configured file descriptors.
The master-worker mode was heavily reworked in this version to improve stability and maintainability. Its previous architecture model proved difficult in maintaining forward compatibility for seamless upgrades; the rework aims to remedy this problem. Per the new model, the master process does nothing after starting until it confirms the worker is ready, and it no longer re-executes itself to read the configuration, which greatly reduces the number of potential race conditions. The configuration is now buffered once for both the master and worker and as a result will be identical for both. As such, environment variables shared by both will be more consistent, and the worker will be isolated from variables applicable to the master only. This all improves the separation between the processes. An additional improvement is that this rework will reduce file descriptor leaks across the processes as they are now better separated. All of this to say: you should not notice anything as a result of this change except for improved reliability.

Deprecated features

The program section is deprecated in HAProxy Enterprise 3.1 and will no longer be supported starting HAProxy Enterprise 3.3.
The configuration options accept-invalid-http-request and accept-invalid-http-response are deprecated. Instead, use accept-unsafe-violations-in-http-request and accept-unsafe-violations-in-http-response.
Duplicate names in various families of proxies, for example, frontend, listen, backend, defaults, and log-forward sections, and between servers, are detected and reported with a deprecation warning, specifying that the duplicate names will not be supported in HAProxy Enterprise 3.3.
The legacy C-based mailers are deprecated and will be removed in HAProxy Enterprise 3.3. Set up mailers using Lua mailers instead.

Version 3.0r1 Jump to heading

Key changes in the HAProxy Enterprise 3.0r1 release include:

HAProxy Enterprise WAF

Support for base64 decoding when defining custom rules.
WAF related Prometheus exporter metrics.
Support for atomic transactions via the Runtime API.
Process request body without Content-Type, even in “raw” mode.
Validate incoming data to ensure it contains valid UTF-8 characters, filtering out potentially harmful characters that may attempt to exploit vulnerabilities.

Global Profiling Engine

New dynamic peer support and better memory management. It’s no longer mandatory to list peers in the Global Profiling Engine configuration file; they are learned dynamically as it runs. The dynamic peers result in a lower memory footprint from peer reuse and session reuse.
New global bind directive and global-exp argument on the from directive.
Support for the new HTTP/2 protocol glitch stick table counters, glitch_cnt and glitch_rate.

Bot management module

New botmgmt-profile section, scoring algorithms, and sample fetches to tune the module and increase observability.
Export Prometheus metrics from bot management data.
The bot management system emits a warning if the data-hapee data file is older than 4 weeks instead of 2 weeks.

UDP module

Basic logging can be enabled by specifying the log directive and its arguments within the udp-lb section. The log output format only contains the source and destination addresses, bytes received or sent, the instance name, and the server, if available.
Simplified configuration with the support of the default-server directive.

DeviceAtlas module

The DeviceAtlas module has been updated to support the DeviceAtlas v3 data format.

51Degrees module

The 51Degrees module now supports the 51Degrees V4 data format.

Security Assertion Markup Language (SAML) module

The new SAML module acts as a SAML Service Provider, providing single sign-on (SSO) to any web application located behind an HAProxy Enterprise server. This updated module simplifies the SSO configuration.

Captcha module

A new Captcha module supports Google reCAPTCHA v2, v3, and reCAPTCHA Enterprise, plus hCaptcha, Friendly Captcha, and Cloudflare Turnstile.

SSL/TLS Enhancements

SSL performance is improved as a result of switching from OpenSSL 3.X to OpenSSL 1.1.1 as the default.
The new crt-store configuration section provides a flexible way to store and consume SSL certificates by separating the certificate storage from their use in a frontend. This provides better visibility for certificate information by moving it from external files and placing it into the main HAProxy Enterprise configuration. The crt-store section allows you to individually specify the locations of each certificate component; for example certificates files, key files, and OCSP response files. Aliases provide support for human-friendly names for referencing the certificates more easily on bind lines.
A new global ssl-security-level directive allows you to set a global OpenSSL internal security level. This enforces the appropriate checks and restrictions per the level specified.
Explicitly set default TLS certificates.
- You can now set multiple default certificates using the new default-crt argument on your bind line. You can also indicate a default certificate in a crt-list by marking it with an asterisk.
Decrypt TLS 1.3 packet captures to backend servers.
- New to this version, you can now enable the logging of TLS secrets for TLS connections between HAProxy Enterprise and backend servers. This allows you to then use those TLS secrets to use Wireshark to decipher your encrypted packet captures.

Dependency on libsystemd removed

This version hardens HAProxy Enterprise’s security with the removal of the dependency on the libsystemd library. This libsystemd library has many additional dependencies of its own, including the library at fault for the XZ Utils backdoor vulnerability. Removing the dependency on libsystemd means that HAProxy Enterprise isn’t exposed.

Prevent HAProxy from accepting traffic on privileged ports

Two new global settings, harden.reject-privileged-ports.tcp for TCP connections and harden.reject-privileged-ports.quic for QUIC connections, enable HAProxy Enterprise to ignore traffic when the client uses a privileged port (0 - 1023) as their source port. Clients using this range of ports are suspicious and such behavior often indicates spoofing, such as to launch DNS/NTP amplification attacks. The benefit in using these settings is that during DNS/NTP amplification attacks, CPU is reserved as HAProxy Enterprise will drop the packets on these privileged ports instead of parsing them.

HTTP/2 tuning options

You can limit the number of total streams processed per incoming connection for HTTP/2 using the global directive tune.h2.fe-max-total-streams. Once this limit is reached, HAProxy Enterprise will send a graceful GOAWAY frame informing the client that it will close the connection after all pending streams have been closed. Usually, this prompts clients to reestablish their connections. This is helpful in situations where there is an imbalance in the load balancing due to clients maintaining long-lived connections.
HAProxy Enterprise now counts protocol “glitches” and allows you to set a limit on them. Protocol glitches are HTTP/2 requests that are valid but have potential to cause issues; for example, sending a single header as a large number of CONTINUATION frames could cause a denial of service. You can also track the glitches in a stick table to identify buggy applications or misbehaving clients.

Diagnostic mode

Diagnostic mode (-dD) will now report on likely problematic ACL pattern values that look like known ACL/sample fetch keywords in addition to its other warnings about suspicious configuration statements. It does not prevent startup. This is helpful for troubleshooting problematic configurations.

Consistent hash server mapping

When load balancing using a hash-based algorithm (balance hash), HAProxy Enterprise must keep track of which server is which. The new hash-key directive allows you to specify how the node keys, that is, the hash for each server, are calculated. This applies to node keys where you have set hash-type to consistent. What this means is that if you have multiple instances of HAProxy Enterprise, each with its own list of servers potentially listed in different orders, routing will be consistent across your load balancers when you use the hash-key directive. There are three options for hash-key:
- id: the server’s numeric id if set, or its position in the server list
- addr: the server’s IP address
- addr-port: the server’s IP address and port

Memory limit

The memory limiting mechanism now uses RLMIT_DATA instead of RLIMIT_AS, which prevents an issue where the processes were reaching out-of-memory conditions below the configured limit when using -m <limit> to limit the total allocatable memory to a specified number of megabytes across all processes. In master-worker mode, the memory limit is applied separately to the master and its forked worker process, as memory is not shared between processes.

Unix sockets and namespaces

This version supports the namespace argument for UNIX sockets specified on bind and server lines. If a permissions issue arises, HAProxy Enterprise will log a permissions error instead of “failed to create socket”, since the capability cap_sys_admin is required for using namespaces for sockets.

gRPC

Prior to this version, cancellations, which are represented as RST_STREAM frames in HTTP/2 or as STOP_SENDING frames in HTTP/3, were not being properly relayed to the server. This has been fixed. Furthermore, this version adds new fetch methods that indicate whether the stream was canceled (aka aborted) and the error code.

Silence warnings on deprecated directives

If you want to use deprecated directives and silence warnings, you must also use the new expose-deprecated-directives global directive. This directive applies to deprecated directives that don’t have alternative solutions.

Emergency memory allocation

The mechanism for emergency memory allocation in low memory conditions has been improved for HTTP/1 and applets. Forward progress in processing is guaranteed as tasks are now queued according to their criticality and which missing buffer they require.

Write HAProxy Enterprise logs as JSON or CBOR

You can now format log lines as JSON and CBOR. When configuring a custom log format, you will indicate which to use, and then in parentheses set the key for each field.

Virtual and optional ACL and Map files

HAProxy Enterprise now supports virtual ACL and virtual map files.
- A virtual file doesn’t exist on disk and exists only in memory. The load balancer creates it as an in-memory representation during startup. Virtual files will not persist after restarting the load balancer.
- When you mark a file as optional, the load balancer will check for it on the filesystem, but if it doesn’t find it, it will assume that the file is virtual. Optional files are useful for allowing startup without a file on disk, but saving the file to disk at a later time for long-term persistence.
- You can use virtual and optional files in cases where you’ll use the Runtime API to dynamically update the values.

Report the key that matched in a map file

New map_*_key converters return the key that was matched, rather than the associated value, making it easier to view the reason why the load balancer rerouted a request or took some other action.

Track specific HTTP errors

Two global directives http-err-codes and http-fail-codes let you specify which HTTP status codes to track when using the http_err_cnt, http_err_rate, http_fail_cnt, and http_fail_rate stick table data types.

Stick table pointers

Prior to this version, you could use the Runtime API to remove or update a record in a stick table based on its key. You can now specify the record’s unique ID (its pointer) instead of its key to update or delete that record. Use the show table command to retrieve the pointer.

Set fwmark on packets to clients and servers

HAProxy Enterprise now supports setting an fwmark on connections to backend servers as well as to clients connected on the frontend using the set-fc-mark and set-bc-mark actions. These replace the set-mark action, which had applied only to frontends. Setting an fwmark on an IP packet classifies it so that, for example, it can use a specific routing table.

Set traffic priority

New in this version, HAProxy Enterprise can modify the header of an IP packet to include the Differentiated Services (DS) field on connections to backend servers in addition to frontend connections to clients. Use the set-fc-tos and set-bc-tos actions (referring to the old Type of Service header field, which has been superseded by DS).

Weights in log backends

In this version, you can set weights on server lines in your mode log backends. mode log is used for load balancing the Syslog protocol.

Support for UUIDv7 identifiers

Generate universally unique identifiers that use the UUIDv7 format by using the unique-id-format directive with the uuid fetch.

OCSP

You can set an HTTP forward proxy to reach the Internet to connect to the OCSP server. Add the ocsp-update.httpproxy global directive to indicate the proxy’s address. This is useful in cases where you’re operating in an air-gapped environment where the HAProxy Enterprise server does not have direct access to the Internet, and therefore can’t connect to the OCSP server.
The tune.ssl.oscp-update global directive has been renamed oscp-update, as an OCSP update is unrelated to SSL tuning.

Reverse HTTP

A new directive named pool-conn-name provides more flexibility when using reverse HTTP, allowing you to set the name to match with an expression of fetch methods and converters, instead of matching via SNI.

Prometheus

When configuring the Prometheus exporter, you can include a URL parameter named extra-counters that enables additional counters that provide information related to the HTTP protocol, QUIC, and SSL.

Runtime API

The new wait command for the Runtime API will wait for some specified time before then performing the command that follows it. You could use this to collect metrics on a certain interval, for example, or in cases where you need to wait until a server becomes removable (the server has been drained of connections) before running additional commands, such as del server.
A reload will no longer reset the HAProxy Stats page, as long as you call the new Runtime API command dump stats-file first to save the current state to a file and then load that file with the stats-file global configuration directive. Upon reload, the new processes set their counters to the previous values (as they appear in the stats file). Only proxy counters are supported which includes values for frontends, backends, servers, and listeners. Ensure that you’ve set a GUID on each frontend, backend, listen, and server object by using the new guid directive. Only objects to which you have assigned a GUID will have their stats persisted across the reload.
Static cookies for session persistence are now supported for dynamically added servers. Dynamic servers refer to servers that don’t have an explicit entry within your HAProxy Enterprise configuration file. They are dynamic in the sense that you can add them programmatically using Runtime API commands. Dynamic servers are valuable in cases where you may have many servers that scale with traffic: when traffic loads are high, you add more servers, and when traffic loads diminish, you remove servers. You can now use the add server command to add the server and specify its static cookie using just one command.
Running multiple Runtime API commands separated by new lines is no longer supported.
- Previously, it was occasionally possible to successfully issue multiple commands by separating them with newlines, which had the potential to produce unexpected results for long-running commands that may only partially complete. A warning will now be emitted when a \n is detected in a command and the command will not be accepted. This change has also been backported to ensure that user scripts that utilize this behavior can be remedied.
- Run multiple Runtime API commands with semicolons instead of new lines.

Lua

Single-threaded Lua scripts using lua-load will see a performance improvement from changes to the thread yielding behavior.
- Use tune.lua.forced-yield to tune the thread yielding behavior. For scripts that use lua-load, the optimal (and default) value was found to be the maximum of either 500 instructions or 10000 instructions divided by the number of threads. As for scripts loaded using lua-load-per-thread, in cases where more responsiveness is required, the value can be lowered from the default of 10000 instructions. In cases where the results of the Lua scripts are mandatory for processing the data, the value can be increased, but with caution, as an increase could cause thread contention.
By default HAProxy Enterprise prevents thread and process creation after it starts. This is particularly important executing Lua files of uncertain origin. Generally speaking, it’s a best practice to make sure that no unexpected background activity can be triggered by traffic. However, this may prevent external checks from working, and it may break some very specific Lua scripts which actively rely on the ability to fork.
- This global directive insecure-fork-wanted disables this protection. As of this version, you can also activate this option by using -dI (-d uppercase “i”) on the HAProxy Enterprise command line. Note that it’s a bad idea to disable it, as a vulnerability in a library may be easier to exploit once disabled.
- In addition, forking from Lua, or anywhere else, is not reliable, as the forked process could embed a lock set by another thread and cause operations to never cease execution. As such, we recommend that you use this option with extreme caution, and that you move any workload requiring such a fork to a safer solution (such as using agents instead of external checks).

QUIC

The QUIC protocol is supported by default because we’re distributing a QUIC-compatible OpenSSL library with the load balancer package, rather than relying on the library that comes standard with the operating system.
Two new global settings improve QUIC performance:
- Use tune.quic.reorder-ratio to change how quickly HAProxy Enterprise detects outgoing packet losses. Lowering the ratio lowers the number of packets that can be missing ACKs before HAProxy Enterprise takes action. The default value is 50%.
- Use tune.quic.cc-hystart to enable the use of the HyStart++ (RFC9406) algorithm instead of the Cubic algorithm. The HyStart++ algorithm provides an alternative to the TCP slow start phase of the congestion control algorithm and may show better recovery patterns regarding packet loss.

Invalid request targets rejected for non-compliant URIs

Parsing is now more strict during HTTP/1 processing for request target validation. This means that where previously, for compatibility, non-standard-compliant URIs were forwarded as-is for HTTP/1, now some invalid request targets are rejected with a 400 Bad Request error. The following rules now apply:
- The asterisk-form is now only allowed for OPTIONS and OTHER methods. There must now be only one asterisk and nothing more.
- The CONNECT method must have a valid authority-form. All other forms are rejected.
- The authority-form is now only supported for the CONNECT method. Origin-form is only checked for the CONNECT method.
- Absolute-form must have a scheme and a valid authority.

Performance upgrades

Applets, such as the cache, can also take advantage of the fast forwarding mechanism introduced in version 2.9 for TCP, HTTP/1, HTTP/2, and HTTP/3. This enhancement avoids queuing more data when the mux buffer is full resulting in significantly less memory usage. Disable this behavior using tune.applet.zero-copy-forwarding for applets only, or tune.disable.zero-copy-forwarding globally. In regards to QUIC, simplification of the internal send API resulted in removal of one buffer copy. In regards to QUIC, the fast forwarding now considers the flow control, which reduces the number of thread wakeups and optimizes packet filling. The HTTP/1 mutex now also supports zero-copy forwarding for chunks of unknown size. For example, chunks whose size may be larger than the buffer.
Performance improved for ebtree on non-x86 machines. This results in approximately 3% faster task switching and approximately 2% faster string lookups.
Configuration parsing time will see an improvement thanks to a change in server name lookups. The lookups now use a tree, which improves lookup time, whereas before the lookup was a linear operation.
An improvement to traces, made possible by the implementation of near lockless rings, will enable their use on servers with moderate to high levels of traffic without risk of server impact.
Stick tables have received a performance boost due to a change in the locking mechanism.
When defining a dynamic server, use of the enabled argument is now rejected with an error, whereas previously it was only silently ignored.

Fetch methods

New fetch methods introduced in this version expose data that you can use in ACL expressions:

Name	Description
`bc_be_queue`	The number of streams de-queued while waiting for a connection slot on the target backend.
`bc_glitches`	The number of protocol glitches counted on the backend connection.
`bc_nb_streams`	The number of streams opened on the backend connection.
`bc_srv_queue`	The number of streams de-queued while waiting for a connection slot on the target server.
`fc_glitches`	The number of protocol glitches counted on the frontend connection.
`bc_settings_streams_limit`	The maximum number of streams allowed on the backend connection.
`fc_nb_streams`	The number of streams opened on the frontend connection.
`fc_settings_streams_limit`	The maximum number of streams allowed on the frontend connection.
`txn.redispatched`	True if the connection has experienced redispatch.

Version 2.9r1 Jump to heading

Key changes in the HAProxy Enterprise 2.9r1 release include:

Bot management module

The new HAProxy Enterprise Bot Management Module uses reputational signals and scoring based on our own large, real-world datasets and data science to identify traffic accurately. The module will categorize it as human, suspicious, bot, verified crawler (search engines), or verified bot/tool/app (non-browser). You can then combine this information with other enterprise security features, such as rate limiting and WAF, to create customized bot management policies.

UDP module

A new enterprise module adds the ability to load balance the UDP protocol in a fast, reliable, and secure way.

HAProxy Enterprise WAF

This version introduces a new deployment pattern for the Web Application Firewall:
- We are deprecating the names Advanced WAF and ModSecurity WAF.
- The Advanced WAF with its improved WAF ruleset is now called the Intelligent WAF Engine. This will be the recommended WAF mode.
- ModSecurity WAF is now called OWASP Core Rule Set, and we recommend using it only if you require ModSecurity. When used in combination with the Intelligent WAF Engine, only traffic that the Intelligent WAF Engine marks suspicious will go through the OWASP Core Rule Set. By sending less traffic to the CRS filter, we get faster performance without sacrificing industry-leading accuracy.
- Managing the HAProxy Enterprise WAF is easier than it had been with the Advanced WAF, due to the design of the Intelligent WAF Engine, which has been built from our deep expertise in security and data science so that it requires far less tuning.

Global Profiling Engine

A new argument named write-to on a stick table definition allows you to store session persistence tracking data and aggregated data in the same stick table. Previously, trying to do this would not work.

Diagnostic archive

This version adds a method for getting a diagnostic archive that will assist Support with troubleshooting.

Performance upgrades

The latest HTTP implementation in HAProxy Enterprise uses less memory and is 40-60% more CPU-efficient for large data transfers. This is due to faster recycling of buffers, which makes zero-copy forwarding operations more likely. To disable this, set the global directive tune.disable-zero-copy-forwarding.
The locking code got an upgrade, leading to a performance gain of 2-4% on x86 systems and 14-33% on Arm systems.
Measures to reduce contention on shared pools improved performance for HTTP/2 and HTTP/3. The HTTP/2 request rate improved from 1.5M to 6.7M requests/second on 80 threads without the shared cache, and from 368k to 4.7M requests/second with the shared cache.
In the log-forward section, the lock used in the part of the code related to updating the log server’s index during sampling has been replaced with atomic operations to reduce bottlenecks and facilitate ~4x faster log production.
Improvements to the caching storage mechanism to reduce its lock contention on large systems allow it to scale much more linearly with threads.
QUIC now uses less memory by releasing resources associated with connections winding down earlier. This improves performance at the termination of a connection when there’s no data left to send.
The http-request set-map, http-response set-map, and http-after-response set-map directives perform better in this version. The improvement comes from the search reference now being implemented as a tree, making map updates more efficient.
Thread locking code for stick tables was refined by disentangling lookups from peer updates, and improving some undesired cache line sharing.

Health checks

Health checks, which in version 2.7 were changed to run on a single thread for better performance, are now more proactive about handing off work to a less busy thread. The new global directive tune.max-checks-per-thread lets you set the number of active checks per thread, above which the thread will try to offload the work. The health check thread will also try to find an existing TLS session to the backend server if it doesn’t have one.
The external-check directive has a new preserve-env argument that will preserve environment variables.

Syslog load balancing

A new load balancing mode called log enables you to load balance syslog servers in a backend. Backends with this mode support syslog over UDP. Consult the balance directive documentation to see which load balancing algorithms are allowed.

QUIC

On each bind line, you can now set whether QUIC connections should share a listener socket or allocate their own. Set the quic-socket argument to either connection (dedicated socket per connection) or listener (shared listener socket). Previously, you could only configure this behavior at the global level with tune.quic.socket-owner. The default value is connection, which performs the best, but some systems don’t support it.
This version adds new performance-tuning directives that change the size of the receive and send buffers: tune.rcvbuf.backend, tune.rcvbuf.frontend, tune.sndbuf.backend, tune.sndbuf.frontend.
The global directive maxsslconn, which sets the maximum, per-process number of concurrent TLS connections, now applies to QUIC connections.
QUIC connections are now counted in maxconn at their allocation time. Previously, they had been counted only after handshake completion.

Connection handling

The backlog directive now limits the number of concurrent handshakes and connections waiting to be accepted.
The new timeout client-hs directive applies to handshakes on TCP, HTTP/1, and HTTP/2. It prevents handshakes from running too long. It defaults to the value of timeout client.

TLS

You can now specify sigalgs and client-sigalgs on server lines in a backend. The first argument, sigalgs, determines allowed signature algorithms for a TLS handshake. The second, client-sigalgs, determines allowed signature algorithms for client certificate authentication. You can also set these globally with the directives ssl-default-server-sigalgs and ssl-default-server-client-sigalgs.
The server directive has a new curves argument for setting elliptic curves to use when sending the ClientHello message to a backend server during a TLS handshake. You can also set this globally for all servers with the ssl-default-server-curves directive.

Linux capabilities

A new global directive named setcap allows you to list which Linux capabilities to preserve after startup. This is necessary when running the load balancer as a non-root user in order to use some features, such as binding to a privileged port for QUIC, for example.

Reverse HTTP

A new, experimental feature named Reverse HTTP facilitates a two-tier load balancer architecture, where the first HAProxy Enterprise tier initiates a connection to the second HAProxy Enterprise tier. The second tier load balancer then converts the connection to be bidirectional and allows client traffic to pass over that connection. For example, an edge load balancer can accept connections from data center load balancers and then, once established, begin sending client traffic over those connections to the data centers. This is in contrast to the traditional way of configuring backend servers, which is via a static list of server addresses. Instead, this allows a load balancer to self-register itself as an available server within the backend.

Load balancing

When you set the balance directive to the hash-based algorithm hash, you can then provide a fetch method to calculate the hash key with the utmost flexibility. This version introduces the ability to set hash-type to none, allowing you to disable the built-in hash function and manually hash the key using a converter.

Proxy Protocol

This version expands support for the Proxy Protocol’s Type-Length-Value (TLV) fields, which let you attach custom data to the Proxy Protocol header. In this version of HAProxy Enterprise, you can:
- set new TLVs by adding the set-proxy-v2-tlv-fmt argument on a server line.
- use the fc_pp_tlv fetch method to extract TLVs from the protocol header.
- use set-proxy-v2-tlv-fmt to forward TLVs you receive with fc_pp_tlv.

Caching

The cache can differentiate cached objects if they have the Vary HTTP header. However, until now, you could only vary on the Accept-Encoding and Referer headers. This version adds support for varying on the Origin header to support caching CORS responses correctly.

Lua

HAProxy Enterprise 2.9r1 adds these Lua features:
- A new function named core.get_var(var_name) returns proc scoped variables. The older txn:get_var(var_name) returns txn scoped variables.
- The httpclient class now supports a retries field, which sets the number of times to retry a failed request.
- The httpclient class now supports a timeout.connect field for setting the maximum time to wait for establishing a connection, in milliseconds. It defaults to 5 seconds.
- The core.register_action now supports http-after-res.
- Setting the new global directive tune.lua.log.loggers to on enables sending log messages from your Lua script to the loggers you’ve configured with log lines. Setting it to off disables the logging. The default value is on.
- Setting tune.lua.log.stderr to on will send log messages to stderr. When set to auto, logging to stderr occurs when tune.lua.log.loggers is set to off or when there’s no proxy-level logger assigned, such as for Lua tasks that run in the background. The default value is auto.

HTTP protocol

Two new directives, option accept-invalid-http-request and option accept-invalid-http-response, enable the load balancer to accept some invalid characters in URIs that it normally wouldn’t, such as the hash mark (#).

Converters

This version adds the following converters:

Name	Description
`ms_ltime`	Same as `ltime` but takes input in milliseconds.
`ms_utime`	Same as `utime` but takes input in milliseconds.
`us_ltime`	Same as `ltime` but takes input in microseconds.
`us_utime`	Same as `utime` but takes input in microseconds.

The bytes converter now accept variables for its offset and length arguments, which makes it easier to work with the TLV fields in the Proxy Protocol, since they have variable lengths.
The json_query converter now supports parsing JSON arrays.

Fetch methods

This version adds the following fetch methods:

Name	Description
`acl`	Evaluates up to 12 named ACLs separated by commas and returns a boolean result.
`pid`	Returns the process ID of the current process, which is usually the worker process.
`act_conn`	Returns the total number of active, concurrent connections on the process.
`bytes_in`	Returns the number of bytes uploaded from the client to the load balancer.
`bytes_out`	Returns the number of bytes transmitted from the load balancer to the client.
`accept_date`	Returns the exact date when the connection was received by the load balancer.
`request_date`	Returns the value for the exact date when the load balancer received the first byte of the HTTP request.
`fc.timer.handshake`	Returns the time spent accepting the TCP connection and executing handshakes and is equivalent to `%Th`.
`fc.timer.total`	Returns the total session duration time and is equivalent to `%Tt`.
`req.timer.idle`	Returns the idle time before the request and is equivalent to `%Ti`.
`req.timer.hdr`	Returns the time spent waiting to get the client’s request and is equivalent to `%TR`.
`req.timer.tq`	Returns the sum of `%Th`, `%Ti` and `%TR` and is equivalent to `%Tq`.
`req.timer.queue`	Returns the time spent queued and is equivalent to `%Tw`.
`bc.timer.connect`	Returns the time spent establishing a connection to the backend server and is equivalent to `%Tc`.
`res.timer.data`	Returns the time spent transferring the response payload to the client and is equivalent to `%Td`.
`res.timer.hdr`	Returns the time spent waiting for the server to send a full response and is equivalent to `%Tr`.
`txn.timer.user`	Returns the estimated time as seen by the client and is equivalent to `%Tu`.
`txn.timer.total`	Returns the active time for the HTTP request and is equivalent to `%Ta`.
`req.cook_names`	Returns the names of all cookies in requests.
`res.cook_names`	Returns the names of all cookies in responses.
`ssl_bc_curve`	Retrieves the name of the curve used in the key agreement when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_fc_curve`	Retrieves the name of the curve used in the key agreement when the incoming connection was made over an SSL/TLS transport layer.
`cur_client_timeout`	Retrieves the value in milliseconds for the currently configured client timeout.
`fc_pp_tlv`	Returns the TLV value for a given TLV ID. This fetch may be useful in detecting errors related to duplicate TLV IDs.

Version 2.8r1 Jump to heading

Key changes in the HAProxy Enterprise 2.8r1 release include:

Email alerts as a Lua module

The mailers feature, which lets you send email alerts about the load balancer, has been ported to be a Lua module. The benefit to you is the ability to edit the Lua code to suit your needs, such as to customize the alert messages that the load balancer produces. To enable it, load the mailers Lua module using the lua-load directive in the global section.

Lua updates

A new event framework allows Lua modules to subscribe to some load balancer events and execute event handler functions. Events you can subscribe to include when servers are added, deleted, or go up or down. Use the function Server.event_sub to subscribe to events.
A new core.queue function lets you create a first-in first-out data structure in Lua.
The Server class gained new functions for accessing information about backend servers, including getting the server’s name (get_name), the number of active sessions (get_cur_sess), the number of pending connections (get_pend_conn), and other functions.
A new global directive named tune.lua.burst-timeout terminates the Lua process if it exceeds the timeout value without ever yielding execution back to the load balancer. A well-behaved Lua module will yield execution on the current thread back to the load balancer during long-running tasks, to be continued later.
The Lua’s HTTP client that ships with the load balancer is able to resolve DNS hostnames thanks to a default resolvers section invisibly added to the configuration. If you do not use the HTTP client in Lua, then you can disable this by setting the new global directive httpclient.resolvers.disabled to on.

HTTP compression

The HTTP compression directives have an updated syntax that allows you to compress both responses and requests. For example, you can now compress requests that contain JSON messages before relaying them to backend servers. See the compression direction, compression algo-req, compression algo-res, compression type-req, and compression type-res directives.

TLS signing algorithms and curves

The new global directives ssl-default-bind-client-sigalgs and ssl-default-bind-sigalgs, as well as the new bind arguments client-sigalgs and sigalgs let you list the TLS signing algorithms the load balancer will use or accept during a TLS handshake and when validating a client certificate.
The curves argument, which sets a list of elliptic curve algorithms negotiated during the TLS handshake, is now available on the server and default-server lines.

Default ALPN values

You no longer need to specify alpn h2 on a bind line to enable HTTP/2. This is now the default value and will fall back to HTTP/1.1 if the client does not support HTTP/2. Also, bind lines that use the QUIC protocol will default to having an alpn argument set to h3 for HTTP/3.

OCSP stapling

OCSP stapling, which improves network latency by sending TLS certificate revocation information to the client preemptively, is now a built-in feature. Set the new ocsp-update argument to on in a CRT file to enable automatic updating of a certificate’s OCSP file. Use the global directives tune.ssl.ocsp-update.mindelay and tune.ssl.ocsp-update.maxdelay to adjust the interval between updates. You can force an update with the Runtime API command update ssl ocsp-response, view the status with show ssl ocsp-updates, and view OCSP responses with show ssl ocsp-response.

HTTP Forwarded header

The new directive option forwarded will add a Forwarded header to HTTP requests, passing the client’s IP address to backend servers. This is the successor to the non-standard X-Forwarded-For header. New converters let you validate and read the header: rfc7239_is_valid, rfc7239_field, rfc7239_n2nn, and rfc7239_n2np.

Configurable sticky counters

The new global directive tune.stick-counters lets you increase the number of stick table sticky counters, which had previously been a built-in constant set to 3.

HTTP actions

The http-after-response directive gained an expanded set of actions, including set-map, sc-add-gpc, set-log-level, and others.
All action directives now support the sc-add-gpc action.

HTTP/2 tuning options

New global directives enable tuning the HTTP/2 protocol. They include tune.h2.be.initialize-window-size, tune.h2.be.max-concurrent-streams, tune.h2.fe.initial-window-size, tune.h2.fe.max-concurrent-streams, tune.h2.initial-window-size and tune.h2.max-concurrent-streams.

Listener shards

The new global directive tune.listener.default-shards simplifies setting shards. Sharding is a feature for servers with a high number of threads that makes it easier to replicate a bind line to create multiple listeners and assign worker threads to the listeners. The goal is to reduce thread contention. You can set the directive to a number of shards, to the value by-thread, or to the value by-group. Setting by-group is especially convenient since you can then group threads and assign them together to a listener, and so this is the default setting.

Other performance tuning options

The bind line’s cpu-map argument, which associates threads with CPUs, now accepts a list, delimited by commas.
The bind line’s thread argument can now be set to a comma-delimited list of thread ranges instead of just a single range.
The new global directive tune.listener.multi-queue controls how work is distributed to threads assigned to a listener. A value of on, which is the default, passes work to the least busy thread. A value of fair passes work in a round-robin rotation.
The new global directive tune.memory.hot-size allows you to set the amount of memory kept hot in the local cache, but the default value of 512 KB should be sufficient for most.

HTTP request body and URL parameters

The req.body_param, url_param, and urlp_val fetch methods now accept an additional argument that enables case-insensitive matching of parameters, which makes it easier to configure the load balancer’s behavior to agree with the web server’s behavior. Also, a new converter named param extracts name-value parameters from a string while letting you set the delimiter used between parameters.

Round-trip time

The new fetch methods bc_rtt and bc_trrvar return the round-trip time between the load balancer and a backend server. You can configure them to return values in milliseconds or microseconds.

Runtime API

The prompt command now accepts an optional argument, timed, that will cause the interactive prompt to display the process’s uptime.
The show quic command, which displays information about active QUIC connections on the frontend, can now display output in one of two formats: oneline or full.

Preprocessor directives

This version introduces two new preprocessor functions. The strstr function returns true if the inspected string contains a given string. The enabled function returns true if the given option is enabled at runtime.

Let’s Encrypt acme.sh client

With special support for the acme.sh Let’s Encrypt client, you can integrate HAProxy Enterprise with your automated TLS certificate infrastructure.

Proxy Protocol

Proxy Protocol v2 has been updated in this version to allow the load balancer to extract extra information from the Proxy Protocol header when a TCP connection is established on the frontend. The new fetch method fc_pp_tlv supports extracting a TLV (Type-Length-Value) from the protocol header.

SSO module

The SSO module, which enables a single sign-on integration with Windows Active Directory, now supports validating Kerberos tickets.

Version 2.7r1 Jump to heading

Key changes in the HAProxy Enterprise 2.7r1 release include:

HTTP/3 over QUIC

This version adds support for the HTTP/3 protocol over QUIC. Because it requires a specialized version of the OpenSSL library, a separate install package is provided.

Debugging

A new global directive named trace lets you configure the HAProxy Enterprise events tracing subsystem from the configuration file, whereas previously it was accessible only through the Runtime API’s trace command. This directive is experimental and requires you to set the global option expose-experimental-directives.
You can anonymize the output of some Runtime API commands to mask IP addresses, names of sections, and hostnames. Add set anon on, followed by a semicolon, before a command such as show sess.
```
nix
echo "set anon on; show sess" |\
  sudo socat stdio /var/run/hapee-2.7/hapee-lb.sock
```
```
nix
echo "set anon on; show sess" |\
  sudo socat stdio /var/run/hapee-2.7/hapee-lb.sock
```
You can also display a version of your HAProxy Enterprise configuration with IP addresses, section names, and hostnames masked by using the hapee-lb command’s -dC<key> flag, where key is an arbitrary number used when creating the masked values. For example, to display an anonymized version of the HAProxy Enterprise configuration:
```
nix
/opt/hapee-2.7/sbin/hapee-lb -dC123456 -f etc/hapee-2.7/hapee-lb.cfg
```
```
nix
/opt/hapee-2.7/sbin/hapee-lb -dC123456 -f etc/hapee-2.7/hapee-lb.cfg
```

TLS

The bind directive’s ca-ignore-err and crt-ignore-err arguments, which let you list TLS errors to ignore, now accept human-readable names in addition to numeric IDs. Refer to the OpenSSL list of error codes to see which values are available.
The new Runtime API command, add ssl ca-file, adds an SSL/TLS certificate to the in-memory CA file list.

Lua

You can now pass optional arguments to your custom Lua scripts from within your HAProxy Enterprise configuration file. The lua-load and lua-load-per-thread directives accept one or more arguments after the first argument, which is always the script’s filepath. Then, in your script use the built-in table.pack(...) function with an ellipsis as its argument to collect arguments into a variable of type table.

Multithreading

A new global directive named thread-group lets you place a range of threads into a group. Once in a group, you can assign those threads to a listening address by adding the thread argument to a bind line in a frontend or listen section. On servers with many CPU cores and thus many threads, allocating a subset of threads to handle connections like this can improve performance by reducing the number of threads competing for work. You can define up to 64 thread groups, each including up to 64 threads.
A new global directive named thread-groups lets you set the number of thread groups to create and HAProxy Enterprise will divide the threads available on your server into that number of groups. The maximum number is 64.

Converters

This version adds the following converters:

Name	Description
`table_expire(<table>[,<default_value>])`	Returns the remaining time before a given key will expire in the stick table, as well as how long ago a given key was last seen.
`table_idle(<table>)`	Returns the time the given key has remained idle since the last time it was updated.
`host_only`	Converts a string that contains a Host header value and removes its port.
`port_only`	Converts a string that contains a Host header value and returns only its integer port.
`x509_v_err_str`	Converts a numerical value to its corresponding X509_V_ERR constant name, which is useful for setting ACL expressions based on different client certificate errors (expired certificate, revoked certificate, etc.) when working with multiple versions of OpenSSL.

Performance upgrades

Server health checks, which run at an interval, now fire on the same thread exclusively rather than allowing any available thread to perform the task. This has reduced latency by decreasing competition among threads.
Stick tables became more efficient by changing the type of lock used to restrict multiple threads from accessing a table simultaneously. By revising internal code to use a read-write lock, which allows multiple threads to read from the table simultaneously, but allows only one thread to write, performance improved.
HAProxy Enterprise 2.7 reduces latency by more aggressively using idle connections when sending HTTP requests to backend servers. If you leave the defaults, or if you manually set the http-reuse directive in a backend to safe, reusing idle connections is enabled and you are choosing the safest mode. That is, a client’s first HTTP request will be dispatched to a backend server on a new thread guaranteed to not be closed, and only subsequent requests will use idle connections, which have the risk of closing unexpectedly. In this release, as long as you have also set the retry-on directive in that backend to retry the connection in case of failure, HAProxy Enterprise will use an idle connection for a client’s first request too. Set retry-on to include conn-failure, empty-response, and response-timeout.

Deprecated keywords

The bind-process directive has been removed.
The process argument on a bind line has been removed.

Version 2.6r1 Jump to heading

Key changes in the HAProxy Enterprise 2.6r1 release include:

Advanced WAF

New match zones, $COOKIES_VAR and $COOKIES_VAR_X allow rules to match violations found in specific HTTP cookies, complementing the less specific $HEADERS_VAR and $HEADERS_VAR_X match zones.
A new variable, txn.<filter>.wlcnt returns the number of disabled rules that would have matched the current request.
The filter waf line now supports a parameter named log-wl that includes disabled WAF rule violations in extended logs.
The filter waf line now supports a parameter named log-ext-nonzero that enables extended logs only when the violated rule had a non-zero score or triggered an immediate action, cutting down on noise in the logs.
The filter waf line now supports a parameter named table-categorized that prefixes entries in the violated rules stick table with the category of the violated rule.
The filter waf line’s body-limit parameter defaults to the global option waf-body-limit, which now defaults to tune.bufsize rather than the compile-time value of BUFSIZE.

ModSecurity WAF

To associate WAF logs with load balancer logs, you could already use the unique identifier that ModSecurity creates by referencing the txn.<filter>.unique_id variable, appending it to your load balancer logs. Now, you can define a different unique ID format by setting the use-unique-id-format parameter on the filter modsecurity line and then defining a new format with the unique-id-format directive.
A new parameter use-vars on the filter modsecurity line disables ModSecurity from denying suspicious requests and delegates that to the load balancer. A variable named txn.<filter>.block returns true when the WAF would have denied the request. You can read this variable and then decide on a response policy to enforce. Other variables that support this include txn.<filter>.error, txn.<filter>.status, and txn.<filter>.url.

Traffic shaping

New directives, filter bwlim-in and filter bwlim-out, support limiting upload and download speeds for clients. Set bandwidth limits that apply per HTTP stream or to all streams associated with a stick table entry, such as to set a limit per client IP address or per backend application.

Load balancing

A generic load balancing algorithm named hash was introduced and serves as a replacement for the more specific source, uri, url_param, and rdp-cookie algorithms. It accepts a fetch method as a parameter, which indicates the data used to calculate the hash.

TLS and mTLS

This version of HAProxy Enterprise supports OpenSSL 3.0.
When you enable client certificate authentication with the verify required parameter on a bind line, you must also specify the ca-file parameter, which indicates the CA certificate used to verify the client’s certificate. Now, ca-file accepts a path to a directory of CA certificates.
Similarly, in a backend, you can set the ca-file parameter on a server line to verify the backend server’s TLS certificate against a known CA. This ca-file parameter now accepts a path to a directory of certificates or you can set it to @system-ca to load your system’s list of trusted CAs.

Runtime API and Master CLI

The Runtime API commands add server and del server, which let you add and remove servers dynamically, are no longer experimental.
The new Runtime API command show ssl providers lists providers loaded into OpenSSL.
The Master CLI added new commands: prompt, expert-mode [on|off], experimental-mode [on|off], mcli-debug-mode [on|off].

Lua

The Lua programming language integration gained the CertCache class, which lets you update an SSL certificate in the load balancer’s runtime memory.
The Lua httpclient class, which lets you make non-blocking HTTP calls, now supports a dst parameter and a timeout parameter. The former sets the destination address and the latter sets a timeout server value. New global directives support this: httpclient.ssl.ca-file, httpclient.ssl.verify, httpclient.ssl.resolvers.id, and httpclient.resolvers.prefer <ipv4|ipv6>.

Fetches and converts

New fetches have been added: last_rule_file, which returns the name of the configuration file (e.g. hapee-lb.cfg) that contains the last line processed during stream analysis, and last_rule_line, which returns the line number. Use this to locate the http-request deny line that stopped a request, for example.
A new converter has been added: add_item, which concatenates strings with a delimiter between them, such as a semicolon.

Other keyword changes

A new global directive named h1-accept-payload-with-any-method lets HTTP/1.0 clients send a request body with GET, HEAD, and DELETE requests. Use with caution because it can make your application more susceptible to request smuggling attacks.
Adding the parameter expose-fd listeners on a stats socket line is no longer necessary to achieve hitless reloads.
The set-var directives, such as http-request set-var, now accept a second parameter to only set the variable if a condition is true. Conditions include: ifexists, ifnotexists, ifempty, ifnotempty, ifset, ifnotset, ifgt, iflt. For example, to set the variable only if it has not already been set: http-request set-var(txn.myvariable,ifnotset) req.hdr(X-MyValue).

Performance upgrades

The new global directive fd-hard-limit enforces a cap on the number of file descriptors HAProxy Enterprise will use, protecting you from consuming too much memory.
The new global directive close-spread-time closes idle client connections gradually. For best results, set this lower than hard-stop-after.
If the load balancer server is a multi-socket machine, HAProxy Enterprise sets an affinity to run on the CPUs of a single node in order to avoid performance penalties caused by the inter-socket bus latency. You can disable this with the no numa-cpu-mapping directive.
Performance optimizations were made to the task scheduler, connection dequeueing, and connection stream code.

Version 2.5r1 Jump to heading

Key changes in the HAProxy Enterprise 2.5r1 release include:

Improved Performance

Memory optimizations on x86 platforms
Lockless memory pool implementation for non-x86 architectures
Faster connection dequeueing due to locking reduction
Revamped and more efficient Domain Name Service (DNS) response processing
Faster HTTP/1 small chunk parsing

Better thread allocation and control

Split the total number of configured threads (nbthread) into ranges and assign them to handle incoming connections for unique IP addresses by adding the thread argument to a bind.
Reduce scheduler contention with the binds and shards options on systems that support multiple listeners per socket.

Observability and access to more information

More robust reporting on startup
Statistics for stopped proxies are viewable in the Stats Pane
The HAProxy Enterprise version and executable path are displayed before the first warning or error to aid with troubleshooting
Retrieve the original source address of an incoming proxy that connects using the PROXY protocol with fetch fc_src and tcp-request content ruleset: set-src, set-src-port
Log connection information for handshake errors, using: ssl_bc_err, ssl_fc_err, ssl_bc_err_str, ssl_fc_err_str, fc_err, bc_err
Retrieve backend connection errors with bc_err, bc_err_str

Configuration

Store up to 100 indexes in a single Stick Table variable, using General Purpose Counter (GPC) and General Purpose Tag (GPT) arrays.
Disable bootstrapping WebSockets with HTTP/2 (RFC8441) for newer browsers that don’t support it by using h2-workaround-bogus-websocket-clients.
Ignore connections without data transfer in the logs with the http-ignore-probes option.
Gracefully stop proxies, allowing enough time for layer 4 devices to detect changes with the global grace option.
Add tcp-request and http-request directives to named defaults sections.
Add more logic to configuration file .if, .elif, and endif conditions with the addition of AND, OR, NOT, and parentheses expressions.
improved reliability of host header field dependent ACLs by automatically stripping the port (80 or 443) from the URI and the host field.
Peers ignore updates on the Stick Table conn_cur counter from other peers, ensuring local data is accurate and reflects actual traffic.

Runtime API commands

Display free memory in thread-local caches with show pools
Display file, line numbers, rules, and filters processed per session with show sess all
Full support for creating, managing and removing servers on the fly. The add server command now supports all keywords including: check, track, slowstart, error-limit, ssl and observe.
View the number of entries in summary output of the show map and show acl commands with the included entry_cnt variable.

Lua scripting

Initiate HTTP requests using the httpclient class.
Inspect or modify TCP and HTTP content using an experimental feature set that should be used only in test environments.

Traffic routing

Redirect rules for empty target URLs are skipped by adding ignore-empty to the http-request redirect directive.

Security

Check the integrity of claims contained within JSON Web Tokens (JWT) and extract data from JWTs to use in rules.
Add, update, and delete Certificate Authorities (CA) and Certificate Revocation Lists (CRL) using Runtime API commands.
Display Online Certificate Status Protocol (OCSP) to validate X.509 digital certificates from the command line (CLI) with show ssl cert and show ssl ocsp-response commands.
OpenSSL 3.0 is fully supported
Automatic sanitizing of Transfer Encoding (TE) headers to conform to HTTP/1 specs by making a request or response TE header or a request with Content-Length and TE header the last on a connection.

Removed directives

grace (per proxy directive replaced by global grace directive)
http-tunnel
nbproc
no option http-use-htx
option forceclose
option http_proxy
set-cookie()
tune.chksize

Name	Description
ssl_sess	Total number of SSL sessions established.
ssl_reused_sess	Total number of SSL sessions reused.
ssl_failed_handshake	Total number of failed handshake.

Function	Description
defined()	Returns true if an environment variable exists, regardless of its contents.
feature()	Returns true if feature is listed as present in the features list reported by `haproxy -vv`.
streq(,)	Returns true if the two strings are equal.
strneq(,)	Returns true if the two strings differ.
version_atleast()	Returns true if the current HAProxy version is at least as recent as otherwise false.
version_before()	Returns true if the current HAProxy version is strictly older than otherwise false.

Directive	Description
.diag “message”	Emit this message only when in diagnostic mode (`-dD`).
.notice “message”	Emit this message at level NOTICE.
.warning “message”	Emit this message at level WARNING.
.alert “message”	Emit this message at level ALERT.

Option	Description
`current`	Relative file paths are loaded from the directory the process was started in. This is the default.
`config`	Relative file paths are loaded from the directory containing the configuration file.
`parent`	Relative file paths should be loaded from the parent directory above the configuration file directory.
`origin <path>`	Relative file paths should be loaded from the designated path.

Variable	Description
`.FILE`	The name of the configuration file currently being parsed.
`.LINE`	The line number of the configuration file currently being parsed, starting at one.
`.SECTION`	The name of the section currently being parsed, or its type if the section doesn’t have a name (e.g. “global”).

Fetch	Description
`be_server_timeout`	Returns the configuration value in millisecond for the server timeout of the current backend.
`be_tunnel_timeout`	Returns the configuration value in millisecond for the tunnel timeout of the current backend.
`cur_server_timeout`	Returns the currently applied server timeout in millisecond for the stream.
`cur_tunnel_timeout`	Returns the currently applied tunnel timeout in millisecond for the stream.

The new tcp-request content switch-mode directive upgrades a mode tcp connection to mode http dynamically. Combined with the built-in HTTP ACL, you can switch to mode http if the traffic is HTTP.

haproxy
frontend fe_main
  mode tcp
  tcp-request inspect-delay 5s
  tcp-request content switch-mode http if HTTP

haproxy
frontend fe_main
  mode tcp
  tcp-request inspect-delay 5s
  tcp-request content switch-mode http if HTTP

Header deletion with pattern matching

The http-request del-header, http-response del-header, and http-after-response del-header directives now support the argument -m <method> to delete a header based on a matched pattern.

HTTP request conditional body wait time

The new http-request wait-for-body and http-response wait-for-body directives allow you to conditionally wait for and buffer a request or response body. These actions may be used as a replacement for option http-buffer-request. In the snippet below, HAProxy Enterprise waits one second at most to receive the POST body of the request, or until it gets at least 1,000 bytes:
```
haproxy
http-request wait-for-body time 1s at-least 1k if METH_POST
```
```
haproxy
http-request wait-for-body time 1s at-least 1k if METH_POST
```

Fetches

This table lists new fetches:

Name	Description
`baseq`	Returns the concatenation of the first Host header and the path part of the request with the query-string, which starts at the first slash.
`bc_dst`	This is the destination ip address of the connection on the server side, which is the server address HAProxy Enterprise connected to.
`bc_dst_port`	Returns an integer value corresponding to the destination TCP port of the connection on the server side, which is the port HAProxy Enterprise connected to.
`bc_src`	This is the source ip address of the connection on the server side, which is the server address HAProxy Enterprise connected from.
`bc_src_port`	Returns an integer value corresponding to the TCP source port of the connection on the server side, which is the port HAProxy Enterprise connected from.
`be_server_timeout`	Returns the configuration value in millisecond for the server timeout of the current backend.
`be_tunnel_timeout`	Returns the configuration value in millisecond for the tunnel timeout of the current backend.
`cur_server_timeout`	Returns the currently applied server timeout in millisecond for the stream.
`cur_tunnel_timeout`	Returns the currently applied tunnel timeout in millisecond for the stream.
`fe_client_timeout`	Returns the configuration value in millisecond for the client timeout of the current frontend.
`sc_http_fail_cnt(<ctr>[,<table>]`	Returns the cumulative number of HTTP response failures from the currently tracked counters. This includes the both response errors and 5xx status codes other than 501 and 505.
`sc_http_fail_rate(<ctr>[,<table>])`	Returns the average rate of HTTP response failures from the currently tracked counters, measured in amount of failures over the period configured in the table. This includes the both response errors and 5xx status codes other than 501 and 505. See also `src_http_fail_rate`.
`ssl_c_der`	Returns the DER formatted certificate presented by the client when the incoming connection was made over an SSL/TLS transport layer.

Converters

The following converters have been added:

Name	Description
`json_query(<json_path>,[<output_type>])`	The json_query converter supports the JSON types string, boolean and number.
`xxh3`	Hashes a binary input sample into a signed 64-bit quantity using the XXH3 64-bit variant of the XXhash hash function.
`ub64dec`	This converter is the base64url variant of b64dec converter. base64url encoding is the “URL and Filename Safe Alphabet” variant of base64 encoding. It is also the encoding used in JWT (JSON Web Token) standard.
`ub64enc`	This converter is the base64url variant of base64 converter.
`url_enc`	Takes a string provided as input and returns the encoded version as output.

Runtime API

This version includes the following changes to the Runtime API:

Help filtering- The help command will attempt to display information for partially spelled commands. Also if you enter an incorrect command, the API will suggest alternatives.

This release adds new Runtime API commands:

Name	Description
`set server <backend>/<server> agent-port <port>`	Change the port used for agent checks.
`set server <backend>/<server> check-addr <ip4 \| ip6> [port <port>]`	Change the IP address used for server health checks. Optionally, change the port used for server health checks.
`set server <backend/server> ssl on`	Activates SSL on outgoing connections to a server at runtime.
`prepare acl`	Begin a transaction for adding ACLs. This returns a number, which you would then reference in the `add acl` command.
`commit acl`	Commit the transaction to add new ACLs.

Lua

Lua has been extended to support multithreading. A new global directive, lua-load-per-thread, has been added to aid with this. For Lua scripts that require threading, this should be used as it will launch the Lua code as an independent state in each thread. The existing lua-load keyword still exists and should be used for Lua modules meant to cover the entire process, such as jobs registered with core.register_task.

Miscellaneous

There are the following miscellaneous changes:

The tune.chksize directive has been deprecated.
During a reload, processes are closed sooner, rather than waiting for idle frontend connections to timeout.
During a reload, idle backend connections are actively killed.
TCP log outputs now automatically create a ring buffer.
Layer 7 retries now support 401 and 403 HTTP status codes.
The http-check send directive now supports adding a Connection header.
The global section now supports setting process level variables.
A new macro HTTP_2.0 has been added, which will return true for HTTP/2 requests.
The server-state-file-name directive has a new argument, use-backend-name, that allows you to load the state file using the name of the backend. This only applies when the directive load-server-state-from-file is set to local.
A new srvkey parameter has been added to the stick-table directive. This allows you to specify how a server is identified. It accepts either name, which will use the current defined name of the server or addr, which will use the current network address including the port. addr is useful when you are using service discovery to generate the addresses.
A new log-format parameter, %HPO, was added, which allows logging of the request path without the query string.

Version 2.3r1 Jump to heading

Key changes in the HAProxy Enterprise 2.3r1 release include:

Connection improvements for upcoming QUIC and HTTP/3 support

The connection layer was optimized to reduce the number of syscalls.
Several debugging entries were added to help better spot anomalies.
Listeners have been reworked and related structures have been reorganized to better suit the new design. File descriptors are no longer manipulated by the listener layer.

TCP keepalive lets the Linux kernel know when a peer on the other end of a connection has stopped responding and that it’s safe to close the idle connection. It discovers this by sending probes. If the peer doesn’t reply, the socket is closed automatically. You can change the number of probes to send, the interval at which to send them, or how long to wait before starting to send probes on both the client and server side through the following directives:

Directive	Description
clitcpka-cnt	Sets the maximum number of keepalive probes TCP should send before dropping the connection on the client side.
clitcpka-idle	Sets the time the connection needs to remain idle before TCP starts sending keepalive probes on the client side, if enabled.
clitcpka-intvl	Sets the time between individual keepalive probes on the client side.
srvtcpka-cnt	Sets the maximum number of keepalive probes TCP should send before dropping the connection on the server side.
srvtcpka-idle	Sets the time the connection needs to remain idle before TCP starts sending keepalive probes on the server side, if enabled.
srvtcpka-intvl	Sets the time between individual keepalive probes on the server side.

Syslog protocol

A new section called log-forward can bind on TCP using the bind keyword and on UDP using dgram-bind for both IPv4 and IPv6. You can thus create a syslog listener over UDP or TCP that can forward, prioritize, and translate syslog messages to a pool of UDP or TCP syslog servers. When combined with the log sampling feature added in HAProxy Enterprise 2.0, you get granular control over how your syslog messages are forwarded. You can also translate syslog messages from one format to another.
The Runtime API show info command also exposes a new counter called CumRecvLogs, which provides a global count of received syslog messages.

Load Balancing

The new balance uri directive’s path-only option indicates that the hash should be calculated using only the path normalizing HTTP/1 and HTTP/2 messages. This avoids inconsistencies between requests received over HTTP/1 and the same ones received over HTTP/2.
If the balance random algorithm returns a server whose maxconn value has been reached, meaning that connections are now queuing up for that server, it will add the request to the backend’s queue and not the server’s queue. The request can be redispatched to another available server, and typically the fastest.
For some load-balancing algorithms (roundrobin, static-rr, leastconn, first), requests were queued in the backend due to a previous attempt at finding a suitable server after trying all of them. Now, the next request skips the part where each server is tried and goes directly to the backend’s queue.
The leastconn algorithm has been improved to take the queue length into account when dispatching requests. This means that if a server has a lot of queued requests we won’t hammer it with extra connections.

Cache

The Expires HTTP header instructs HAProxy Enterprise how long it should cache the response.
The cache now supports the ETag, If-None-Match, and If-Modified-Since.
The cache can return an HTTP status code 304 instead of the full object.
HAProxy Enterprise will now also reject any configuration that has a duplicate cache section name.
The new fetch methods added-res.cache_hit and res.cache_name tell you whether a response came from the cache and, if so, the name of the cache used.

SSL/TLS Enhancements

If you load SSL/TLS certificates separately from the certificate key through the ssl-load-extra-files global directive, the key no longer needs to be named the exact same as the certificate with .key appended to it.
The new global directive ssl-load-extra-del-ext instructs HAProxy Enterprise to remove the certificate file’s extension before adding a new one. For example, the key can be named mycert.key instead of mycert.crt.key.
The generate-certificates directive adds a Subject Alternative Name (SAN) to all generated certificates, which is a requirement in modern browsers. It now also supports chaining CAs and attaching a trust chain in addition to the generated certificate. The chain is loaded from the one provided in the ca-sign-file PEM file.
If the SNI is hardcoded on the server line using, as an example, sni str(example.local) there’s no risk in reusing the connection. This release allows reusing connections that hardcode the SNI to the backend server. It will mark connections as private only if you’ve configured a variable expression for the SNI.
If a crt-list does not end with a new line, a warning indicates that the file might have been truncated.

Observability

The new show stat Runtime API command’s option domain, allows you to change the context of the statistics. The proxy default value displays the core proxy statistics that were available before. The dns value displays statistics related to DNS resolution that HAProxy Enterprise performs.
The stats show-modules directive enables extra statistics related to HTTP/2 on the HAProxy Enterprise Stats.
The Stats page displays a new field under the Wght column, which previously only showed the live or effective weight. The Wght column now contains the effective weight separated with a “/” followed by the configured weight.
The show stat Runtime API command now allows you to use show stat up to filter on servers that are up or show stat no-maint to show those that are not in maintenance mode.

The Prometheus exporter received some new process and per-server metrics, as outlined here:

Metric	Description
haproxy_process_failed_resolutions	Total number of failed DNS resolutions.
haproxy_process_bytes_out_total	Total number of bytes emitted.
haproxy_process_spliced_bytes_out_total	Total number of bytes emitted through a kernel pipe.
haproxy_process_bytes_out_rate	Number of bytes emitted over the last elapsed second.
haproxy_server_unsafe_idle_connections_current	Current number of unsafe idle connections.
haproxy_server_safe_idle_connections_current	Current number of safe idle connections.
haproxy_server_used_connections_current	Current number of connections in use.
haproxy_server_need_connections_current	Estimated needed number of connections.

The Runtime API’s show stat output has been extended and adds a new delimiter, a dash (“-”), after which additional dynamic fields can be added. Those fields won’t be shown unless the relevant component is in use.

OpenTracing (SPOE)

An OpenTracing SPOA allows HAProxy Enterprise to send data directly to distributed tracing systems via the OpenTracing API.

HTTP Request Actions

The new HTTP request action http-request replace-pathq, does the same as http-request replace-path, except that the replacement value may contain a modified query string.
The new HTTP request action http-request set-pathq works similarly to http-request set-path, except that the query string is also rewritten. Unlike http-request replace-pathq, it does not take a regular expression and replacement value, but a formatted string to use as the new path. It can also be used to remove the query string, including the question mark.

New sample fetches

Name	Description
`pathq`	This extracts the request’s URL path with the query-string, which starts at the first slash.
`res.cache_hit`	Returns the boolean “true” value if the response has been built out of an HTTP cache entry, otherwise returns boolean “false”.
`res.cache_name`	Returns a string containing the name of the HTTP cache that was used to build the HTTP response if res.cache_hit is true, otherwise returns an empty string.
`srv_iweight([<backend>/]<server>)`	Returns an integer corresponding to the server’s initial weight. If is omitted, then the server is looked up in the current backend.
`srv_uweight([<backend>/]<server>)`	Returns an integer corresponding to the current (or effective) server’s weight. If is omitted, then the server is looked up in the current backend.
`srv_weight([<backend>/]<server>)`	Returns an integer corresponding to the current (or effective) server’s weight. If is omitted, then the server is looked up in the current backend.
`ssl_c_der_chain`	Returns the DER formatted chain certificate presented by the client when the incoming connection was made over an SSL/TLS transport layer. When used for an ACL, the value(s) to match against can be passed in hexadecimal form.
`ssl_s_chain_der`	Returns the DER formatted chain certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer. When used for an ACL, the value(s) to match against can be passed in hexadecimal form.

New converters

iif returns the <true> string if the input value is true. Returns the <false> string otherwise. |

Lua

Support for Lua 5.4, which was initially released in June 2020.
This release exports the sample fetches http_auth() and http_auth_group().
You can now use regular expressions in fetches and converter arguments.
Sample fetches and converters that require arguments are now supported as well.

Deprecated and Removed Directives

The obsolete keyword monitor-net was removed. It supported only a single IPv4 network, was incompatible with SSL, and required HTTP/1.x. It is now recommended to use http-request return status 200 if { src 10.1.1.3 } instead.
The obsolete keyword mode health was removed. It was incompatible with SSL and worked with only HTTP/1. It is now recommended to use http-request return status 200 instead.
The global keyword debug has been removed. It had, on occasion, trapped users by disrupting their system’s ability to boot. You can continue to use -d on the command line.
The nbproc directive is now deprecated and is set for removal in 2.5. It used too much memory, led to high network overhead (poor reuse, multiple health checks), lacked peers syncing and stats, caused problems with seamless reloads, and would not support QUIC at all. If nbproc is found with more than one process while nbthread is not set, a warning will be emitted encouraging you to remove it or migrate to nbthread.
The grace directive has been marked as deprecated and is scheduled tentatively for removal in 2.4 with a hard deadline of 2.5. It was meant to postpone stopping of a process during a soft-stop, but is incompatible with soft reloading.

Miscellaneous

The strict-limits directive defaults to on. You’ll now get a startup error if you configure too large a maxconn for your system’s limits.
The process no longer reports proxy has started.
An optimization for PCRE2 was made, which uses the JIT match when a JIT optimization has occurred. This should shorten the code path to call the match function.
Several deinit() fixes were made to improve the results from Valgrind.
Support for upgradable locks was added. These cut the scheduler overhead in half and reduce the locking time during map and ACL updates.

Version 2.2r1 Jump to heading

Key changes in the HAProxy Enterprise 2.2r1 release include:

Enterprise addons & native modules

A new native module for performing TLS/SSL fingerprinting.
Support for Single Sign On SAML.
New validate-crawler daemon to verify the legitimacy of clients claiming to be Googlebot & Bingbot.

Dynamic SSL certificate storage

SSL certificates can now be added/removed/updated through the Runtime API using the set ssl cert and commit ssl cert commands.

SSL/TLS enhancements

TLSv1.2 the new default minimum version TLS version
tune.ssl.default-dh-param now defaults to 2048 bits
New global directive: ssl-default-bind-curves to specify the list of elliptic curve algorithms that are negotiated during the SSL/TLS handshake when using Elliptic-curve Diffie-Hellman Ephemeral (ECDHE).
New global directives: ssl-load-extra-files and issuers-chain-path to load the TLS/SSL private key and intermediate certificates separately.
New global directive: ssl-skip-self-issued-ca to avoid sending global CA certificates to clients when using OCSP stapling.
New bind parameter ca-verify-file to store the root CA in a separate file to use for validating an intermediate CA.
Support for fetching and logging the secrets necessary for decrypting TLS 1.3

Native response generator

New actions http-request return and http-response return make it possible to return a custom response from HAProxy Enterprise, with any status code, based on an error file, a file or a string.
In addition, using a log-format string or a log-format file, it is possible to have responses with a dynamic content.
Definition of extra headers by passing the hdr argument.

Dynamic error handling

A new section http-errors to create global groups of custom HTTP error files.
Expanded directives error-files, http-request deny, and http-response deny to support loading http-errors groups.
Dynamic error processing to define error file templates with log-format parameters.

Health check overhaul

New directives {http|tcp}-check connect to tune health check connection parameters by enabling SNI, connecting over SSL/TLS, performing health checks over SOCKS4, and choosing the protocol, such as HTTP/2 or FastCGI.
New directive http-check send to customize fully an HTTP health check, including sending a custom method, URI, headers, and POST body data.
New {http|tcp-check} comment directives to define a comment to report in the logs if the http-check rule fails.
Ability to query multiple endpoints at once to allow for multistage health checking.
New directives {http|tcp}-check set-var and {http|tcp}-check unset-var to set or unset a variable during HTTP and TCP health checks.
MySQL based health checks using the option mysql-check directive were rebuilt on top of the new tcp-check rules. They now default to a MySQL 4.1 and above client-compatible check when there is a defined username.

Syslog over TCP

New support for sending to syslog over TCP.
A new section ring to define custom ring buffers to use for queuing up messages and ensuring message delivery.

Performance improvements

Automatic deduplication of ca-file and crl-file directives to improve the overall startup speed.
New thread-local pool of recently used pipes to improve cache locality and eliminate unnecessary allocation.
ACL unique id value improvements.
Significant reduction of the number of syscalls per request for a connection using keep-alive mode.
Memory pools now release when there are an abundance of objects created after a traffic surge.
The connection layer underwent several performance improvements, essentially resulting in fewer syscalls on average, primarily for epoll.
HAProxy Enterprise no longer closes a client connection after it serves an internal response code (such as a 401 or 503), unless requested. Note: Status codes 400 (Bad Request) and 408 (Request Timeout) are excluded from this.
A new directive pool-low-conn to optimize server connection pooling. It is also possible to tune it to indicate the number of required idling connections to a server before a thread can reuse a connection.
The scheduler is now also latency aware, which means that the Runtime API is usable regardless of HAProxy Enterprise’s load.

Observability & debugging

A new Runtime API command show servers conn to see the current and idle connection state of the servers within a backend.
HAProxy Enterprise Enterprise Stats page now reports connect, queue and response time metrics with more accuracy.
A new timing metric, %Tu to return the total estimated time as seen from the client, from the moment the proxy accepts the request to the moment both ends get closed, not including the idle time before the request began.
Improved HAProxy Enterprise internal watchdog and expanded support to FreeBSD.
The debug converter is now always available and sends the output to a defined event sink. The currently available event sinks are buf0, stdout and stderr.

HTTP actions

A new ruleset http-after-response that HAProxy Enterprise evaluates on all responses prior to forwarding.
A new http-{request|response|after-response} action called strict-mode to enable or disable a strict rewriting mode on all the rules that follow it.
A new http-request action called replace-path is similar to replace-uri, except that it only acts on the path component.

Security hardening

HAProxy Enterprise now prevents the creation of new processes at default, effectively disabling the use of external programs for checks completely.
A new global directive insecure-fork-wanted to disable the above security capability and allow the use of the external-check command.
HAProxy Enterprise now prevents the process from executing setuid binaries by default to prevent it from switching uids after the initial switch to the uid defined within the global section.
To re-enable the execution of setuid binaries, you can use the new global directive insecure-setuid-wanted.

Fetches

Name	Description
`fc_pp_unique_id`	Returns the unique ID TLV from the client in the PROXY protocol header, if any.
`res.body`	Returns the HTTP response’s available body as a block of data.
`res.body_len`	Returns the length of the HTTP response available body in bytes.
`res.body_size`	Returns the advertised length of the HTTP response body in bytes. It will represent the advertised Content-Length header, or the size of the available data in case of chunked encoding.
`res.hdrs`	Returns the current response headers as string including the last empty line separating headers from the response body.
`res.hdrs_bin`	Returns the current response headers contained in preparsed binary form. This is useful for offloading some processing with SPOE.
`so_name`	Returns a string containing the current listening socket’s name, as defined with name on a bind line.
`ssl_fc_client_early_traffic_secret`	Returns the `CLIENT_EARLY_TRAFFIC_SECRET` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_client_handshake_traffic_secret`	Returns the `CLIENT_HANDSHAKE_TRAFFIC_SECRET` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_client_traffic_secret_0`	Returns the `CLIENT_TRAFFIC_SECRET_0` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_exporter_secret`	Returns the `EXPORTER_SECRET` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_early_exporter_secret`	Returns the `EARLY_EXPORTER_SECRET` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_server_handshake_traffic_secret`	Returns the `SERVER_HANDSHAKE_TRAFFIC_SECRET` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_fc_server_traffic_secret_0`	Returns the `SERVER_TRAFFIC_SECRET_0` as a hexadecimal string when the incoming connection was made over TLS 1.3.
`ssl_s_der`	Returns the DER formatted certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_key_alg`	Returns the name of the algorithm used to generate the key of the certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_notafter`	Returns the end date presented by the server as a formatted string YYMMDDhhmmss[Z] when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_notbefore`	Returns the start date presented by the server as a formatted string YYMMDDhhmmss[Z] when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_i_dn([<entry>[,<occ>[,<format>]]])`	When the outgoing connection was made over an SSL/TLS transport layer, returns the full distinguished name of the issuer of the certificate presented by the server when no is specified, or the value of the first given entry found from the beginning of the DN.
`ssl_s_s_dn`	When the outgoing connection was made over an SSL/TLS transport layer, returns the full distinguished name of the subject of the certificate presented by the server when no is specified, or the value of the first given entry found from the beginning of the DN.
`ssl_s_serial`	Returns the serial of the certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_sha1`	Returns the SHA-1 fingerprint of the certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_sig_alg`	Returns the name of the algorithm used to sign the certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.
`ssl_s_version`	Returns the version of the certificate presented by the server when the outgoing connection was made over an SSL/TLS transport layer.

Converters

Name	Description
`cut_crlf`	Cuts the string representation of the input sample on the first carriage return (‘\r’) or newline (‘\n’) character found.
`digest (<algorithm>)`	Converts a binary input sample to a message digest.
`hmac (<algorithm>, <key>)`	Converts a binary input sample to a message authentication code with the given key. The result is a binary sample.
`htonl`	Converts the input integer value to its 32-bit binary representation in the network byte order.
`ltrim (<chars>)`	Skips any characters from from the beginning of the string representation of the input sample.
`rtrim (<chars>)`	Skips any characters from from the end of the string representation of the input sample.
`secure_memcmp (<var>)`	Compares the contents of `with the input value. Both values are treated as a binary string. Returns a Boolean indicating whether both binary strings match.`

Lua

You can now prepend the lookup path for Lua modules using lua-prepend-path.
It is now possible to intercept HTTP messages from a Lua action and reply to clients.
Lua declared actions can now yield using wake_time (). This function can define a timeout when a Lua action returns act:YIELD. It is a way to force the script to re-execute after a short time (defined in milliseconds).
set_var and unset_var now returns a Boolean to indicate success.
A new parameter ifexist is added to set_var to allow a Lua developer to set variables to ignore unless the variable name was used elsewhere before.
The Server class now has a set_addr function to change a backend server’s address and port.
A new function is_resp to determine whether a channel is a response channel.

Miscellaneous improvements

The parser now supports quotes, braces, and square brackets in arguments.
The parser now also shows you the location of where a parsing error occurred.
The use-server directive now supports rules using dynamic names.
The bits argument for the sha2 converter will now properly be verified.
Added the ‘sub-second’ and ‘timezone’ fields to the RFC5424 log format.
The number of connections reported in the output of a quitting proxy now indicates cumulative connections and not active connections.
The Runtime API’s show table command now supports filtering stick table output by multiple filters, allowing for filtering on many columns.
DNS Service Discovery now reuses information available within the extension parts of an SRV record response.
The cookie directive now has an ‘attr’ field to set attributes on persistence cookies. This is helpful when adding the ‘SameSite’ attribute, which is required in Chrome 80 and above.
You can specify the local peer name with ‘localpeer’ within the peers section. You can override it with the -L parameter on startup.
The Runtime API now allows for escaping spaces.
ACLs can no longer be named “or”.
Error files that are larger than tune.bufsize now emit a warning message on startup.
The http-request deny directive now supports returning status codes ‘404 Not Found’, ‘410 Gone’, and ‘413 Payload Too Large’.
Improved UUID random generation is now thread safe.
You can now send and receive a unique-id in the PROXY Protocol for connection tracking purposes.
The default maxconn is now automatically set based on the configured ulimit -n.
Invalid hex sequences now cause a fatal error.
A new option pp2-never-send-local to revert the old bogus behavior on the server side when using proxy-protocol-v2 in health checks.

Version 2.1r1 Jump to heading

Key changes in the HAProxy Enterprise 2.1r1 release include:

Strict Rewriting Mode

By default, HAProxy Enterprise triggers an internal error when a rule that performs a rewrite on an HTTP message fails.

New HTTP Errors section

Create global groups of custom HTTP error files by using the new section http-errors.
Support the loading of http-errors groups using the expanded directives error-files, http-request deny, and http-response deny.

New HTTP Return Actions

Return a custom response from HAProxy Enterprise with any status code based on an error file, a file, or a string using the new actions http-request return and http-response return.
Enable responses with dynamic content by using a log-format string or a log-format file.
Define extra headers by passing the hdr argument

HTTP “After Response” Rulesets

Evaluate a new ruleset http-after-response on all responses prior to forwarding
Let HAProxy Enterprise evaluate these rules at the end of the response analysis on all HTTP responses, just before it forwards the data. This includes responses from the server as well as responses from HAProxy. This makes it possible to add headers to the responses that the stats applet generates.

Cookie Attributes

Supply an attr option to insert any attribute when HAProxy Enterprise inserts a cookie.
Use with the Chrome 80 update that requires the “SameSite” attribute. (Example: cookie SRV insert attr "SameSite=Strict")
Repeat the attr option to add several attributes.

Dynamic SSL Certificate Updates

Centralized SSL certificate information that only loads once when multiple bind lines reference the same certificate.
Ability to update SSL certificates with the Runtime API using the set ssl cert and commit ssl cert commands.

FastCGI

Direct communication between HAProxy Enterprise and FastCGI
New protocol fcgi
Definition of parameters for communicating with a FastCGI application in a new section called fcgi-app
Backends can relay requests to a defined application using the use-fcgi-app directive

Native Protocol Tracing

Integration of a new tracing infrastructure to allow systems engineers and developers to collect low-level trace messages
Tracing ability and access through the Runtime API using the trace and show trace commands

Removal of the File Descriptor Cache

Complete removal of the file descriptor (FD)
This change has shown an increase in performance of up to ~20% on some artificially tailored workloads. Realistically, production environments can expect to see a 5-10% improvement.

Scheduler Improvements

Improved internal scheduler supports waking up tasks that belong to another thread
The scheduler now uses a combination of a locked and a lock-free list to regain 5-10% performance on workloads involving high connection rates.

Defaulted HTTP Representation to HTX

Removal of support for legacy HTTP mode
Support only of the Native HTTP Representation (HTX)
No configuration change is necessary, unless you try to specify no option http-use-htx, in which case you get an error.
Assistance for a seamless transition from legacy applications with new global directives h1-case-adjust and h1-case-adjust-file.
Option case-adjust-bogus-client and option h1-case-adjust-bogus-server directives to enable explicitly the case adjustment within defined frontend, listen, and backend sections.

Fetches

Name	Description
`srv_name`	Returns a string containing the name of the server that processed the request. It can be useful to return this to the client for debugging purposes.
`srv_queue`	Takes an input, either a server name or / format and returns the number of queued sessions on that server.
`fc_pp_authority`	Returns the PP2_TYPE_AUTHORITY Type-Length-Value (TLV) from the client in the PROXY protocol header.
`uuid`	Returns a universally unique identifier (UUID) following the RFC4122 standard. Currently, there is only support for version 4.

Converters

Name	Description
`sha2(number_of_bits)`	Generates a checksum for a binary string using the SHA-2 cryptographic hash function. The result is a binary value with a byte length equal to number_of_bits / 8. You can set the number_of_bits parameter to 224, 256, 384, or 512. The default is 256.

Deprecated Configuration Options

Deprecated directive	Replacement directive
`block`	`http-request deny`
`reqrep`	`http-request <replace-uri\|replace-header>`
`rsprep`	`http-response replace-header`
`clitimeout`	`timeout client`
`contimeout`	`timeout connect`
`srvtimeout`	`timeout server`
`redispatch`	`option redispatch`
`resolution_pool_size`	NONE
`option independant-streams`	`option independent-streams`

Strict Limits Setting

Allows HAProxy Enterprise to abort at startup if it cannot get the required limits, such as in cases where HAProxy Enterprise is unable to increase necessary limits upon startup.
A new global directive strict-limits will cause HAProxy Enterprise to fail to start if it cannot increase the limits through setrlimit().

Version Info Show Links

Passing -v now displays End of Life (EOL) information for this release.

Runtime API Field Descriptions

The show info and show stat Runtime API commands now accept a new parameter called desc that adds a short description to each field.

Prometheus Improvements

You can now pass a new scope query string parameter to filter exported metrics. The following values are supported: global, frontend, backend, server, * (all).

Miscellaneous Improvements

Moving the storage of the server-state global file to a tree, which provides much faster reloads.
Acceptance as an expression: http-request / http-response sc-set-gpt0.
The directive resolve-opts now accepts ignore-weight. Hence, when HAProxy Enterprise generates servers with DNS SRV records, it can set server weights dynamically using agent health checks or the Runtime API, and not have DNS SRV reset the weights subsequently.
Ability to export the Stats page in JSON format by appending “/;json” to the URI.
Ability to send the PP2_TYPE_AUTHORITY value to allow it to chain layers using SNI, using the new directive send-proxy-v2.
Additional support for the user and group directives in the program’s Process Manager section.
Connections require significantly less memory as HAProxy Enterprise allocates dynamically the source and destination addresses as needed. This translates to 128 to 256 bytes saved per connection and per side in the common case.

Version 2.0r1 Jump to heading

Key changes in the HAProxy Enterprise 2.0r1 release include:

Enterprise modules

Javascript Challenge module now supports templates
DeviceAtlas module
ScientiaMobile WURFL module
Extensive ModSecurity hardening improvements

Cloud-native threading

HAProxy Enterprise sets the number of worker threads to match the machine’s number of available CPU cores to scale and accommodate any environment with less manual configuration.

Cloud-native logging

Easier to adapt for containerized environments, allowing you to log directly to stdout and stderr or to a file descriptor.

HTTP representation (HTX)

Introduced with HAProxy Enterprise 1.9, HTX makes any future HTTP protocols easier to integrate. It is enabled by default.

End-to-End HTTP/2

Official support for end-to-end HTTP/2 through new parameters alpn h2 and proto h2.

gRPC

Full support for the open-source gRPC framework. This allows for bidirectional streaming of data, detection of gRPC messages, and logging gRPC traffic. Activated using a standard end-to-end HTTP/2 configuration.

Layer 7 retries

Can retry a connection at Layer 7 for failed HTTP requests using the new configuration directive retry-on.

Data Plane API

Provides a modern REST API to configure HAProxy Enterprise on the fly by dynamically adding and removing frontends, backends, and servers. You can create ACL rules, insert HTTP routing directives, set IP and port bindings, etc.

Process Manager

Allows you to specify external binaries for HAProxy Enterprise to start and manage as child processes.

Traffic mirroring

Allows you to mirror requests from one environment to another. The new Traffic Shadowing daemon is developed as a Stream Processing Offload Agent (SPOA) and takes advantage of HAProxy Enterprise’s SPOE which allows you to extend HAProxy Enterprise using any programming language.

Kubernetes Ingress Controller

Provides a high-performance ingress for your Kubernetes-hosted applications. It supports TLS offloading, Layer 7 routing, rate limiting, IP allow lists,

Prometheus Exporter

HAProxy Enterprise can export metrics to Prometheus for monitoring and alerting purposes.

Peers & Stick Tables

HAProxy Enterprise 2.0 introduces several improvements to the Peers Protocol including:

Heartbeat
Stick tables in peers sections
SSL support
Runtime API command: show peers
New stick table counters
New stick table data type, server_name
Peers section expanded to allow using the bind, default-bind, server, and default-server configuration directives. It also now supports having stick tables directly within itself.

Power of Two Random Choices Algorithm

Added a new load-balancing algorithm called random that chooses a random number as the key for the consistent hashing function. This is useful with large farms or when servers are frequently added or removed.

Log Distribution and Sampling

It is now possible to do sampling directly within HAProxy Enterprise by using the log directive’s sample parameter. You can specify multiple log and sample directives simultaneously.

Built-in Automatic Profiling

This version now features the profiling.tasks directive to specify in the global section. CPU profiling per task shows where the time is spent and which requests have what effect on which other request.

Enhanced TCP Fast Open (TFO)

Added TFO for connections to backend servers on systems that support it. This requires Linux kernel 4.11 or newer.

New request actions

Introduced several new http-request and tcp-request actions below:

http-request do-resolve: Performs DNS resolution of the output and stores the result in the variable.
http-request disable-l7-retry: Disables any attempt to retry the request if it fails for any reason other than a connection failure. This ensures that POST requests aren’t retried upon failure.
http-request replace-uri: Matches the regular expression in the URI part of the request and replaces it.
tcp-request content do-resolve: Performs DNS resolution of the output and stores the result in the variable.
tcp-request content set-dst: Sets the destination IP address.
tcp-request content set-dst-port: Sets the destination port.

New converters

Converters allow you to transform data captured by fetch methods. Below are new converters in this version:

aes_gcm_dev: Decrypts the raw byte input using the AES128-GCM, AES192-GCM or AES256-GCM algorithm.
protobuf: Extracts the raw field of an input binary sample representation of a Protocol Buffers message.
ungrpc: Extracts the raw field of an input binary sample representation of a gRPC message.
sha2: Converts a binary input sample to a digest in the SHA-2 family
srv_queue: Takes an input value of type string and returns the number of queued sessions on that server

New fetches

Fetches provide a source of information from either an internal state or from layers 4, 5, 6, and 7. New fetches in this release return a random of the front or back connection when the incoming connection was made over an SSL/TLS transport layer. This release’s new fetches (below) return a random of the front or back connection when the incoming connection was made over an SSL/TLS transport layer:

ssl_fc_client_random: Returns the client random of the front connection
ssl_fc_server_random: Returns the server random of the front connection
sl_bc_client_random: Returns the client random of the back connection
ssl_bc_server_random: Returns the server random of the back connection; this requires OpenSSL >= 1.1.0, or BoringSSL.
uuid: Returns a UUID following the RFC4122 standard
http_auth_type: Returns the authentication method from the Authorization header
http_auth_user: Returns the authorized user from the Authorization header
http_auth_pass: Returns the authentication password from the Authorization header

Miscellaneous improvements

TLS session tickets help to speed up session resumption for clients that support them. HAProxy Enterprise 2.0 adds support for AES256-bit ticket keys specified in both a file or through the Runtime API.
A new global directive set-dumpable makes it easier to retrieve a core file.
Introduces 2 new server keywords, socks4 and check-via-socks4 used for communicating with servers within a backend over SOCKS4 and adds similar functionality for health checking over SOCKS4.
Stats page export in JSON
Additional support for logging to a ring buffer
SSL memory enhancements
FastCGI support

Version 1.9r1 Jump to heading

Key changes in the HAProxy Enterprise 1.9r1 release include:

Optimizations

Connection scheduling reworked to feature true ASYNC-I/O at every level
Varnish Test suite Integration
HTTP Status code 103 (Early Hints)
Support for HTTP Status code 421 (Misdirected Request)
CLI / Runtime API supports payloads
Master/Worker CLI (master has its own socket, communicate with all workers, even those exiting)
Server queues now have a “set-priority” option: deliver JS/CSS before images, boost premium level users compared to regular ones, or give lower priority to bots
Stick-tables extended with gpc1/gpc1_rate counters
Buffer modifications
Improved task scheduler: scales much better with large thread counts
Native HTTP Representation (HTX) which immediately brings:
- End-to-end HTTP/2
- Server-side connection pooling
- Server-side connection multiplexing
- gRPC

Cache improvements

Age headers
Chunked Transfer Encoding support
max-object-size = 2GB, total-object-size = 4GB

New converters

strcmp
concat
length
crc32c
ipv6 added to “ipmask” converter
field/word converter extended

New fetches

date_us: Microseconds part of the date
cpu_calls: Number of calls to the task processing the stream or current request since it was allocated. It is reset for each new request on the same connection.
cpu_ns_avg: The average number of nanoseconds spent in each call to the task processing the stream or current request.
cpu_ns_tot: Total number of nanoseconds spent in each call to the task processing the stream or current request.
lat_ns_avg: Average number of nanoseconds spent between the moment the task handling the stream is woken up and the moment is is effectively called.
lat_ns_tot: The total number of nanoseconds between the moment the task handling the stream is woken up and the moment it is effectively called.
srv_conn_free / be_conn_free : Determine the number of available connections on server/backend.
ssl_bc_is_resumed : Returns true when the back connection was made over an SSL/TLS transport layer and the newly created SSL session was resumed using a cached session or a TLS ticket.
fe_defbe: Fetches frontend default backend name.
ssl_fc_session_key / ssl_bc_session_key: Returns the SSL master key of the front/back connection.
ssl_bc_alpn / ssl_bc_npn: Provides the ALPN and the NPN for an outgoing connection.
prio_class: Returns the priority class of the current session for http mode or the connection for TCP mode.
prio_offset: Returns the priority offset of the current session for http mode or the connection for TCP mode.

Other enhancements

Random based load-balancing algorithm
Cloud Native Logging (Log to stdout / fd)
“Resolvers” section now supports resolv.conf
“Show activity” - shows the average and total CPU time consumed by the processing of each task, as well as the average and total latency inflicted by the processing of all other tasks. Values can also be logged if profiling is enabled either through the global section using “profiling.tasks on” or through the Runtime API.
Busy-polling - allows reduction of request processing latency by 30 - 100 microseconds on machines using frequency scaling or supporting deep idle states.
Lua Server class gained the ability to change a server’s maxconn value
Lua TXN class gained the ability to adjust a connections priority within the server queue
Lua has a new StickTable class that allows access to the content of a stick-table by key and allows dumping of an entire stick-table
SSL: Added support of AES-256 bits ticket keys on file and CLI
51Degrees: Enabled multi-threaded operation in the 51Degrees module
Prometheus-exporter: New Prometheus exporter for HAProxy Enterprise
Listener: Implemented multi-queue accept for threads
Listener: Used the multi-queue for multi-thread listeners
Configuration: Added global tune.listener.multi-queue setting
Threads: Enabled one thread per CPU by default
Configuration: Disabled support for nbproc and nbthread in parallel
Backend: Made the random algorithm support a number of draws
Multiplex: Made the H2 MAX_FRAME_SIZE setting configurable
Configuration: No longer enforced a low frontend maxconn value

Version 1.8r2 Jump to heading

Key changes in the HAProxy Enterprise 1.8r2 release include:

Enterprise modules

New netacuity module which allows the loading of DigitalElement/NetAcuity geolocation database into HAProxy Enterprise. This module also allows for live updating from a central URL at a defined interval.
New maxmind module which allows the loading of MaxMind GeoIP databases into HAProxy Enterprise. This module also allows for live updating from a central URL at a defined interval.
New send-metrics module allows HAProxy Enterprise to send statistics at a defined interval to an external HTTP/HTTPS based API. This is useful for integrating with services such as NS1, which can collect HAProxy Enterprise statistics and make DNS decisions. This also allows for integration with graphing and monitoring solutions.
New htmldom module which allows HTML or Javascript to be injected into the response data.
New modsecurity module which allows the loading of modsecurity rulesets into HAProxy Enterprise, such as the OWASP CRS.

Feature enhancements

HTTP Status code 103 (Early Hints)
Support for HTTP Status code 421 (Misdirected Request)
Cloud Native Logging (Log to stdout / fd)
CLI / Runtime API supports payloads (ocsp, map)
Master/Worker CLI (the master has its own socket and communicates with all workers, even those exiting)
Server queues now have a set-priority option: Delivers JS/CSS before images; boosts premium level users compared to regular users, or gives lower priority to bots
Random based load-balancing algorithm
localpeer as an environment variable
stick-tables extended with gpc1/gpc1_rate counters

Cache optimization

Age headers
Chunked Transfer Encoding support
max-object-size = 2GB, total-max-size = 4GB
Updated list of cacheable status codes ( 204, 404, 405, 414, 501 )

New fetches

ssl_fc_session_key / ssl_bc_session_key: Returns the SSL master key of the front/back connection
set SSL_OP_PRIORITIZE_CHACHA: Uses the client’s preference when selecting the cipher suite
fe_defbe: Fetches frontend default backend name

New converters

strcmp
field/word converter extended
ipv6 added to ipmask converter

Stick Table Aggregator

Now supports:

SIGHUP to reload
Environment variables
source keyword for outbound connections
Syslog debug logging

Version 1.8r1 Jump to heading

Key changes in the HAProxy Enterprise 1.8r1 release include:

HTTP/2 Protocol Support

HAProxy Enterprise now supports HTTP/2 on the client side (i.e. the front-end sections) and can act as a gateway between HTTP/2 clients and your HTTP/1.1 and HTTP/1.0 applications.

Multi-threading Model

The multi-threading model is a feature that allows HAProxy Enterprise to start multiple threads within a single process.

Server-template Configuration Directive

The server-template configuration directive allows you to instantiate a number of placeholder backend servers in a single configuration line, instead of having to add tens or hundreds of configuration lines explicitly.

Dynamic Cookies

A new keyword dynamic for the directive cookie now exists to enable cookie-based application persistence.

Improved Runtime API

The Runtime API allows administrators and automation systems to interact with HAProxy Enterprise during runtime. HAProxy Enterprise version 1.8 comes with further improvements and new directives:

Directive	Action
`disable dynamic-cookie backend backend`	Disable dynamic cookie for the backend.
`enable dynamic-cookie backend backend`	Enable dynamic cookie for the backend.
`set dynamic-cookie-key backend <backend> <value>`	Set cookie key hash.
`set server <backend>/<server> agent-addr <addr>`	Change server agent address.
`set server <backend>/<server> agent-send <value>`	Change server agent string.
`set server <backend>/<server> fqdn <FQDN>`	Change server’s FQDN.
`show acl [<acl>]`	Show list of ACLs in HAProxy Enterprise or dump ACL contents.
`show cli sockets`	Show list of Runtime API clients.
`show fd`	Show list of open file descriptors (for debugging only)
`show info json`	Famous “show info” in JSON output format.
`show stat [{<iid>\|<proxy>} <type> <sid>] json`	Famous “show stat” in JSON output format format and displayed per frontend or backend
`show schema json`	Dump JSON schema used for “show stat” and “show info” commands

SSL/TLS Mode Async

The OpenSSL library version 1.1.0 and above supports asynchronous I/O operations. This means that the key computation phase (the heaviest part of the TLS protocol) can be run in a non-blocking way in HAProxy Enterprise.

New Sample Fetches

HAProxy Enterprise can extract data from traffic passing through it. We call these functions “sample fetches”. More information about sample fetches can be found under “Fetching Data Samples” in the HAProxy Enterprise documentation.

This version has a group of new sample fetches:

Fetch	Description
`distcc_body`	Parses distcc message and returns body associated to it.
`distcc_param`	Parses distcc message and returns parameter associated to it.
`hostname`	Returns system hostname.
`srv_queue`	Returns number of connections currently pending in server’s queue.
`ssl_fc_cipherlist_bin`	Returns binary form of client hello cipher list.
`ssl_fc_cipherlist_hex`	Same as above, but encoded as hexadecimal.
`ssl_fc_cipherlist_str`	Returns decoded text form of client hello cipher list.
`ssl_fc_cipherlist_xxh`	Returns xxh64 of client hello cipher list.
`req.hdrs`	Returns current request headers.
`req.hdrs_bin`	Returns current request headers in preparsed binary form.

New Converters

Converter	Description
`b64dec`	Decodes base64 encoded string into binary.
`nbsrv`	Returns number of available servers in a backend.
`sha1`	Computes SHA1 digest of binary input.
`xxh32`	Hashes binary input into unsigned 32 bit.
`xxh64`	Hashes binary input into unsigned 64 bit

Version 1.7r2 Jump to heading

Key changes in the HAProxy Enterprise 1.7r2 release include:

Stick Table Aggregator

The stick-table aggregator stores stick-table entries and aggregates them before it transfers the results back to HAProxy Enterprise peers on the network. Thus you can see and reference a single stick table which holds the combined values from all nodes in the cluster. You can find its user documentation on the HAProxy Enterprise Customer Portal.

Master-Worker Model

In the master-worker model, HAProxy Enterprise launches one master process and starts a number of additional worker processes under it.
The new master process monitors all worker processes and controls them from a single instance to relieve systemd or other software from these tasks.

HTTP Small Object Caching

The small object caching mechanism allows HAProxy Enterprise to store some small static files from application servers in memory, and thereby accelerate the delivery of CSS, JS, or icon files.

DNS SRV Record Support

DNS SRV records allow HAProxy Enterprise to get a list of backend servers from DNS records. This is an alternative to building the configuration from template whenever the backends change or using the run-time API to modify backends.

Version 1.7r1 Jump to heading

HAProxy Enterprise 1.7r1 comprises the following new functionalities:

Core improvements

PCRE JIT support
Namespace support
DNS improvements
Server: init-addr: indicate in what order the server’s address should be resolved upon startup. Method none specifically indicates that the server should start without any valid IP address in a down state.
New srv_admin flag: SRV_ADMF_RMAINT
Listener: accept-netscaler-cip option added to the bind keyword
Added tcp-request connection expect-netscaler-cip layer4
log-format: error management improvements

New Fetches

fe_name: gives the current frontend’s name
be_name: gives the current backend’s name
fc_rcvd_proxy shows if the client initiated the connection with a PROXY protocol header
xx-hash converters

Runtime API

show cli sockets list the CLI sockets
show stat now supports a proxy name
show errors now supports a proxy name
show errors is now capable of dumping only request or response
RMAINT state remains when setting an IP address on the CLI

LUA Scripting

CLI handler for LUA
Allow argument for actions
HAProxy Enterprise now has variable access to applets
Function which returns true if the channel is full
IP addresses and network manipulation function
Utility function to check for a boolean argument
A tokenize function

Version 1.6r2 Jump to heading

Key changes in the HAProxy Enterprise 1.6r2 release include:

HTTP REST API
List HAProxy Enterprise modules and their versions from the CLI
LUA support is now enabled in HAProxy Enterprise
New lb-update features
- TLS session ticket keys
- Ability to launch an update from the CLI
- Optional logs
- Various fixes
TCP layer statistics are available in the form of new sample fetches (rtt, rtt variance, retransmits, losses, etc.)
New dst_is_local, src_is_local sample fetches make it easier to take care of locally initiated connections in contrast with remote connections
The peers protocol was updated to version 2.1 with support for synchronization of expiration dates
CLI keyword registration to allow modules to plug-in on the command line and receive actions (for example: refresh now)
SO_REUSEPORT can now be disabled with a new bind directive: noreuseport
New possibilities for accessing load balancer internals with LUA
Idle time in logs is now ignored by default for time measurements so that HTTP request time corresponds to the time elapsed between the first character and the full request. The handshake time is also available to measure the time spent in SSL/PROXY handshakes.
stick-tables now use native types to guarantee better accuracy of tracked information, especially binary keys
Mailers now support a configurable connection timeout
Changing server address, port and checkport is now possible from the CLI
New tcp-request session rule-sets makes it possible to track some L5-only information, for example anything negotiated in the handshake such as SSL DN or the client’s IP address as passed by the PROXY protocol. Previously it was needed to do it in tcp-request content rules, which would count one new connection event per request.
New stats field for denied connections and denied sessions
New hash-balance-factor directive for consistent hashing method
New http-response rule track-sc
SSL/TLS
- SNI filters supported in multi-type certificates

Version 1.6r1 Jump to heading

Key changes in the HAProxy Enterprise 1.6r1 release include:

New http-reuse to share unused server-side connections for multiple clients
Support of double quotes in the configuration
DNS resolution of server name at run time
GZIP stateless compression
Support of variables at run time
Stick tables statistics synchronization
New hash-type load-balancing algorithm crc32
Redispatch interval
Dynamic generation of SSL certificates
Mathematical converters can use a variable generated at runtime
New http-request rules
- set-var(<var name>) <expr>
- sc-inc-gpc0(<sc-id>)
- sc-set-gpt0(<sc-id>) <int>
- silent-drop
- set-src <expr>
New http-response rules
- set-status
- set-var(<var-name>) <expr>
- sc-inc-gpc0(<sc-id>)
- sc-set-gpt0(<sc-id>) <int>
- silent-drop
New tcp-request connection rules
- sc-inc-gpc0(<sc-id>)
- sc-set-gpt0(<sc-id>) <int>
- silent-drop
New tcp-response content rules
- set-var(<var-name>) <expr>
- sc-inc-gpc0(<sc-id>)
- sc-set-gpt0(<sc-id>) <int>
- silent-drop
New tcp-request content rules
- sc-inc-gpc0(<sc-id>)
- sc-set-gpt0(<sc-id>) <int>
- set-var(<var-name>) <expr>
- silent-drop
New converters
- capture-req(<id>)
- capture-res(<id>)
- set-var(<var name>)
- table_gpt0(<table>)
New fetches
- Internal states
  - bin(<hexa>)
  - bool(<bool>)
  - int(<integer>)
  - ipv4(<ipv4>)
  - ipv6(<ipv6>)
  - meth(<method>)
  - str(<string>)
  - var(<var-name>)
- Layer 4
  - sc_get_gpt0(<ctr>[,<table>]), sc0_get_gpt0([<table>]), ...
  - src_get_gpt0([<table>])
- Layer 6
  - req.ssl_st_ext
  - res.ssl_hello_type
global section features
- presetenv
- resetenv
- setenv
- unsetenv
SSL/TLS - multi-certificate key algorithm support per certificate
New map type: map_regm to allow matching zones in the regex.
Layer 7 fetches
- unique-id
Pre-defined ACLs
- METH_DELETE
- METH_PUT
New filters trace and HTTP compression.

Version 1.5r2 Jump to heading

Key changes in the HAProxy Enterprise 1.5r2 release include:

Support for 2 new operating systems: Debian 8 and Ubuntu 12.04
Support of systemd on Debian 8, RedHat 7 and CentOS 7
Analysis of HTTP request body (req.body HAProxy Enterprise fetch)
Peers on a single process when nbproc > 1
Support for HTTP stateless compression
Configurable tls-tickets keys
Automatic maxconn and maxsslconn to match enforced memory limits
External checker (fork of current process) (be careful with chroots…)
tcp-check comments
Email alerts
Fetches for http request body
Server state reload proof
HTTP capture on response
HTTP capture with a slot number
Server side TLS extension SNI
Server side tcp-ut (TCP user timeout at kernel layer)
Management of environment variables through the global section
New HAProxy Enterprise fetches:
- ssl_fc_is_resumed
- req.ssl_ec_ext
- req.body
- req.body_param
- req.body_len
- req.body_size
New commands available over the stats socket:
- set ssl tls-key
- show backend
- show servers state
- show tls-keys
- show env

Product Documentation

API Documentation

Configuration Documentation

Install the VM image

Release notes

Version 3.1r1 Jump to heading

Version 3.0r1 Jump to heading

Version 2.9r1 Jump to heading

Version 2.8r1 Jump to heading

Version 2.7r1 Jump to heading

Version 2.6r1 Jump to heading

Version 2.5r1 Jump to heading

Version 2.4r1 Jump to heading

Version 2.3r1 Jump to heading

Version 2.2r1 Jump to heading

Version 2.1r1 Jump to heading

Version 2.0r1 Jump to heading

Version 1.9r1 Jump to heading

Version 1.8r2 Jump to heading

Version 1.8r1 Jump to heading

Version 1.7r2 Jump to heading

Version 1.7r1 Jump to heading

Version 1.6r2 Jump to heading

Version 1.6r1 Jump to heading

Version 1.5r2 Jump to heading

Install the VM image

Product Documentation

API Documentation

Configuration Documentation

Version 3.1r1 Jump to heading #

Version 3.0r1 Jump to heading #

Version 2.9r1 Jump to heading #

Version 2.8r1 Jump to heading #

Version 2.7r1 Jump to heading #

Version 2.6r1 Jump to heading #

Version 2.5r1 Jump to heading #

Version 2.4r1 Jump to heading #

Version 2.3r1 Jump to heading #

Version 2.2r1 Jump to heading #

Version 2.1r1 Jump to heading #

Version 2.0r1 Jump to heading #

Version 1.9r1 Jump to heading #

Version 1.8r2 Jump to heading #

Version 1.8r1 Jump to heading #

Version 1.7r2 Jump to heading #

Version 1.7r1 Jump to heading #

Version 1.6r2 Jump to heading #

Version 1.6r1 Jump to heading #

Version 1.5r2 Jump to heading #

Privacy Settings

Version 3.1r1 Jump to heading

Version 3.0r1 Jump to heading

Version 2.9r1 Jump to heading

Version 2.8r1 Jump to heading

Version 2.7r1 Jump to heading

Version 2.6r1 Jump to heading

Version 2.5r1 Jump to heading

Version 2.4r1 Jump to heading

Version 2.3r1 Jump to heading

Version 2.2r1 Jump to heading

Version 2.1r1 Jump to heading

Version 2.0r1 Jump to heading

Version 1.9r1 Jump to heading

Version 1.8r2 Jump to heading

Version 1.8r1 Jump to heading

Version 1.7r2 Jump to heading

Version 1.7r1 Jump to heading

Version 1.6r2 Jump to heading

Version 1.6r1 Jump to heading

Version 1.5r2 Jump to heading