HAProxy 2.9 is now the latest version. Learn more
HAProxy 2.8 is now available, and HAProxy Enterprise 2.8 will be released later this year.
HAProxy 2.8 contains features that build upon HAProxy's flexibility, performance, and programmability. This version gives you more control over how to tune the load balancer for your use case, while also setting default values to make life easier. Performance enhancements include HTTP compression for requests, a simple way to combine listener shards and thread groups, and built-in OCSP stapling. Anyone using Lua to customize HAProxy will be pleased with the new event framework, queues, and functions for retrieving server information in Lua code.
HAProxy 2.8 cements HAProxy’s reputation as the world’s fastest software load balancer, and extends HAProxy’s versatility for new and experienced users alike, making HAProxy the best choice for delivering modern applications at enterprise-scale.
This release came together through the efforts of all the community members who got involved. A release like this requires feature requests, bug reports, forum discussions, code submissions, QA tests, and documentation! In other words, this project is fueled by people like you! If you're interested in joining this vibrant community, it can be found on GitHub, Slack, Discourse, and the HAProxy mailing list.
How to Get HAProxy 2.8
You can install HAProxy version 2.8 in any of the following ways:
Compile it from source. View the compilation instructions.
Email Alerts With Lua
This syntax for configuring email alerts in HAProxy stays the same in HAProxy 2.8, but the underlying implementation, previously compiled into HAProxy's C code, is now duplicated as a Lua module. As a Lua module, you can more easily edit the code to customize the format of the emails and add extra, relevant information, such as links to operating manuals and on-call personnel. Because Lua is a scripting language, you can edit the file with a text editor, save, restart the HAProxy service, and the changes will be in effect.
HAProxy can notify you by email when important server state changes occur, such as when HAProxy removes a server from load balancing due to failing health checks or adds it back due to passing health checks. The mailers section in a HAProxy configuration lists the addresses of your SMTP mail servers.
Read More: How to Create Lua Mailers in HAProxy
A simple example of a
|mailer mailserver1 mailserver1.example.com:25
Then, to enable notifications, you would add
email-alert directives to your defaults:
|email-alert mailers smtp_servers
|email-alert from email@example.com
|email-alert to firstname.lastname@example.org
|email-alert level info
To enable the Lua incarnation of this feature, and disable the C implementation, download the mailers.lua file from the examples folder in the GitHub repository and then add a
lua-load directive to the global section of your configuration:
To test email alerts with a fake mail server, try the open-source smtp4dev.
New Event Framework in Lua
If you extend HAProxy's functionality by writing a Lua module, that module can now take advantage of the new HAProxy Lua event framework. Lua code can already hook into HAProxy in a number of ways, including:
as actions that authenticate, validate, or alter requests and responses on the fly
services that intercept traffic and immediately return a response
tasks that run continuously in the background
custom fetches and converters that capture and transform data as it passes through.
The event framework augments all of that by letting you subscribe to events and then invoke a callback function when those events fire.
The email alerts Lua module mentioned before is a good example of how to leverage events. Essentially, there is a new function named
core.event_sub(), which lets you subscribe to events that occur across all servers, and also a function named
Server.event_sub(), which lets you subscribe to events related to a specific backend server. There's also an unsub() function for unsubscribing from events.
The following event types exist:
when a server is added
when a server is removed
when a server state goes from UP to DOWN
when a server state goes from DOWN to UP
when a server state changes
when a server administrative state changes (goes into drain mode)
when a server's check status changes
Queues in Lua
With HAProxy 2.8, your Lua code can utilize queues, first-in-first-out storage, for transferring data between coroutines. Lua does not provide such a structure on its own.
The email alerts Lua module provides an example of using a queue. It adds alert messages to a queue when a server event fires. Then, a separate background task fetches messages from the queue to send them as email notifications. Central to this feature is the
core.queue() function, which returns a Queue object.
Lua Functions for Server Information
Custom Lua modules can leverage new functions for accessing information about backend servers. The table below gives a brief description:
Returns the name of the server.
Returns the proxy's unique identifier of the server.
Returns the RID (revision ID) of the server.
Returns true if the server is a backup server.
Returns true if the server was instantiated at runtime via the Runtime API.
Returns the number of currently active sessions on the server.
Returns the number of pending connections to the server.
Returns the parent proxy to which the server belongs.
Checks if the current server is tracking another server.
Checks if the current server is being tracked by other servers.
Registers a function that will be called on specific server events.
A Timeout for Lua Execution
tune.lua.burst-timeout squashes the problem of Lua scripts that take too long to execute due to inefficient code. It kills them after a period of time, with a default period of one second. This safeguards the load balancer from locking up while waiting for a script to complete and could help locate custom Lua modules in need of optimization. You can disable this by setting the value to zero.
Disable the Default Resolvers Section for the HTTP Client
You can now disable the creation of the default
resolvers section, which could be helpful to anyone not making use of Lua modules that use the HTTP client.
Since version 2.6, HAProxy has generated an invisible DNS
resolvers section to support domain name resolution for the Lua HTTP client. This default resolver parses the
/etc/resolv.conf file on Linux to give generic DNS resolution support to Lua modules that use
core.httpclient() for making HTTP requests.
You can disable this by adding the global configuration option
HTTP Compression of Requests & Responses
HAProxy 2.8 expands its HTTP compression feature so that you can now compress requests in addition to responses. Request compression aids the transfer of large data sets to backend servers, which can save you money in cloud environments where you pay for data sent between regions. The new syntax has you specify whether you want to compress requests, responses, or both. In the example below, we want to compress JSON sent to the server, so we configure HAProxy to compress requests that have a
Content-Type header set to
|server web1 192.168.56.10:8080 check maxconn 30
|compression direction both
|compression algo-req gzip
|compression type-req application/json
|compression algo-res gzip
The backend server must have the ability to decompress the messages.
Signing Algorithms for TLS
You can now specify which signing algorithms HAProxy supports for digital signatures used in TLS handshakes and client certificate authentication. When using client certificate authentication, the client sends a certificate to HAProxy as proof of identity. The certificate itself will have been digitally signed by a CA your organization trusts. Additionally, the client will use its private key to sign a CertificateVerify message, proving that it is the rightful owner of the certificate. Digital signatures also come into play during the TLS handshake, when the server (HAProxy) signs the ephemeral key exchanged with the client.
Two new global directives allow HAProxy to tell the client which cryptographic algorithms it supports when producing digital signatures, enabling you to disallow weak cryptography such as SHA-1, when supporting older clients is not a concern.
Lists signing algorithms HAProxy relays to the client in a CertificateRequest message. The client should use only these supported algorithms in its CertificateVerify message sent as part of certificate authentication.
Lists signing algorithms HAProxy can use when signing the ephemeral public key sent to the client during the TLS handshake.
Additionally, you can set supported algorithms on individual
bind lines in
frontend sections by using the
Default ALPN Values
When you do not set the
alpn argument on a
bind line, HAProxy 2.8 now sets a default value of h2,http/1.1 for you. Also,
bind lines that use the QUIC protocol will get a default ALPN value of h3.
ALPN (Application-layer Protocol Negotiation) is a TLS extension that lets a client and HAProxy agree on a mutually supported protocol to use for communication. It's how a client can switch from HTTP/1.1 to HTTP/2 without switching to a different port. Exchanging this info during the TLS handshake saves a network round trip.
The new default values tell the client that HAProxy prefers HTTP/2 and HTTP/3, but can fall back to HTTP.1.1 if the client doesn't support them. You can also disable ALPN by adding the
no-alpn argument to the
HAProxy 2.8 includes improved support for OCSP stapling, making it a built-in feature of HAProxy. Previously, this functionality could be achieved only through custom scripts that called the HAProxy Runtime API's
set ssl ocsp-response method.
OCSP (Online Certificate Status Protocol) provides an efficient way for clients to check the revocation status of TLS certificates. HAProxy supports OCSP stapling wherein it sends revocation status information to clients preemptively. To configure this feature, you would set
on on a certificate line in a CRT list to enable automatic updates to that certificate's OCSP file.
Two new global directives give you control over how often OCSP responses are updated. The
tune.ssl.ocsp-update.mindelay directive sets the minimal interval between updates, while
tune.ssl.ocsp-update.maxdelay sets the maximum interval.
There are also new Runtime API commands:
update ssl ocsp-response for requesting an updated OCSP response immediately and
show ssl ocsp-updates for viewing the expected time of the next update and the status of the last update.
show ssl ocsp-response method got a refresh that allows you to view OCSP response data in text format or as a base64-encoded string.
A lot of progress has been made to support WolfSSL as a TLS encryption library. WolfSSL has built-in support for the QUIC protocol and shows good performance. HAProxy engineers continue to work with WolfSSL engineers to iron out missing features, and you can expect a fully working version in the near future. Check the install guide in GitHub to learn details about which WolfSSL features are complete and which are in progress.
HTTP Forwarded Header
A new option adds a Forwarded header to requests, making it simple to pass the client's source IP address and information about the HAProxy server that handled the request to backend servers.
option forwarded directive enables the header, and you can configure whether to set the header's fields automatically or define expressions that generate the values in custom ways. For example, you could obfuscate the for field, which contains the client's source IP address, for privacy.
In addition to generating a Forwarded header, this version adds new converters for validating and extracting information from a Forwarded header found on the incoming request. The header is defined in RFC 7239, which explains the naming convention:
Returns true if input header is an RFC 7239 compliant header value and false otherwise.
Extracts the by, for, host, or proto field from an RFC 7239 compliant header.
Converts an RFC 7239 node (provided by 'for' or 'by' 7239 header fields) into its corresponding node name final form (for example, into an IPv4 address).
Converts an RFC 7239 node (provided by 'for' or 'by' 7239 header fields) into its corresponding node port final form (for example, into a port number).
Set the Number of Sticky Counters
It is now possible to change the number of stick table sticky counters available within your HAProxy configuration.
HAProxy's in-memory storage, called stick tables, have a finite number of slots in which to store connection tracking information. Historically, that number defaulted to 3 and was a compile-time constant, which you could not change later. Now, just set the
tune.stick-counters directive in the global section. Note that the higher the number, the more memory HAProxy will consume per HTTP request.
HAProxy 2.8 gains the following
Deletes an entry from an ACL.
Deletes an entry from a map.
Increments a general purpose counter at the given index of the array associated with the sticky counter.
Increments a general purpose counter at the given index of the array associated with the sticky counter.
Increments the general purpose counter GPC0 associated with the given sticky counter.
Increments the general purpose counter GPC1 associated with the given sticky counter.
Sets the general purpose tag at the given index of the array associated with the sticky counter.
Sets the general purpose tag GTP0 associated with the sticky counter.
Changes the log level of the current request.
Adds a new entry to a map.
Additionally, all action directives now have an
sc-add-gpc action, which increments a general-purpose counter. General-purpose counters are useful for defining custom counters stored in stick tables. For example, you could count each time a client attempts to log into your website as a way of detecting login brute forcing.
Tuning HTTP/2 Performance
New and updated global directives help you get the best performance when using the HTTP/2 protocol.
Sets the HTTP/2 initial window size for outgoing connections, which is the number of bytes the server can respond to before waiting for an acknowledgment from HAProxy.
Sets the HTTP/2 maximum number of concurrent streams per outgoing connection (i.e. the number of outstanding requests on a single connection to a server).
Sets the HTTP/2 initial window size for incoming connections, which is the number of bytes the client can upload before waiting for an acknowledgment from HAProxy.
Sets the HTTP/2 maximum number of concurrent streams per incoming connection (i.e. the number of outstanding requests on a single connection from a client).
Sets the default value for the HTTP/2 initial window size, on both incoming and outgoing connections. This value is used for incoming connections when tune.h2.fe.initial-window-size is not set, and by outgoing connections when tune.h2.be.initial-window-size is not set.
Sets the default HTTP/2 maximum number of concurrent streams per connection (i.e. the number of outstanding requests on a single connection). This value is used for incoming connections when tune.h2.fe.max-concurrent-streams is not set, and for outgoing connections when tune.h2.be.max-concurrent-streams is not set.
Defaults for Listener Sharding
HAProxy 2.5 introduced a shortcut called shards for defining multiple listeners (
bind lines) in a frontend. The
shards argument on a
bind line replicates the listener so that threads can pull work from multiple listeners for the same application at once, reducing contention. This is especially useful on servers with many cores/threads in order to make the best use of multithreading.
In HAProxy 2.8, a new, global directive called
tune.listener.default-shards helps you set the optimal number of listener shards. Instead of using the
shards argument on a
bind line, this global directive sets the number of listeners to match either the number of threads or the number of thread groups. Or, you can set it to one listener per process. The per-thread-group option is the default.
If you want to override the global default, you can set the mode directly on the
bind line. When you don't set the
shards argument to a number, it expects a value of either
Thread groups, which became a stable feature in HAProxy 2.7, allow you to assign threads to groups. For example, two groups, each with 64 threads, and then assign each group to handle incoming connections on a listener. Having control over the number of threads in a group and then setting
tune.listener.default-shards to allocate a listener for each group provides a streamlined way to optimize for multithreading.
Other Performance Tuning Options
This version adds or updates directives for tuning HAProxy's performance on your system.
cpu-mapargument on a
bindline can now define multiple, comma-separated CPU sets (pairings of CPU number and thread ranges).
The global directive
tune.listener.multi-queuegives you more control over how work is distributed among threads assigned to a listener. The default value, on, sends work to the least busy thread. Setting this to fair distributes connections to threads in a round-robin rotation, which suits traffic consisting of short-lived connections.
Each thread is allocated an amount of memory that will be kept hot in the local cache. Although the default amount of 512 KB should suit most, the new
tune.memory.hot-sizeglobal directive lets you change it if needed.
threadargument on a
bindline now accepts multiple, comma-separated sets of threads allowed to process incoming connections for the listener, instead of just a single set.
An experimental, global directive called
tune.disable-fast-forwardallows you to disable data fast-forwarding. You can also set this using the HAProxy startup argument,
-dF. Use this only when directed by project developers to do so.
Fetching URL & Body Parameters
The fetch method for getting a parameter from an HTTP request body,
req.body_param, now accepts an additional parameter, the letter i, for finding the parameter by name but ignoring the case. Similarly, the
urlp_val fetch methods also accepts the i parameter for matching a URL parameter name while ignoring the case. This can be the crucial ingredient for making HAProxy's behavior consistent with a backend server's behavior in how they both interpret the same parameter.
Also, the new
param converter extracts name-value parameters from a string. It offers an alternative to using the
urlp fetch, but with the ability to set the expected delimiter to something other than an ampersand.
Getting the Round-Trip Time
Two new fetch methods,
bc_rttvar, return the round-trip time between HAProxy and a backend server, which can be helpful for troubleshooting. You can configure these to return the value in milliseconds or microseconds.
Runtime API Updates
HAProxy's Runtime API received updates to several of its methods:
When using the Runtime API interactively via the
promptnow takes an optional parameter,
timed. This causes the process's uptime to be displayed as part of the prompt. This timer can alert you to the fact that the process didn't reload when you expected it to and is running an older configuration, for example.
show quicmethod that was introduced in version 2.7 displays information about all active QUIC frontend connections. You can now change the display format to either oneline or full, depending on how verbose you want the output to be. With the QUIC protocol being implemented entirely in userland, as opposed to lower in the network stack, having built-in debugging tools is crucial.
Preprocessor directives enable you to change what goes into your HAProxy configuration based on conditional logic. For example, if you wanted to include a set of configuration lines only if the version of HAProxy supported them, you could add an if statement that checked for that.
HAProxy 2.8 adds two new preprocessor functions:
strstrfunction returns true if a given string is found within another string.
The enabled function returns true if the passed in option is enabled at runtime. Options you can check for are: POLL, EPOLL, KQUEUE, EVPORTS, SPLICE, GETADDRINFO, REUSEPORT, FAST-FORWARD, and SERVER-SSL-VERIFY-NONE.
The Let's Encrypt client, acme.sh, provides a simple way to integrate with Let's Encrypt for getting TLS certificates. As a UNIX shell script, it has no dependencies on programming languages like Python. As a part of this release, HAProxy now integrates with acme.sh to enable populating TLS certificates in HAProxy. We will publish a separate blog post soon on how to configure this integration.
HAProxy 2.8 has been made possible by a long list of contributors, all providing invaluable work and collaboration. Contribution to the project comes in all forms, from design choice discussions, bug reporting, testing development releases, and maintaining documentation, to assisting users on both Discourse and the mailing list, classifying Coverity reports, reviewing patches, and contributing code. With a contributor list too long to include here, please know that the community appreciates each and every one of you who made 2.8 possible!Subscribe to our blog. Get the latest release updates, tutorials, and deep-dives from HAProxy experts.