Release notes

Version 2.8r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.8r1 release include:

Email alerts as a Lua module

• The mailers feature, which lets you send email alerts about the load balancer, has been ported to be a Lua module. The benefit to you is the ability to edit the Lua code to suit your needs, such as to customize the alert messages that the load balancer produces. To enable it, load the mailers Lua module using the lua-load directive in the global section.

Lua updates

• A new event framework allows Lua modules to subscribe to some load balancer events and execute event handler functions. Events you can subscribe to include when servers are added, deleted, or go up or down. Use the function Server.event_sub to subscribe to events.
• A new core.queue function lets you create a first-in first-out data structure in Lua.
• The Server class gained new functions for accessing information about backend servers, including getting the server’s name (get_name), the number of active sessions (get_cur_sess), the number of pending connections (get_pend_conn), and other functions.
• A new global directive named tune.lua.burst-timeout terminates the Lua process if it exceeds the timeout value without ever yielding execution back to the load balancer. A well-behaved Lua module will yield execution on the current thread back to the load balancer during long-running tasks, to be continued later.
• The Lua’s HTTP client that ships with the load balancer is able to resolve DNS hostnames thanks to a default resolvers section invisibly added to the configuration. If you do not use the HTTP client in Lua, then you can disable this by setting the new global directive httpclient.resolvers.disabled to on.

HTTP compression

• The HTTP compression directives have an updated syntax that allows you to compress both responses and requests. For example, you can now compress requests that contain JSON messages before relaying them to backend servers. See the compression direction, compression algo-req, compression algo-res, compression type-req, and compression type-res directives.

TLS signing algorithms and curves

• The new global directives ssl-default-bind-client-sigalgs and ssl-default-bind-sigalgs, as well as the new bind arguments client-sigalgs and sigalgs let you list the TLS signing algorithms the load balancer will use or accept during a TLS handshake and when validating a client certificate.
• The curves argument, which sets a list of elliptic curve algorithms negotiated during the TLS handshake, is now availble on the server and default-server lines.

Default ALPN values

• You no longer need to specify alpn h2 on a bind line to enable HTTP/2. This is now the default value and will fall back to HTTP/1.1 if the client does not support HTTP/2. Also, bind lines that use the QUIC protocol will default to having an alpn argument set to h3 for HTTP/3.

OCSP stapling

• OCSP stapling, which improves network latency by sending TLS certificate revocation information to the client preemptively, is now a built-in feature. Set the new ocsp-update argument to on in a CRT file to enable automatic updating of a certificate’s OCSP file. Use the global directives tune.ssl.ocsp-update.mindelay and tune.ssl.ocsp-update.maxdelay to adjust the interval between updates. You can force an update with the Runtime API command update ssl ocsp-response, view the status with show ssl ocsp-updates, and view OCSP responses with show ssl ocsp-response.

HTTP Forwarded header

• The new directive option forwarded will add a Forwarded header to HTTP requests, passing the client’s IP address to backend servers. This is the successor to the non-standard X-Forwarded-For header. New converters let you validate and read the header: rfc7239_is_valid, rfc7239_field, rfc7239_n2nn, and rfc7239_n2np.

Configurable sticky counters

• The new global directive tune.stick-counters lets you increase the number of stick table sticky counters, which had previously been a built-in constant set to 3.

HTTP actions

• The http-after-response directive gained an expanded set of actions, including set-map, sc-add-gpc, set-log-level, and others.
• All action directives now support the sc-add-gpc action.

HTTP/2 tuning options

• New global directives enable tuning the HTTP/2 protocol. They include tune.h2.be.initialize-window-size, tune.h2.be.max-concurrent-streams, tune.h2.fe.initial-window-size, tune.h2.fe.max-concurrent-streams, tune.h2.initial-window-size and tune.h2.max-concurrent-streams.

Listener shards

• The new global directive tune.listener.default-shards simplifies setting shards. Sharding is a feature for servers with a high number of threads that makes it easier to replicate a bind line to create multiple listeners and assign worker threads to the listeners. The goal is to reduce thread contention. You can set the directive to a number of shards, to the value by-thread, or to the value by-group. Setting by-group is especially convenient since you can then group threads and assign them together to a listener, and so this is the default setting.

Other performance tuning options

• The bind line’s cpu-map argument, which associates threads with CPUs, now accepts a list, delimited by commas.
• The bind line’s thread argument can now be set to a comma-delimited list of thread ranges instead of just a single range.
• The new global directive tune.listener.multi-queue controls how work is distributed to threads assigned to a listener. A value of on, which is the default, passes work to the least busy thread. A value of fair passes work in a round-robin rotation.
• The new global directive tune.memory.hot-size allows you to set the amount of memory kept hot in the local cache, but the default value of 512 KB should be sufficient for most.

HTTP request body and URL parameters

• The req.body_param, url_param, and urlp_val fetch methods now accept an additional argument that enables case-insensitive matching of parameters, which makes it easier to configure the load balancer’s behavior to agree with the web server’s behavior. Also, a new converter named param extracts name-value parameters from a string while letting you set the delimiter used between parameters.

Round-trip time

• The new fetch methods bc_rtt and bc_trrvar return the round-trip time between the load balancer and a backend server. You can configure them to return values in milliseconds or microseconds.

Runtime API

• The prompt command now accepts an optional argument, timed, that will cause the interactive prompt to display the process’s uptime.
• The show quic command, which displays information about active QUIC connections on the frontend, can now display output in one of two formats: oneline or full.

Preprocessor directives

• This version introduces two new preprocessor functions. The strstr function returns true if the inspected string contains a given string. The enabled function returns true if the given option is enabled at runtime.

Let’s Encrypt acme.sh client

• With special support for the acme.sh Let’s Encrypt client, you can integrate HAProxy Enterprise with your automated TLS certificate infrastructure.

Proxy Protocol

• Proxy Protocol v2 has been updated in this version to allow the load balancer to extract extra information from the Proxy Protocol header when a TCP connection is established on the frontend. The new fetch method fc_pp_tlv supports extracting a TLV (Type-Length-Value) from the protocol header.

SSO module

• The SSO module, which enables a single sign-on integration with Windows Active Directory, now supports validating Kerberos tickets.

Version 2.7r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.7r1 release include:

HTTP/3 over QUIC

• This version adds support for the HTTP/3 protocol over QUIC. Because it requires a specialized version of the OpenSSL library, a separate install package is provided.

Debugging

• A new global directive named trace lets you configure the HAProxy Enterprise events tracing subsystem from the configuration file, whereas previously it was accessible only through the Runtime API’s trace command. This directive is experimental and requires you to set the global option expose-experimental-directives.

• You can anonymize the output of some Runtime API commands to mask IP addresses, names of sections, and hostnames. Add set anon on, followed by a semicolon, before a command such as show sess.

  nix
  echo "set anon on; show sess" |\
    sudo socat stdio /var/run/hapee-2.7/hapee-lb.sock

  nix
  echo "set anon on; show sess" |\
    sudo socat stdio /var/run/hapee-2.7/hapee-lb.sock

• You can also display a version of your HAProxy Enterprise configuration with IP addresses, section names, and hostnames masked by using the hapee-lb command’s -dC<key> flag, where key is an arbitrary number used when creating the masked values. For example, to display an anonymized version of the HAProxy Enterprise configuration:

  nix
  /opt/hapee-2.7/sbin/hapee-lb -dC123456 -f etc/hapee-2.7/hapee-lb.cfg

  nix
  /opt/hapee-2.7/sbin/hapee-lb -dC123456 -f etc/hapee-2.7/hapee-lb.cfg

TLS

• The bind directive’s ca-ignore-err and crt-ignore-err arguments, which let you list TLS errors to ignore, now accept human-readable names in addition to numeric IDs. Refer to the OpenSSL list of error codes to see which values are available.
• The new Runtime API command, add ssl ca-file, adds an SSL/TLS certificate to the in-memory CA file list.

Lua

• You can now pass optional arguments to your custom Lua scripts from within your HAProxy Enterprise configuration file. The lua-load and lua-load-per-thread directives accept one or more arguments after the first argument, which is always the script’s filepath. Then, in your script use the built-in table.pack(...) function with an ellipsis as its argument to collect arguments into a variable of type table.

Multithreading

• A new global directive named thread-group lets you place a range of threads into a group. Once in a group, you can assign those threads to a listening address by adding the thread argument to a bind line in a frontend or listen section. On servers with many CPU cores and thus many threads, allocating a subset of threads to handle connections like this can improve performance by reducing the number of threads competing for work. You can define up to 64 thread groups, each including up to 64 threads.
• A new global directive named thread-groups lets you set the number of thread groups to create and HAProxy Enterprise will divide the threads available on your server into that number of groups. The maximum number is 64.

Converters

This version adds the following converters:

Performance upgrades

• Server health checks, which run at an interval, now fire on the same thread exclusively rather than allowing any available thread to perform the task. This has reduced latency by decreasing competition among threads.
• Stick tables became more efficient by changing the type of lock used to restrict multiple threads from accessing a table simultaneously. By revising internal code to use a read-write lock, which allows multiple threads to read from the table simultaneously, but allows only one thread to write, performance improved.
• HAProxy Enterprise 2.7 reduces latency by more aggressively using idle connections when sending HTTP requests to backend servers. If you leave the defaults, or if you manually set the http-reuse directive in a backend to safe, reusing idle connections is enabled and you are choosing the safest mode. That is, a client’s first HTTP request will be dispatched to a backend server on a new thread guaranteed to not be closed, and only subsequent requests will use idle connections, which have the risk of closing unexpectedly. In this release, as long as you have also set the retry-on directive in that backend to retry the connection in case of failure, HAProxy Enterprise will use an idle connection for a client’s first request too. Set retry-on to include conn-failure, empty-response, and response-timeout.

Deprecated keywords

• The bind-process directive has been removed.
• The process argument on a bind line has been removed.

Version 2.6r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.6r1 release include:

Advanced WAF

• New match zones, $COOKIES_VAR and $COOKIES_VAR_X allow rules to match violations found in specific HTTP cookies, complementing the less specific $HEADERS_VAR and $HEADERS_VAR_X match zones.
• A new variable, txn.<filter>.wlcnt returns the number of disabled rules that would have matched the current request.
• The filter waf line now supports a parameter named log-wl that includes disabled WAF rule violations in extended logs.
• The filter waf line now supports a parameter named log-ext-nonzero that enables extended logs only when the violated rule had a non-zero score or triggered an immediate action, cutting down on noise in the logs.
• The filter waf line now supports a parameter named table-categorized that prefixes entries in the violated rules stick table with the category of the violated rule.
• The filter waf line’s body-limit parameter defaults to the global option waf-body-limit, which now defaults to tune.bufsize rather than the compile-time value of BUFSIZE.

ModSecurity WAF

• To associate WAF logs with load balancer logs, you could already use the unique identifier that ModSecurity creates by referencing the txn.<filter>.unique_id variable, appending it to your load balancer logs. Now, you can define a different unique ID format by setting the use-unique-id-format parameter on the filter modsecurity line and then defining a new format with the unique-id-format directive.
• A new parameter use-vars on the filter modsecurity line disables ModSecurity from denying suspicious requests and delegates that to the load balancer. A variable named txn.<filter>.block returns true when the WAF would have denied the request. You can read this variable and then decide on a response policy to enforce. Other variables that support this include txn.<filter>.error, txn.<filter>.status, and txn.<filter>.url.

Traffic shaping

• New directives, filter bwlim-in and filter bwlim-out, support limiting upload and download speeds for clients. Set bandwidth limits that apply per HTTP stream or to all streams associated with a stick table entry, such as to set a limit per client IP address or per backend application.

Load balancing

• A generic load balancing algorithm named hash was introduced and serves as a replacement for the more specific source, uri, url_param, and rdp-cookie algorithms. It accepts a fetch method as a parameter, which indicates the data used to calculate the hash.

TLS and mTLS

• This version of HAProxy Enterprise supports OpenSSL 3.0.
• When you enable client certificate authentication with the verify required parameter on a bind line, you must also specify the ca-file parameter, which indicates the CA certificate used to verify the client’s certificate. Now, ca-file accepts a path to a directory of CA certificates.
• Similarly, in a backend, you can set the ca-file parameter on a server line to verify the backend server’s TLS certificate against a known CA. This ca-file parameter now accepts a path to a directory of certificates or you can set it to @system-ca to load your system’s list of trusted CAs.

Runtime API and Master CLI

• The Runtime API commands add server and del server, which let you add and remove servers dynamically, are no longer experimental.
• The new Runtime API command show ssl providers lists providers loaded into OpenSSL.
• The Master CLI added new commands: prompt, expert-mode [on|off], experimental-mode [on|off], mcli-debug-mode [on|off].

Lua

• The Lua programming language integration gained the CertCache class, which lets you update an SSL certificate in the load balancer’s runtime memory.
• The Lua httpclient class, which lets you make non-blocking HTTP calls, now supports a dst parameter and a timeout parameter. The former sets the destination address and the latter sets a timeout server value. New global directives support this: httpclient.ssl.ca-file, httpclient.ssl.verify, httpclient.ssl.resolvers.id, and httpclient.resolvers.prefer <ipv4|ipv6>.

Fetches and converts

• New fetches have been added: last_rule_file, which returns the name of the configuration file (e.g. hapee-lb.cfg) that contains the last line processed during stream analysis, and last_rule_line, which returns the line number. Use this to locate the http-request deny line that stopped a request, for example.
• A new converter has been added: add_item, which concatenates strings with a delimiter between them, such as a semicolon.

Other keyword changes

• A new global directive named h1-accept-payload-with-any-method lets HTTP/1.0 clients send a request body with GET, HEAD, and DELETE requests. Use with caution because it can make your application more susceptible to request smuggling attacks.
• Adding the parameter expose-fd listeners on a stats socket line is no longer necessary to achieve hitless reloads.
• The set-var directives, such as http-request set-var, now accept a second parameter to only set the variable if a condition is true. Conditions include: ifexists, ifnotexists, ifempty, ifnotempty, ifset, ifnotset, ifgt, iflt. For example, to set the variable only if it has not already been set: http-request set-var(txn.myvariable,ifnotset) req.hdr(X-MyValue).

Performance upgrades

• The new global directive fd-hard-limit enforces a cap on the number of file descriptors HAProxy Enterprise will use, protecting you from consuming too much memory.
• The new global directive close-spread-time closes idle client connections gradually. For best results, set this lower than hard-stop-after.
• If the load balancer server is a multi-socket machine, HAProxy Enterprise sets an affinity to run on the CPUs of a single node in order to avoid performance penalties caused by the inter-socket bus latency. You can disable this with the no numa-cpu-mapping directive.
• Performance optimizations were made to the task scheduler, connection dequeueing, and connection stream code.

Version 2.5r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.5r1 release include:

Improved Performance

• Memory optimizations on x86 platforms
• Lockless memory pool implementation for non-x86 architectures
• Faster connection dequeueing due to locking reduction
• Revamped and more efficient Domain Name Service (DNS) response processing
• Faster HTTP/1 small chunk parsing

Better thread allocation and control

• Split the total number of configured threads (nbthread) into ranges and assign them to handle incoming connections for unique IP addresses by adding the thread argument to a bind.
• Reduce scheduler contention with the binds and shards options on systems that support multiple listeners per socket.

Observability and access to more information

• More robust reporting on startup
• Statistics for stopped proxies are viewable in the Stats Pane
• The HAProxy Enterprise version and executable path are displayed before the first warning or error to aid with troubleshooting
• Retrieve the original source address of an incoming proxy that connects using the PROXY protocol with fetch fc_src and tcp-request content ruleset: set-src, set-src-port
• Log connection information for handshake errors, using: ssl_bc_err, ssl_fc_err, ssl_bc_err_str, ssl_fc_err_str, fc_err, bc_err
• Retrieve backend connection errors with bc_err, bc_err_str

Configuration

• Store up to 100 indexes in a single Stick Table variable, using General Purpose Counter (GPC) and General Purpose Tag (GPT) arrays.
• Disable bootstrapping WebSockets with HTTP/2 (RFC8441) for newer browsers that don’t support it by using h2-workaround-bogus-websocket-clients.
• Ignore connections without data transfer in the logs with the http-ignore-probes option.
• Gracefully stop proxies, allowing enough time for layer 4 devices to detect changes with the global grace option.
• Add tcp-request and http-request directives to named defaults sections.
• Add more logic to configuration file .if, .elif, and endif conditions with the addition of AND, OR, NOT, and parentheses expressions.
• improved reliability of host header field dependent ACLs by automatically stripping the port (80 or 443) from the URI and the host field.
• Peers ignore updates on the Stick Table conn_cur counter from other peers, ensuring local data is accurate and reflects actual traffic.

Runtime API commands

• Display free memory in thread-local caches with show pools
• Display file, line numbers, rules, and filters processed per session with show sess all
• Full support for creating, managing and removing servers on the fly. The add server command now supports all keywords including: check, track, slowstart, error-limit, ssl and observe.
• View the number of entries in summary output of the show map and show acl commands with the included entry_cnt variable.

Lua scripting

• Initiate HTTP requests using the httpclient class.
• Inspect or modify TCP and HTTP content using an experimental feature set that should be used only in test environments.

Traffic routing

• Redirect rules for empty target URLs are skipped by adding ignore-empty to the http-request redirect directive.

Security

• Check the integrity of claims contained within JSON Web Tokens (JWT) and extract data from JWTs to use in rules.
• Add, update, and delete Certificate Authorities (CA) and Certificate Revocation Lists (CRL) using Runtime API commands.
• Display Online Certificate Status Protocol (OCSP) to validate X.509 digital certificates from the command line (CLI) with show ssl cert and show ssl ocsp-response commands.
• OpenSSL 3.0 is fully supported
• Automatic sanitizing of Transfer Encoding (TE) headers to conform to HTTP/1 specs by making a request or response TE header or a request with Content-Length and TE header the last on a connection.

Removed directives

• grace (per proxy directive replaced by global grace directive)
• http-tunnel
• nbproc
• no option http-use-htx
• option forceclose
• option http_proxy
• set-cookie()
• tune.chksize

Version 2.4r1 Jump to heading #

New and/or improved features in HAProxy Enterprise 2.4r1 include:

Advanced WAF

• This release adds support for PCRE2 regular expressions in rules. It also supports a content-type of application/vnd.api+json.
• The WAF can now parse the body of PATCH requests.
• You can now set body-limit per filter line. It is the maximum size of the buffer used for capturing the request body. It defaults to global.waf-body-limit. You can also set global.waf-body-limit to be lower than global.tune.bufsize.

ModSecurity WAF

• Rule profiling with the filter directive’s new rule-profile parameter helps you to identify rules that run longer than normal.

• A new variable will be set to a number of microseconds spent for the ModSecurity processing, ie. a time interval between transaction start and the ModSecurity transaction end. The variable name is prefixed with an identifier that is specified in the filter declaration:

  haproxy
  txn.<identifier>.total.duration

  haproxy
  txn.<identifier>.total.duration

• If the server has already responded before getting the full request body, ModSecurity will now stop processing the request body.

Global Profiling Engine

• The Global Profiling Engine is a drop-in replacement for the Stick Table Aggregator, with the ability to aggregate stick table data from across a cluster of load balancers in real time. It can also aggregate historical data over a timespan. For example, you can store the average HTTP request rate over an entire day and then calculate statistics from it automatically, such as averages and percentiles, which enables you to alter rate limiting thresholds dynamically based on average traffic load at a given time of day. This simplifies the task of setting rate limit thresholds, making the experience more adaptive and turnkey.

Improved reloading of map files

• Map files now reload faster.

Arm packages

• Packages that target Arm processors on Ubuntu 20.04 were added.

HTTP/2 WebSockets

• Support for the WebSocket protocol over HTTP/2 through the extended CONNECT HTTP method, which is outlined in RFC 8441, allows multiple WebSocket tunnels to share a single TCP connection.

Financial Information eXchange (FIX)

• HAProxy Enterprise can now accept, validate, and route FIX protocol messages. FIX is an open standard that has become the de facto protocol in the Fintech world. HAProxy Enterprise can also inspect tag values within the Logon message sent by the client, which you can use to make authorization and routing decisions using ACLs.

MQTT

• MQTT is a lightweight messaging protocol typically used for communicating with Internet of Things (IoT) devices. you can use the mqtt_is_valid converter to check whether the first message sent by the client (a CONNECT message) or by the server (a CONNACK message) is a valid MQTT packet. You can also read the fields in those initial messages to perform an authorization check and/or initialize session persistence to a server. Also, the mqtt_field_value converter reads a field from a CONNECT or CONNACK message.

DNS TCP resolution

• The resolvers section now allows listing DNS servers over TCP, which supports larger DNS responses. HAProxy Enterprise will accept TCP responses as large as 65,535 bytes. Prefix the nameservers addresses with tcp@.

Circuit breaking

• New stick table counters named http_fail_cnt and http_fail_rate track server-side, HTTP 5xx errors. You can use these in circuit breaking to disable a backend server if it returns too many errors.

Dynamic SSL Certificate Storage (Server side)

• The Runtime API commands show ssl cert, set ssl cert, and commit ssl cert can now be applied to certificates referenced by server lines in a backend. You can now also take advantage of ssl-load-extra-files and ssl-load-extra-del-ext directives in your backend section to allow storing your certificate’s private key in a separate file from the certificate itself.

Connection reuse improvements

• When a connection is handled over TLS, many connections need to be marked private (for example, when sending SNI) to prevent reusing the wrong information (e.g. wrong SNI). That prevents a connection from being reused in many cases. With this release, connections to backend servers can now be reused even when the SNI is calculated dynamically, such as from the request’s Host header (e.g. sni req.hdr(host)).

SSL/TLS statistics

• When you add stats show-modules to your stats frontend, it adds a new column called Extra modules to the page that shows HTTP/2 related statistics. HAProxy Enterprise 2.4 adds more fields there for tracking SSL/TLS handshake and session statistics. You will find the following new fields, which are also displayed when you call the Runtime API’s show stat command, given you have at least one bind statement that terminates SSL:

  Name	Description
  ssl_sess	Total number of SSL sessions established.
  ssl_reused_sess	Total number of SSL sessions reused.
  ssl_failed_handshake	Total number of failed handshake.

Built-in OpenTracing

• OpenTracing is now compiled directly into the core codebase.

Prometheus metrics

• State values for frontends, backends, and servers (up, down, maint, etc.) have been changed from being gauge values to being labels, making it easier to group by this criteria.

• New metrics have been added to report on listeners, stick tables, and backend/server weight information, which we list below:

  • haproxy_process_uptime_seconds
  • haproxy_process_recv_logs_total
  • haproxy_process_build_info
  • haproxy_listener_current_sessions
  • haproxy_listener_max_sessions
  • haproxy_listener_limit_sessions
  • haproxy_listener_sessions_total
  • haproxy_listener_bytes_in_total
  • haproxy_listener_bytes_out_total
  • haproxy_listener_requests_denied_total
  • haproxy_listener_responses_denied_total
  • haproxy_listener_request_errors_total
  • haproxy_listener_status
  • haproxy_listener_denied_connections_total
  • haproxy_listener_denied_sessions_total
  • haproxy_listener_failed_header_rewriting_total
  • haproxy_listener_internal_errors_total
  • haproxy_backend_uweight
  • haproxy_server_uweight
  • haproxy_sticktable_size
  • haproxy_sticktable_used

Vary header

• This release brings support for the Vary header via a new keyword process-vary, which is set to on or off. It defaults to being off, which means that a response containing a Vary header will simply not be cached. It will also automatically normalize the accept-encoding header to improve the use of the cache.
• Also, a new directive max-secondary-entries allows you to control the maximum number of cached entries with the same primary key. For example, if you cache based on URL, but vary on the user-agent, you may create a lot of slightly different cached entries for the same URL. This prevents the cache from being filled with duplicates of the same resource. It requires process-vary to be on and it defaults to 10.

Conditionals

• New, nestable, preprocessor-like directives allow you to include or skip some blocks of configuration markup. The following directives have been introduced to form conditional blocks:

  • .if … .endif
  • .elif
  • .else
  • .diag
  • .notice
  • .warning
  • .alert

  The .if and .elif statements are followed by a function name, such as:

  haproxy
  .if version_atleast(2.4)
     # include configuration lines
  .endif

  haproxy
  .if version_atleast(2.4)
     # include configuration lines
  .endif

  Available functions are:

  Function	Description
  defined()	Returns true if an environment variable exists, regardless of its contents.
  feature()	Returns true if feature is listed as present in the features list reported by haproxy -vv.
  streq(,)	Returns true if the two strings are equal.
  strneq(,)	Returns true if the two strings differ.
  version_atleast()	Returns true if the current HAProxy version is at least as recent as otherwise false.
  version_before()	Returns true if the current HAProxy version is strictly older than otherwise false.

  Also, the following pre-processor directives can be placed inside a conditional block:

  Directive	Description
  .diag “message”	Emit this message only when in diagnostic mode (-dD).
  .notice “message”	Emit this message at level NOTICE.
  .warning “message”	Emit this message at level WARNING.
  .alert “message”	Emit this message at level ALERT.

Default path

• A new global directive default-path allows you to specify the path from which you would like HAProxy Enterprise to load additional files, such as map and ACL files. It accepts one of the following options:

Pseudo variables

• Pseudo-variables have been added, which can be helpful when troubleshooting.

Named defaults sections

• You can now assign a name to a defaults section and inherit its settings specifically in a frontend or backend.

  haproxy
  defaults http-defaults
    # default settings ...
  frontend fe_http from http-defaults
    # inherits the settings

  haproxy
  defaults http-defaults
    # default settings ...
  frontend fe_http from http-defaults
    # inherits the settings

• You can also extend a named defaults section from another defaults section:

  haproxy
  defaults tcp-defaults
    mode tcp
    timeout connect 5s
    timeout client 5s
    timeout server 5s
  default http-defaults from tcp-defaults
    mode http

  haproxy
  defaults tcp-defaults
    mode tcp
    timeout connect 5s
    timeout client 5s
    timeout server 5s
  default http-defaults from tcp-defaults
    mode http

Dynamic server timeouts

• You can change the timeout server and timeout tunnel settings dynamically using the http-request set-timeout directive. You can use this to set custom timeouts on a per-host or per-URI basis, pull a timeout value from an HTTP header, or change timeouts using a Map file.

• The following fetches are available:

  Fetch	Description
  be_server_timeout	Returns the configuration value in millisecond for the server timeout of the current backend.
  be_tunnel_timeout	Returns the configuration value in millisecond for the tunnel timeout of the current backend.
  cur_server_timeout	Returns the currently applied server timeout in millisecond for the stream.
  cur_tunnel_timeout	Returns the currently applied tunnel timeout in millisecond for the stream.

HTTP protocol upgrade

• The new tcp-request content switch-mode directive upgrades a mode tcp connection to mode http dynamically. Combined with the built-in HTTP ACL, you can switch to mode http if the traffic is HTTP.

  haproxy
  frontend fe_main
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content switch-mode http if HTTP

  haproxy
  frontend fe_main
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content switch-mode http if HTTP

Header deletion with pattern matching

• The http-request del-header, http-response del-header, and http-after-response del-header directives now support the argument -m <method> to delete a header based on a matched pattern.

HTTP request conditional body wait time

• The new http-request wait-for-body and http-response wait-for-body directives allow you to conditionally wait for and buffer a request or response body. These actions may be used as a replacement for option http-buffer-request. In the snippet below, HAProxy Enterprise waits one second at most to receive the POST body of the request, or until it gets at least 1,000 bytes:

  haproxy
  http-request wait-for-body time 1s at-least 1k if METH_POST

  haproxy
  http-request wait-for-body time 1s at-least 1k if METH_POST

Fetches

• This table lists new fetches:

  Name	Description
  baseq	Returns the concatenation of the first Host header and the path part of the request with the query-string, which starts at the first slash.
  bc_dst	This is the destination ip address of the connection on the server side, which is the server address HAProxy Enterprise connected to.
  bc_dst_port	Returns an integer value corresponding to the destination TCP port of the connection on the server side, which is the port HAProxy Enterprise connected to.
  bc_src	This is the source ip address of the connection on the server side, which is the server address HAProxy Enterprise connected from.
  bc_src_port	Returns an integer value corresponding to the TCP source port of the connection on the server side, which is the port HAProxy Enterprise connected from.
  be_server_timeout	Returns the configuration value in millisecond for the server timeout of the current backend.
  be_tunnel_timeout	Returns the configuration value in millisecond for the tunnel timeout of the current backend.
  cur_server_timeout	Returns the currently applied server timeout in millisecond for the stream.
  cur_tunnel_timeout	Returns the currently applied tunnel timeout in millisecond for the stream.
  fe_client_timeout	Returns the configuration value in millisecond for the client timeout of the current frontend.
  sc_http_fail_cnt(<ctr>[,<table>]	Returns the cumulative number of HTTP response failures from the currently tracked counters. This includes the both response errors and 5xx status codes other than 501 and 505.
  sc_http_fail_rate(<ctr>[,<table>])	Returns the average rate of HTTP response failures from the currently tracked counters, measured in amount of failures over the period configured in the table. This includes the both response errors and 5xx status codes other than 501 and 505. See also src_http_fail_rate.
  ssl_c_der	Returns the DER formatted certificate presented by the client when the incoming connection was made over an SSL/TLS transport layer.

Converters

• The following converters have been added:

  Name	Description
  json_query(<json_path>,[<output_type>])	The json_query converter supports the JSON types string, boolean and number.
  xxh3	Hashes a binary input sample into a signed 64-bit quantity using the XXH3 64-bit variant of the XXhash hash function.
  ub64dec	This converter is the base64url variant of b64dec converter. base64url encoding is the “URL and Filename Safe Alphabet” variant of base64 encoding. It is also the encoding used in JWT (JSON Web Token) standard.
  ub64enc	This converter is the base64url variant of base64 converter.
  url_enc	Takes a string provided as input and returns the encoded version as output.

Runtime API

This version includes the following changes to the Runtime API:

• Help filtering- The help command will attempt to display information for partially spelled commands. Also if you enter an incorrect command, the API will suggest alternatives.

• This release adds new Runtime API commands:

  Name	Description
  set server <backend>/<server> agent-port <port>	Change the port used for agent checks.
  set server <backend>/<server> check-addr <ip4 | ip6> [port <port>]	Change the IP address used for server health checks. Optionally, change the port used for server health checks.
  set server <backend/server> ssl on	Activates SSL on outgoing connections to a server at runtime.
  prepare acl	Begin a transaction for adding ACLs. This returns a number, which you would then reference in the add acl command.
  commit acl	Commit the transaction to add new ACLs.

Lua

• Lua has been extended to support multithreading. A new global directive, lua-load-per-thread, has been added to aid with this. For Lua scripts that require threading, this should be used as it will launch the Lua code as an independent state in each thread. The existing lua-load keyword still exists and should be used for Lua modules meant to cover the entire process, such as jobs registered with core.register_task.

Miscellaneous

There are the following miscellaneous changes:

• The tune.chksize directive has been deprecated.
• During a reload, processes are closed sooner, rather than waiting for idle frontend connections to timeout.
• During a reload, idle backend connections are actively killed.
• TCP log outputs now automatically create a ring buffer.
• Layer 7 retries now support 401 and 403 HTTP status codes.
• The http-check send directive now supports adding a Connection header.
• The global section now supports setting process level variables.
• A new macro HTTP_2.0 has been added, which will return true for HTTP/2 requests.
• The server-state-file-name directive has a new argument, use-backend-name, that allows you to load the state file using the name of the backend. This only applies when the directive load-server-state-from-file is set to local.
• A new srvkey parameter has been added to the stick-table directive. This allows you to specify how a server is identified. It accepts either name, which will use the current defined name of the server or addr, which will use the current network address including the port. addr is useful when you are using service discovery to generate the addresses.
• A new log-format parameter, %HPO, was added, which allows logging of the request path without the query string.

Version 2.3r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.3r1 release include:

Connection improvements for upcoming QUIC and HTTP/3 support

• The connection layer was optimized to reduce the number of syscalls.

• Several debugging entries were added to help better spot anomalies.

• Listeners have been reworked and related structures have been reorganized to better suit the new design. File descriptors are no longer manipulated by the listener layer.

• TCP keepalive lets the Linux kernel know when a peer on the other end of a connection has stopped responding and that it’s safe to close the idle connection. It discovers this by sending probes. If the peer doesn’t reply, the socket is closed automatically. You can change the number of probes to send, the interval at which to send them, or how long to wait before starting to send probes on both the client and server side through the following directives:

  Directive	Description
  clitcpka-cnt	Sets the maximum number of keepalive probes TCP should send before dropping the connection on the client side.
  clitcpka-idle	Sets the time the connection needs to remain idle before TCP starts sending keepalive probes on the client side, if enabled.
  clitcpka-intvl	Sets the time between individual keepalive probes on the client side.
  srvtcpka-cnt	Sets the maximum number of keepalive probes TCP should send before dropping the connection on the server side.
  srvtcpka-idle	Sets the time the connection needs to remain idle before TCP starts sending keepalive probes on the server side, if enabled.
  srvtcpka-intvl	Sets the time between individual keepalive probes on the server side.

Syslog protocol

• A new section called log-forward can bind on TCP using the bind keyword and on UDP using dgram-bind for both IPv4 and IPv6. You can thus create a syslog listener over UDP or TCP that can forward, prioritize, and translate syslog messages to a pool of UDP or TCP syslog servers. When combined with the log sampling feature added in HAProxy Enterprise 2.0, you get granular control over how your syslog messages are forwarded. You can also translate syslog messages from one format to another.

• The Runtime API show info command also exposes a new counter called CumRecvLogs, which provides a global count of received syslog messages.

Load Balancing

• The new balance uri directive’s path-only option indicates that the hash should be calculated using only the path normalizing HTTP/1 and HTTP/2 messages. This avoids inconsistencies between requests received over HTTP/1 and the same ones received over HTTP/2.

• If the balance random algorithm returns a server whose maxconn value has been reached, meaning that connections are now queuing up for that server, it will add the request to the backend’s queue and not the server’s queue. The request can be redispatched to another available server, and typically the fastest.

• For some load-balancing algorithms (roundrobin, static-rr, leastconn, first), requests were queued in the backend due to a previous attempt at finding a suitable server after trying all of them. Now, the next request skips the part where each server is tried and goes directly to the backend’s queue.

• The leastconn algorithm has been improved to take the queue length into account when dispatching requests. This means that if a server has a lot of queued requests we won’t hammer it with extra connections.

Cache

• The Expires HTTP header instructs HAProxy Enterprise how long it should cache the response.
• The cache now supports the ETag, If-None-Match, and If-Modified-Since.
• The cache can return an HTTP status code 304 instead of the full object.
• HAProxy Enterprise will now also reject any configuration that has a duplicate cache section name.
• The new fetch methods added-res.cache_hit and res.cache_name tell you whether a response came from the cache and, if so, the name of the cache used.

SSL/TLS Enhancements

• If you load SSL/TLS certificates separately from the certificate key through the ssl-load-extra-files global directive, the key no longer needs to be named the exact same as the certificate with .key appended to it.

• The new global directive ssl-load-extra-del-ext instructs HAProxy Enterprise to remove the certificate file’s extension before adding a new one. For example, the key can be named mycert.key instead of mycert.crt.key.

• The generate-certificates directive adds a Subject Alternative Name (SAN) to all generated certificates, which is a requirement in modern browsers. It now also supports chaining CAs and attaching a trust chain in addition to the generated certificate. The chain is loaded from the one provided in the ca-sign-file PEM file.

• If the SNI is hardcoded on the server line using, as an example, sni str(example.local) there’s no risk in reusing the connection. This release allows reusing connections that hardcode the SNI to the backend server. It will mark connections as private only if you’ve configured a variable expression for the SNI.

• If a crt-list does not end with a new line, a warning indicates that the file might have been truncated.

Observability

• The new show stat Runtime API command’s option domain, allows you to change the context of the statistics. The proxy default value displays the core proxy statistics that were available before. The dns value displays statistics related to DNS resolution that HAProxy Enterprise performs.

• The stats show-modules directive enables extra statistics related to HTTP/2 on the HAProxy Enterprise Stats.

• The Stats page displays a new field under the Wght column, which previously only showed the live or effective weight. The Wght column now contains the effective weight separated with a “/” followed by the configured weight.

• The show stat Runtime API command now allows you to use show stat up to filter on servers that are up or show stat no-maint to show those that are not in maintenance mode.

• The Prometheus exporter received some new process and per-server metrics, as outlined here:

  Metric	Description
  haproxy_process_failed_resolutions	Total number of failed DNS resolutions.
  haproxy_process_bytes_out_total	Total number of bytes emitted.
  haproxy_process_spliced_bytes_out_total	Total number of bytes emitted through a kernel pipe.
  haproxy_process_bytes_out_rate	Number of bytes emitted over the last elapsed second.
  haproxy_server_unsafe_idle_connections_current	Current number of unsafe idle connections.
  haproxy_server_safe_idle_connections_current	Current number of safe idle connections.
  haproxy_server_used_connections_current	Current number of connections in use.
  haproxy_server_need_connections_current	Estimated needed number of connections.

• The Runtime API’s show stat output has been extended and adds a new delimiter, a dash (“-”), after which additional dynamic fields can be added. Those fields won’t be shown unless the relevant component is in use.

OpenTracing (SPOE)

• An OpenTracing SPOA allows HAProxy Enterprise to send data directly to distributed tracing systems via the OpenTracing API.

HTTP Request Actions

• The new HTTP request action http-request replace-pathq, does the same as http-request replace-path, except that the replacement value may contain a modified query string.

• The new HTTP request action http-request set-pathq works similarly to http-request set-path, except that the query string is also rewritten. Unlike http-request replace-pathq, it does not take a regular expression and replacement value, but a formatted string to use as the new path. It can also be used to remove the query string, including the question mark.

New sample fetches

New converters

• iif returns the <true> string if the input value is true. Returns the <false> string otherwise. |

Lua

• Support for Lua 5.4, which was initially released in June 2020.
• This release exports the sample fetches http_auth() and http_auth_group().
• You can now use regular expressions in fetches and converter arguments.
• Sample fetches and converters that require arguments are now supported as well.

Deprecated and Removed Directives

• The obsolete keyword monitor-net was removed. It supported only a single IPv4 network, was incompatible with SSL, and required HTTP/1.x. It is now recommended to use http-request return status 200 if { src 10.1.1.3 } instead.

• The obsolete keyword mode health was removed. It was incompatible with SSL and worked with only HTTP/1. It is now recommended to use http-request return status 200 instead.

• The global keyword debug has been removed. It had, on occasion, trapped users by disrupting their system’s ability to boot. You can continue to use -d on the command line.

• The nbproc directive is now deprecated and is set for removal in 2.5. It used too much memory, led to high network overhead (poor reuse, multiple health checks), lacked peers syncing and stats, caused problems with seamless reloads, and would not support QUIC at all. If nbproc is found with more than one process while nbthread is not set, a warning will be emitted encouraging you to remove it or migrate to nbthread.

• The grace directive has been marked as deprecated and is scheduled tentatively for removal in 2.4 with a hard deadline of 2.5. It was meant to postpone stopping of a process during a soft-stop, but is incompatible with soft reloading.

Miscellaneous

• The strict-limits directive defaults to on. You’ll now get a startup error if you configure too large a maxconn for your system’s limits.
• The process no longer reports proxy has started.
• An optimization for PCRE2 was made, which uses the JIT match when a JIT optimization has occurred. This should shorten the code path to call the match function.
• Several deinit() fixes were made to improve the results from Valgrind.
• Support for upgradable locks was added. These cut the scheduler overhead in half and reduce the locking time during map and ACL updates.

Version 2.2r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.2r1 release include:

Enterprise addons & native modules

• A new native module for performing TLS/SSL fingerprinting.
• Support for Single Sign On SAML.
• New validate-crawler daemon to verify the legitimacy of clients claiming to be Googlebot & Bingbot.

Dynamic SSL certificate storage

• SSL certificates can now be added/removed/updated through the Runtime API using the set ssl cert and commit ssl cert commands.

SSL/TLS enhancements

• TLSv1.2 the new default minimum version TLS version
• tune.ssl.default-dh-param now defaults to 2048 bits
• New global directive: ssl-default-bind-curves to specify the list of elliptic curve algorithms that are negotiated during the SSL/TLS handshake when using Elliptic-curve Diffie-Hellman Ephemeral (ECDHE).
• New global directives: ssl-load-extra-files and issuers-chain-path to load the TLS/SSL private key and intermediate certificates separately.
• New global directive: ssl-skip-self-issued-ca to avoid sending global CA certificates to clients when using OCSP stapling.
• New bind parameter ca-verify-file to store the root CA in a separate file to use for validating an intermediate CA.
• Support for fetching and logging the secrets necessary for decrypting TLS 1.3

Native response generator

• New actions http-request return and http-response return make it possible to return a custom response from HAProxy Enterprise, with any status code, based on an error file, a file or a string.
• In addition, using a log-format string or a log-format file, it is possible to have responses with a dynamic content.
• Definition of extra headers by passing the hdr argument.

Dynamic error handling

• A new section http-errors to create global groups of custom HTTP error files.
• Expanded directives error-files, http-request deny, and http-response deny to support loading http-errors groups.
• Dynamic error processing to define error file templates with log-format parameters.

Health check overhaul

• New directives {http|tcp}-check connect to tune health check connection parameters by enabling SNI, connecting over SSL/TLS, performing health checks over SOCKS4, and choosing the protocol, such as HTTP/2 or FastCGI.
• New directive http-check send to customize fully an HTTP health check, including sending a custom method, URI, headers, and POST body data.
• New {http|tcp-check} comment directives to define a comment to report in the logs if the http-check rule fails.
• Ability to query multiple endpoints at once to allow for multistage health checking.
• New directives {http|tcp}-check set-var and {http|tcp}-check unset-var to set or unset a variable during HTTP and TCP health checks.
• MySQL based health checks using the option mysql-check directive were rebuilt on top of the new tcp-check rules. They now default to a MySQL 4.1 and above client-compatible check when there is a defined username.

Syslog over TCP

• New support for sending to syslog over TCP.
• A new section ring to define custom ring buffers to use for queuing up messages and ensuring message delivery.

Performance improvements

• Automatic deduplication of ca-file and crl-file directives to improve the overall startup speed.
• New thread-local pool of recently used pipes to improve cache locality and eliminate unnecessary allocation.
• ACL unique id value improvements.
• Significant reduction of the number of syscalls per request for a connection using keep-alive mode.
• Memory pools now release when there are an abundance of objects created after a traffic surge.
• The connection layer underwent several performance improvements, essentially resulting in fewer syscalls on average, primarily for epoll.
• HAProxy Enterprise no longer closes a client connection after it serves an internal response code (such as a 401 or 503), unless requested. Note: Status codes 400 (Bad Request) and 408 (Request Timeout) are excluded from this.
• A new directive pool-low-conn to optimize server connection pooling. It is also possible to tune it to indicate the number of required idling connections to a server before a thread can reuse a connection.
• The scheduler is now also latency aware, which means that the Runtime API is usable regardless of HAProxy Enterprise’s load.

Observability & debugging

• A new Runtime API command show servers conn to see the current and idle connection state of the servers within a backend.
• HAProxy Enterprise Enterprise Stats page now reports connect, queue and response time metrics with more accuracy.
• A new timing metric, %Tu to return the total estimated time as seen from the client, from the moment the proxy accepts the request to the moment both ends get closed, not including the idle time before the request began.
• Improved HAProxy Enterprise internal watchdog and expanded support to FreeBSD.
• The debug converter is now always available and sends the output to a defined event sink. The currently available event sinks are buf0, stdout and stderr.

HTTP actions

• A new ruleset http-after-response that HAProxy Enterprise evaluates on all responses prior to forwarding.
• A new http-{request|response|after-response} action called strict-mode to enable or disable a strict rewriting mode on all the rules that follow it.
• A new http-request action called replace-path is similar to replace-uri, except that it only acts on the path component.

Security hardening

• HAProxy Enterprise now prevents the creation of new processes at default, effectively disabling the use of external programs for checks completely.
• A new global directive insecure-fork-wanted to disable the above security capability and allow the use of the external-check command.
• HAProxy Enterprise now prevents the process from executing setuid binaries by default to prevent it from switching uids after the initial switch to the uid defined within the global section.
• To re-enable the execution of setuid binaries, you can use the new global directive insecure-setuid-wanted.

Fetches

Converters

Lua

• You can now prepend the lookup path for Lua modules using lua-prepend-path.
• It is now possible to intercept HTTP messages from a Lua action and reply to clients.
• Lua declared actions can now yield using wake_time (). This function can define a timeout when a Lua action returns act:YIELD. It is a way to force the script to re-execute after a short time (defined in milliseconds).
• set_var and unset_var now returns a Boolean to indicate success.
• A new parameter ifexist is added to set_var to allow a Lua developer to set variables to ignore unless the variable name was used elsewhere before.
• The Server class now has a set_addr function to change a backend server’s address and port.
• A new function is_resp to determine whether a channel is a response channel.

Miscellaneous improvements

• The parser now supports quotes, braces, and square brackets in arguments.
• The parser now also shows you the location of where a parsing error occurred.
• The use-server directive now supports rules using dynamic names.
• The bits argument for the sha2 converter will now properly be verified.
• Added the ‘sub-second’ and ‘timezone’ fields to the RFC5424 log format.
• The number of connections reported in the output of a quitting proxy now indicates cumulative connections and not active connections.
• The Runtime API’s show table command now supports filtering stick table output by multiple filters, allowing for filtering on many columns.
• DNS Service Discovery now reuses information available within the extension parts of an SRV record response.
• The cookie directive now has an ‘attr’ field to set attributes on persistence cookies. This is helpful when adding the ‘SameSite’ attribute, which is required in Chrome 80 and above.
• You can specify the local peer name with ‘localpeer’ within the peers section. You can override it with the -L parameter on startup.
• The Runtime API now allows for escaping spaces.
• ACLs can no longer be named “or”.
• Error files that are larger than tune.bufsize now emit a warning message on startup.
• The http-request deny directive now supports returning status codes ‘404 Not Found’, ‘410 Gone’, and ‘413 Payload Too Large’.
• Improved UUID random generation is now thread safe.
• You can now send and receive a unique-id in the PROXY Protocol for connection tracking purposes.
• The default maxconn is now automatically set based on the configured ulimit -n.
• Invalid hex sequences now cause a fatal error.
• A new option pp2-never-send-local to revert the old bogus behavior on the server side when using proxy-protocol-v2 in health checks.

Version 2.1r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.1r1 release include:

Strict Rewriting Mode

• By default, HAProxy Enterprise triggers an internal error when a rule that performs a rewrite on an HTTP message fails.

New HTTP Errors section

• Create global groups of custom HTTP error files by using the new section http-errors.
• Support the loading of http-errors groups using the expanded directives error-files, http-request deny, and http-response deny.

New HTTP Return Actions

• Return a custom response from HAProxy Enterprise with any status code based on an error file, a file, or a string using the new actions http-request return and http-response return.
• Enable responses with dynamic content by using a log-format string or a log-format file.
• Define extra headers by passing the hdr argument

HTTP “After Response” Rulesets

• Evaluate a new ruleset http-after-response on all responses prior to forwarding
• Let HAProxy Enterprise evaluate these rules at the end of the response analysis on all HTTP responses, just before it forwards the data. This includes responses from the server as well as responses from HAProxy. This makes it possible to add headers to the responses that the stats applet generates.

Cookie Attributes

• Supply an attr option to insert any attribute when HAProxy Enterprise inserts a cookie.
• Use with the Chrome 80 update that requires the “SameSite” attribute. (Example: cookie SRV insert attr "SameSite=Strict")
• Repeat the attr option to add several attributes.

Dynamic SSL Certificate Updates

• Centralized SSL certificate information that only loads once when multiple bind lines reference the same certificate.
• Ability to update SSL certificates with the Runtime API using the set ssl cert and commit ssl cert commands.

FastCGI

• Direct communication between HAProxy Enterprise and FastCGI
• New protocol fcgi
• Definition of parameters for communicating with a FastCGI application in a new section called fcgi-app
• Backends can relay requests to a defined application using the use-fcgi-app directive

Native Protocol Tracing

• Integration of a new tracing infrastructure to allow systems engineers and developers to collect low-level trace messages
• Tracing ability and access through the Runtime API using the trace and show trace commands

Removal of the File Descriptor Cache

• Complete removal of the file descriptor (FD)
• This change has shown an increase in performance of up to ~20% on some artificially tailored workloads. Realistically, production environments can expect to see a 5-10% improvement.

Scheduler Improvements

• Improved internal scheduler supports waking up tasks that belong to another thread
• The scheduler now uses a combination of a locked and a lock-free list to regain 5-10% performance on workloads involving high connection rates.

Defaulted HTTP Representation to HTX

• Removal of support for legacy HTTP mode
• Support only of the Native HTTP Representation (HTX)
• No configuration change is necessary, unless you try to specify no option http-use-htx, in which case you get an error.
• Assistance for a seamless transition from legacy applications with new global directives h1-case-adjust and h1-case-adjust-file.
• Option case-adjust-bogus-client and option h1-case-adjust-bogus-server directives to enable explicitly the case adjustment within defined frontend, listen, and backend sections.

Fetches

Converters

Deprecated Configuration Options

Strict Limits Setting

• Allows HAProxy Enterprise to abort at startup if it cannot get the required limits, such as in cases where HAProxy Enterprise is unable to increase necessary limits upon startup.
• A new global directive strict-limits will cause HAProxy Enterprise to fail to start if it cannot increase the limits through setrlimit().

Version Info Show Links

• Passing -v now displays End of Life (EOL) information for this release.

Runtime API Field Descriptions

• The show info and show stat Runtime API commands now accept a new parameter called desc that adds a short description to each field.

Prometheus Improvements

• You can now pass a new scope query string parameter to filter exported metrics. The following values are supported: global, frontend, backend, server, * (all).

Miscellaneous Improvements

• Moving the storage of the server-state global file to a tree, which provides much faster reloads.
• Acceptance as an expression: http-request / http-response sc-set-gpt0.
• The directive resolve-opts now accepts ignore-weight. Hence, when HAProxy Enterprise generates servers with DNS SRV records, it can set server weights dynamically using agent health checks or the Runtime API, and not have DNS SRV reset the weights subsequently.
• Ability to export the Stats page in JSON format by appending “/;json” to the URI.
• Ability to send the PP2_TYPE_AUTHORITY value to allow it to chain layers using SNI, using the new directive send-proxy-v2.
• Additional support for the user and group directives in the program’s Process Manager section.
• Connections require significantly less memory as HAProxy Enterprise allocates dynamically the source and destination addresses as needed. This translates to 128 to 256 bytes saved per connection and per side in the common case.

Version 2.0r1 Jump to heading #

Key changes in the HAProxy Enterprise 2.0r1 release include:

Enterprise modules

• Javascript Challenge module now supports templates
• DeviceAtlas module
• ScientiaMobile WURFL module
• Extensive ModSecurity hardening improvements

Cloud-native threading

• HAProxy Enterprise sets the number of worker threads to match the machine’s number of available CPU cores to scale and accommodate any environment with less manual configuration.

Cloud-native logging

• Easier to adapt for containerized environments, allowing you to log directly to stdout and stderr or to a file descriptor.

HTTP representation (HTX)

• Introduced with HAProxy Enterprise 1.9, HTX makes any future HTTP protocols easier to integrate. It is enabled by default.

End-to-End HTTP/2

• Official support for end-to-end HTTP/2 through new parameters alpn h2 and proto h2.

gRPC

• Full support for the open-source gRPC framework. This allows for bidirectional streaming of data, detection of gRPC messages, and logging gRPC traffic. Activated using a standard end-to-end HTTP/2 configuration.

Layer 7 retries

• Can retry a connection at Layer 7 for failed HTTP requests using the new configuration directive retry-on.

Data Plane API

• Provides a modern REST API to configure HAProxy Enterprise on the fly by dynamically adding and removing frontends, backends, and servers. You can create ACL rules, insert HTTP routing directives, set IP and port bindings, etc.

Process Manager

• Allows you to specify external binaries for HAProxy Enterprise to start and manage as child processes.

Traffic mirroring

• Allows you to mirror requests from one environment to another. The new Traffic Shadowing daemon is developed as a Stream Processing Offload Agent (SPOA) and takes advantage of HAProxy Enterprise’s SPOE which allows you to extend HAProxy Enterprise using any programming language.

Kubernetes Ingress Controller

• Provides a high-performance ingress for your Kubernetes-hosted applications. It supports TLS offloading, Layer 7 routing, rate limiting, IP allow lists,

Prometheus Exporter

• HAProxy Enterprise can export metrics to Prometheus for monitoring and alerting purposes.

Peers & Stick Tables

HAProxy Enterprise 2.0 introduces several improvements to the Peers Protocol including:

• Heartbeat
• Stick tables in peers sections
• SSL support
• Runtime API command: show peers
• New stick table counters
• New stick table data type, server_name
• Peers section expanded to allow using the bind, default-bind, server, and default-server configuration directives. It also now supports having stick tables directly within itself.

Power of Two Random Choices Algorithm

• Added a new load-balancing algorithm called random that chooses a random number as the key for the consistent hashing function. This is useful with large farms or when servers are frequently added or removed.

Log Distribution and Sampling

• It is now possible to do sampling directly within HAProxy Enterprise by using the log directive’s sample parameter. You can specify multiple log and sample directives simultaneously.

Built-in Automatic Profiling

• This version now features the profiling.tasks directive to specify in the global section. CPU profiling per task shows where the time is spent and which requests have what effect on which other request.

Enhanced TCP Fast Open (TFO)

• Added TFO for connections to backend servers on systems that support it. This requires Linux kernel 4.11 or newer.

New request actions

Introduced several new http-request and tcp-request actions below:

• http-request do-resolve: Performs DNS resolution of the output and stores the result in the variable.
• http-request disable-l7-retry: Disables any attempt to retry the request if it fails for any reason other than a connection failure. This ensures that POST requests aren’t retried upon failure.
• http-request replace-uri: Matches the regular expression in the URI part of the request and replaces it.
• tcp-request content do-resolve: Performs DNS resolution of the output and stores the result in the variable.
• tcp-request content set-dst: Sets the destination IP address.
• tcp-request content set-dst-port: Sets the destination port.

New converters

Converters allow you to transform data captured by fetch methods. Below are new converters in this version:

• aes_gcm_dev: Decrypts the raw byte input using the AES128-GCM, AES192-GCM or AES256-GCM algorithm.
• protobuf: Extracts the raw field of an input binary sample representation of a Protocol Buffers message.
• ungrpc: Extracts the raw field of an input binary sample representation of a gRPC message.
• sha2: Converts a binary input sample to a digest in the SHA-2 family
• srv_queue: Takes an input value of type string and returns the number of queued sessions on that server

New fetches

Fetches provide a source of information from either an internal state or from layers 4, 5, 6, and 7. New fetches in this release return a random of the front or back connection when the incoming connection was made over an SSL/TLS transport layer. This release’s new fetches (below) return a random of the front or back connection when the incoming connection was made over an SSL/TLS transport layer:

• ssl_fc_client_random: Returns the client random of the front connection
• ssl_fc_server_random: Returns the server random of the front connection
• sl_bc_client_random: Returns the client random of the back connection
• ssl_bc_server_random: Returns the server random of the back connection; this requires OpenSSL >= 1.1.0, or BoringSSL.
• uuid: Returns a UUID following the RFC4122 standard
• http_auth_type: Returns the authentication method from the Authorization header
• http_auth_user: Returns the authorized user from the Authorization header
• http_auth_pass: Returns the authentication password from the Authorization header

Miscellaneous improvements

• TLS session tickets help to speed up session resumption for clients that support them. HAProxy Enterprise 2.0 adds support for AES256-bit ticket keys specified in both a file or through the Runtime API.
• A new global directive set-dumpable makes it easier to retrieve a core file.
• Introduces 2 new server keywords, socks4 and check-via-socks4 used for communicating with servers within a backend over SOCKS4 and adds similar functionality for health checking over SOCKS4.
• Stats page export in JSON
• Additional support for logging to a ring buffer
• SSL memory enhancements
• FastCGI support

Version 1.9r1 Jump to heading #

Key changes in the HAProxy Enterprise 1.9r1 release include:

Optimizations

• Connection scheduling reworked to feature true ASYNC-I/O at every level

• Varnish Test suite Integration

• HTTP Status code 103 (Early Hints)

• Support for HTTP Status code 421 (Misdirected Request)

• CLI / Runtime API supports payloads

• Master/Worker CLI (master has its own socket, communicate with all workers, even those exiting)

• Server queues now have a “set-priority” option: deliver JS/CSS before images, boost premium level users compared to regular ones, or give lower priority to bots

• Stick-tables extended with gpc1/gpc1_rate counters

• Buffer modifications

• Improved task scheduler: scales much better with large thread counts

• Native HTTP Representation (HTX) which immediately brings:

  • End-to-end HTTP/2
  • Server-side connection pooling
  • Server-side connection multiplexing
  • gRPC

Cache improvements

• Age headers
• Chunked Transfer Encoding support
• max-object-size = 2GB, total-object-size = 4GB

New converters

• strcmp
• concat
• length
• crc32c
• ipv6 added to “ipmask” converter
• field/word converter extended

New fetches

• date_us: Microseconds part of the date
• cpu_calls: Number of calls to the task processing the stream or current request since it was allocated. It is reset for each new request on the same connection.
• cpu_ns_avg: The average number of nanoseconds spent in each call to the task processing the stream or current request.
• cpu_ns_tot: Total number of nanoseconds spent in each call to the task processing the stream or current request.
• lat_ns_avg: Average number of nanoseconds spent between the moment the task handling the stream is woken up and the moment is is effectively called.
• lat_ns_tot: The total number of nanoseconds between the moment the task handling the stream is woken up and the moment it is effectively called.
• srv_conn_free / be_conn_free : Determine the number of available connections on server/backend.
• ssl_bc_is_resumed : Returns true when the back connection was made over an SSL/TLS transport layer and the newly created SSL session was resumed using a cached session or a TLS ticket.
• fe_defbe: Fetches frontend default backend name.
• ssl_fc_session_key / ssl_bc_session_key: Returns the SSL master key of the front/back connection.
• ssl_bc_alpn / ssl_bc_npn: Provides the ALPN and the NPN for an outgoing connection.
• prio_class: Returns the priority class of the current session for http mode or the connection for TCP mode.
• prio_offset: Returns the priority offset of the current session for http mode or the connection for TCP mode.

Other enhancements

• Random based load-balancing algorithm
• Cloud Native Logging (Log to stdout / fd)
• “Resolvers” section now supports resolv.conf
• “Show activity” - shows the average and total CPU time consumed by the processing of each task, as well as the average and total latency inflicted by the processing of all other tasks. Values can also be logged if profiling is enabled either through the global section using “profiling.tasks on” or through the Runtime API.
• Busy-polling - allows reduction of request processing latency by 30 - 100 microseconds on machines using frequency scaling or supporting deep idle states.
• Lua Server class gained the ability to change a server’s maxconn value
• Lua TXN class gained the ability to adjust a connections priority within the server queue
• Lua has a new StickTable class that allows access to the content of a stick-table by key and allows dumping of an entire stick-table
• SSL: Added support of AES-256 bits ticket keys on file and CLI
• 51Degrees: Enabled multi-threaded operation in the 51Degrees module
• Prometheus-exporter: New Prometheus exporter for HAProxy Enterprise
• Listener: Implemented multi-queue accept for threads
• Listener: Used the multi-queue for multi-thread listeners
• Configuration: Added global tune.listener.multi-queue setting
• Threads: Enabled one thread per CPU by default
• Configuration: Disabled support for nbproc and nbthread in parallel
• Backend: Made the random algorithm support a number of draws
• Multiplex: Made the H2 MAX_FRAME_SIZE setting configurable
• Configuration: No longer enforced a low frontend maxconn value

Version 1.8r2 Jump to heading #

Key changes in the HAProxy Enterprise 1.8r2 release include:

Enterprise modules

• New netacuity module which allows the loading of DigitalElement/NetAcuity geolocation database into HAProxy Enterprise. This module also allows for live updating from a central URL at a defined interval.
• New maxmind module which allows the loading of MaxMind GeoIP databases into HAProxy Enterprise. This module also allows for live updating from a central URL at a defined interval.
• New send-metrics module allows HAProxy Enterprise to send statistics at a defined interval to an external HTTP/HTTPS based API. This is useful for integrating with services such as NS1, which can collect HAProxy Enterprise statistics and make DNS decisions. This also allows for integration with graphing and monitoring solutions.
• New htmldom module which allows HTML or Javascript to be injected into the response data.
• New modsecurity module which allows the loading of modsecurity rulesets into HAProxy Enterprise, such as the OWASP CRS.

Feature enhancements

• HTTP Status code 103 (Early Hints)
• Support for HTTP Status code 421 (Misdirected Request)
• Cloud Native Logging (Log to stdout / fd)
• CLI / Runtime API supports payloads (ocsp, map)
• Master/Worker CLI (the master has its own socket and communicates with all workers, even those exiting)
• Server queues now have a set-priority option: Delivers JS/CSS before images; boosts premium level users compared to regular users, or gives lower priority to bots
• Random based load-balancing algorithm
• localpeer as an environment variable
• stick-tables extended with gpc1/gpc1_rate counters

Cache optimization

• Age headers
• Chunked Transfer Encoding support
• max-object-size = 2GB, total-max-size = 4GB
• Updated list of cacheable status codes ( 204, 404, 405, 414, 501 )

New fetches

• ssl_fc_session_key / ssl_bc_session_key: Returns the SSL master key of the front/back connection
• set SSL_OP_PRIORITIZE_CHACHA: Uses the client’s preference when selecting the cipher suite
• fe_defbe: Fetches frontend default backend name

New converters

• strcmp
• field/word converter extended
• ipv6 added to ipmask converter

Stick Table Aggregator

Now supports:

• SIGHUP to reload
• Environment variables
• source keyword for outbound connections
• Syslog debug logging

Version 1.8r1 Jump to heading #

Key changes in the HAProxy Enterprise 1.8r1 release include:

HTTP/2 Protocol Support

• HAProxy Enterprise now supports HTTP/2 on the client side (i.e. the front-end sections) and can act as a gateway between HTTP/2 clients and your HTTP/1.1 and HTTP/1.0 applications.

Multi-threading Model

• The multi-threading model is a feature that allows HAProxy Enterprise to start multiple threads within a single process.

Server-template Configuration Directive

• The server-template configuration directive allows you to instantiate a number of placeholder backend servers in a single configuration line, instead of having to add tens or hundreds of configuration lines explicitly.

Dynamic Cookies

• A new keyword dynamic for the directive cookie now exists to enable cookie-based application persistence.

Improved Runtime API

The Runtime API allows administrators and automation systems to interact with HAProxy Enterprise during runtime. HAProxy Enterprise version 1.8 comes with further improvements and new directives:

SSL/TLS Mode Async

• The OpenSSL library version 1.1.0 and above supports asynchronous I/O operations. This means that the key computation phase (the heaviest part of the TLS protocol) can be run in a non-blocking way in HAProxy Enterprise.

New Sample Fetches

HAProxy Enterprise can extract data from traffic passing through it. We call these functions “sample fetches”. More information about sample fetches can be found under “Fetching Data Samples” in the HAProxy Enterprise documentation.

This version has a group of new sample fetches:

New Converters

Version 1.7r2 Jump to heading #

Key changes in the HAProxy Enterprise 1.7r2 release include:

Stick Table Aggregator

The stick-table aggregator stores stick-table entries and aggregates them before it transfers the results back to HAProxy Enterprise peers on the network. Thus you can see and reference a single stick table which holds the combined values from all nodes in the cluster. You can find its user documentation on the HAProxy Enterprise Customer Portal.

Master-Worker Model

• In the master-worker model, HAProxy Enterprise launches one master process and starts a number of additional worker processes under it.
• The new master process monitors all worker processes and controls them from a single instance to relieve systemd or other software from these tasks.

HTTP Small Object Caching

The small object caching mechanism allows HAProxy Enterprise to store some small static files from application servers in memory, and thereby accelerate the delivery of CSS, JS, or icon files.

DNS SRV Record Support

DNS SRV records allow HAProxy Enterprise to get a list of backend servers from DNS records. This is an alternative to building the configuration from template whenever the backends change or using the run-time API to modify backends.

Version 1.7r1 Jump to heading #

HAProxy Enterprise 1.7r1 comprises the following new functionalities:

Core improvements

• PCRE JIT support
• Namespace support
• DNS improvements
• Server: init-addr: indicate in what order the server’s address should be resolved upon startup. Method none specifically indicates that the server should start without any valid IP address in a down state.
• New srv_admin flag: SRV_ADMF_RMAINT
• Listener: accept-netscaler-cip option added to the bind keyword
• Added tcp-request connection expect-netscaler-cip layer4
• log-format: error management improvements

New Fetches

• fe_name: gives the current frontend’s name
• be_name: gives the current backend’s name
• fc_rcvd_proxy shows if the client initiated the connection with a PROXY protocol header
• xx-hash converters

Runtime API

• show cli sockets list the CLI sockets
• show stat now supports a proxy name
• show errors now supports a proxy name
• show errors is now capable of dumping only request or response
• RMAINT state remains when setting an IP address on the CLI

LUA Scripting

• CLI handler for LUA
• Allow argument for actions
• HAProxy Enterprise now has variable access to applets
• Function which returns true if the channel is full
• IP addresses and network manipulation function
• Utility function to check for a boolean argument
• A tokenize function

Version 1.6r2 Jump to heading #

Key changes in the HAProxy Enterprise 1.6r2 release include:

• HTTP REST API

• List HAProxy Enterprise modules and their versions from the CLI

• LUA support is now enabled in HAProxy Enterprise

• New lb-update features

  • TLS session ticket keys
  • Ability to launch an update from the CLI
  • Optional logs
  • Various fixes

• TCP layer statistics are available in the form of new sample fetches (rtt, rtt variance, retransmits, losses, etc.)

• New dst_is_local, src_is_local sample fetches make it easier to take care of locally initiated connections in contrast with remote connections

• The peers protocol was updated to version 2.1 with support for synchronization of expiration dates

• CLI keyword registration to allow modules to plug-in on the command line and receive actions (for example: refresh now)

• SO_REUSEPORT can now be disabled with a new bind directive: noreuseport

• New possibilities for accessing load balancer internals with LUA

• Idle time in logs is now ignored by default for time measurements so that HTTP request time corresponds to the time elapsed between the first character and the full request. The handshake time is also available to measure the time spent in SSL/PROXY handshakes.

• stick-tables now use native types to guarantee better accuracy of tracked information, especially binary keys

• Mailers now support a configurable connection timeout

• Changing server address, port and checkport is now possible from the CLI

• New tcp-request session rule-sets makes it possible to track some L5-only information, for example anything negotiated in the handshake such as SSL DN or the client’s IP address as passed by the PROXY protocol. Previously it was needed to do it in tcp-request content rules, which would count one new connection event per request.

• New stats field for denied connections and denied sessions

• New hash-balance-factor directive for consistent hashing method

• New http-response rule track-sc

• SSL/TLS

  • SNI filters supported in multi-type certificates

Version 1.6r1 Jump to heading #

Key changes in the HAProxy Enterprise 1.6r1 release include:

• New http-reuse to share unused server-side connections for multiple clients

• Support of double quotes in the configuration

• DNS resolution of server name at run time

• GZIP stateless compression

• Support of variables at run time

• Stick tables statistics synchronization

• New hash-type load-balancing algorithm crc32

• Redispatch interval

• Dynamic generation of SSL certificates

• Mathematical converters can use a variable generated at runtime

• New http-request rules

  • set-var(<var name>) <expr>
  • sc-inc-gpc0(<sc-id>)
  • sc-set-gpt0(<sc-id>) <int>
  • silent-drop
  • set-src <expr>

• New http-response rules

  • set-status
  • set-var(<var-name>) <expr>
  • sc-inc-gpc0(<sc-id>)
  • sc-set-gpt0(<sc-id>) <int>
  • silent-drop

• New tcp-request connection rules

  • sc-inc-gpc0(<sc-id>)
  • sc-set-gpt0(<sc-id>) <int>
  • silent-drop

• New tcp-response content rules

  • set-var(<var-name>) <expr>
  • sc-inc-gpc0(<sc-id>)
  • sc-set-gpt0(<sc-id>) <int>
  • silent-drop

• New tcp-request content rules

  • sc-inc-gpc0(<sc-id>)
  • sc-set-gpt0(<sc-id>) <int>
  • set-var(<var-name>) <expr>
  • silent-drop

• New converters

  • capture-req(<id>)
  • capture-res(<id>)
  • set-var(<var name>)
  • table_gpt0(<table>)

• New fetches

  • Internal states

    • bin(<hexa>)
    • bool(<bool>)
    • int(<integer>)
    • ipv4(<ipv4>)
    • ipv6(<ipv6>)
    • meth(<method>)
    • str(<string>)
    • var(<var-name>)

  • Layer 4

    • sc_get_gpt0(<ctr>[,<table>]), sc0_get_gpt0([<table>]), ...
    • src_get_gpt0([<table>])

  • Layer 6

    • req.ssl_st_ext
    • res.ssl_hello_type

• global section features

  • presetenv
  • resetenv
  • setenv
  • unsetenv

• SSL/TLS - multi-certificate key algorithm support per certificate

• New map type: map_regm to allow matching zones inthe regex.

• Layer 7 fetches

  • unique-id

• Pre-defined ACLs

  • METH_DELETE
  • METH_PUT

• New filters trace and HTTP compression.

Version 1.5r2 Jump to heading #

Key changes in the HAProxy Enterprise 1.5r2 release include:

• Support for 2 new operating systems: Debian 8 and Ubuntu 12.04

• Support of systemd on Debian 8, RedHat 7 and CentOS 7

• Analysis of HTTP request body (req.body HAProxy Enterprise fetch)

• Peers on a single process when nbproc > 1

• Support for HTTP stateless compression

• Configurable tls-tickets keys

• Automatic maxconn and maxsslconn to match enforced memory limits

• External checker (fork of current process) (be careful with chroots…)

• tcp-check comments

• Email alerts

• Fetches for http request body

• Server state reload proof

• HTTP capture on response

• HTTP capture with a slot number

• Server side TLS extension SNI

• Server side tcp-ut (TCP user timeout at kernel layer)

• Management of environment variables through the global section

• New HAProxy Enterprise fetches:

  • ssl_fc_is_resumed
  • req.ssl_ec_ext
  • req.body
  • req.body_param
  • req.body_len
  • req.body_size

• New commands available over the stats socket:

  • set ssl tls-key
  • show backend
  • show servers state
  • show tls-keys
  • show env