Our R&D department published a well-received blog post titled “Truly Seamless Reloads with HAProxy – No More Hacks!”. In it, we explained why some established connections could be dropped during reloads under high utilization, and how the problem has been solved in HAProxy Enterprise Edition 1.8r1 and HAProxy 1.8. (“High utilization” typically means handling thousands of new connections per second and simultaneously reloading HAProxy processes multiple times per second.)
That blog post has generated a great deal of user interest for actually enabling this feature in their configurations, so we have decided to post a quick and efficient HOWTO.
As the original blog post explained, HAProxy 1.8 implements hitless reloads by transferring listening sockets from the old HAProxy instance to the new, reloaded one. This not only allows for the configuration to be updated but also for HAProxy to be upgraded without any dropped connections or downtime. In order for this to work, the old and the new processes need to be configured with the Runtime API option that will enable handing off the connections.
Making it work is very simple – the HAProxy configuration file needs to be updated, and in some cases, a command line option needs to be added.
Configuration File Update
To update the configuration file, simply add the option “expose-fd listeners” to an existing Runtime API socket definition, or add the whole line if you currently do not have it enabled:
stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user
Command Line Options Update
Please note that this step does not need to be performed if your HAProxy configuration already contains the directive “master-worker”, or if it is started with the option -W.
That is the default with newer HAProxy versions running on GNU/Linux distributions which use systemd. In those cases, the option “-x” is added to new worker processes automatically if there is at least one socket with option “expose-fd listeners” set on it, and so no changes are necessary.
If, however, your HAProxy command line options do need to be updated, simply add the option “-x [socket_file]” to the haproxy arguments, pointing it to the location of the Runtime API socket.
haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -sf $(cat /run/haproxy.pid) -x /var/run/haproxy.sock
And that is all there is to enabling hitless reloads with HAProxy!
Hitless Reloads are available in HAProxy Enterprise Edition 1.7r1 and above as well as HAProxy 1.8 and above.
For HAProxy 1.7 users who would like to take advantage of this feature without a major upgrade in your production environment, consider HAProxy Enterprise Edition 1.7r1 which provides cutting edge, stable features backported from HAProxy 1.8
We hope you have enjoyed this summarized blog post providing the HOWTO for enabling truly hitless reloads with HAProxy.
If you would like to run HAProxy in your infrastructure backed by enterprise support from HAProxy Technologies, please see our HAProxy Enterprise Edition – Trial Version or contact us for expert advice.
Happy hitless reloading and stay tuned!