HAProxy and SSL
The history of SSL in HAProxy is very short: around one month ago, we announced the ability for HAProxy to offload SSL from the servers.
HAProxy SSL stack comes with some advanced features like TLS extension SNI.
Well, since yesterday afternoon (Tuesday the 2nd), HAProxy can also offload the client certificate management from the server, with some advanced features. This is the purpose of today’s article.
Again, all the dev is provided by HAProxy Technologies.
For the people using the ALOHA Load-Balancer, these features will be included in the next release without GUI integration (which will come later).
Concerning HAProxy, just git clone the latest version or wait for HAProxy-1.5-dev13. When compiling, don’t forget the USE_OPENSSL=yes flag.
Introduction
Why client certificates?
The main purpose of using client-side certificates is to increase the level of authentication.
Since HAProxy is often in front of web platform, it is the right place to do this authentication. That way, it could do all the certificate checking before allowing the user to pass through. Then it can process SSL on behalf of server and apply any standard features.
The main purpose of the article is to introduce the new HAProxy features related to SSL client certificates.
Basically, we’ll see how to protect access to our application with client-side certificates and how to properly redirect users to the right page when there is an issue with their certificates.
SSL Client certificate generation: thanks nginx!
Well, we’ll have to create a CA, a server certificate and clients certificates!
Nathan, a nginx user, has written a very nice and well documented article on how to generate a CA and client certificate here: http://blog.nategood.com/client-side-certificate-authentication-in-ngi.
So I won’t rewrite all the procedure here, just follow Nathan instructions to create your own CA and generate a few client certificates. All you need is available here:
https://github.com/exceliance/haproxy/blob/master/blog/ssl_client_certificate_management_at_application_level/
Other stuff
During this article, we’ll use a few client certificates: client1, client2 and client_expired (whose certificate has … expired!).
On the githbu link above, you’ll find as well how to generate the PEM file required by HAProxy.
Phase 1: Client Certificate mandatory
In the configuration below, only users with a client certificate are allowed to get connected on the application. This is achieved by the keywords “verify required“.
frontend ft_ssltests mode http bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify required default_backend bk_ssltests backend ssltests mode http server s1 192.168.10.101:80 check server s2 192.168.10.102:80 check
If the client does not provide any certificate, then HAProxy would shut the connection during the SSL handshake. It’s up the user’s software to report the right error…
Testing:
- Connection with a certificate is allowed:
$ openssl s_client -connect 192.168.10.1:443 -cert ./client1.crt -key ./client1.key
- Connection without a certificate is refused:
$ openssl s_client -connect 192.168.10.1:443 [...]ssl handshake failure[...]
- Connection with an expired certificate is refused too:
$ openssl s_client -connect 192.168.10.1:443 -cert ./client_expired.crt -key ./client_expired.key [...]ssl handshake failure[...]
Phase 2: Client Certificate optional
In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
We can redirect users to different farm, based on the presence of the certificate or not:
frontend ssltests mode http bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional use_backend sharepoint if { ssl_fc_has_crt } # check if the certificate has been provided and give access to the application default_backend webmail backend sharepoint mode http server srv1 192.168.10.101:80 check server srv2 192.168.10.102:80 check backend webmail mode http server srv3 192.168.10.103:80 check server srv4 192.168.10.104:80 check
- If the client does not provide any certificate, then HAProxy route him to the webmail.
- If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
- If the client provides an expired certificate, then HAProxy refuses the connection like in the phase 1
Phase 3: Client Certificate optional and managing expired certificates
In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err 10” tells HAProxy to ignore Certificate errors 10 which actually matches the expired certificate.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate
frontend ssltests mode http bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional crt-ignore-err 10 use_backend static if { ssl_c_verify 10 } # if the certificate has expired, route the user to a less sensitive server to print an help page use_backend sharepoint if { ssl_fc_has_crt } # check if the certificate has been provided and give access to the application default_backend webmail backend static mode http option http-server-close redirect location /certexpired.html if { ssl_c_verify 10 } ! { path /certexpired.html } server srv5 192.168.10.105:80 check server srv6 192.168.10.106:80 check backend sharepoint mode http server srv1 192.168.10.101:80 check server srv2 192.168.10.102:80 check backend webmail mode http server srv3 192.168.10.103:80 check server srv4 192.168.10.104:80 check
- If the client does not provide any certificate, then HAProxy route him to the webmail.
- If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
- If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).
Phase 4: Client Certificate optional, managing expired certificates and a revocation list
In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err all” tells HAProxy to ignore any client Certificate error.
The option “crl-file ./ca_crl.pem” tells HAProxy to check if the client has not been revoked in the Certificate Revocation List provided in argument.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate. We can also present a dedicate page to users whose certificate has been revoked.
frontend ssltests mode http bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional crt-ignore-err all crl-file ./ca_crl.pem use_backend static unless { ssl_c_verify 0 } # if there is an error with the certificate, then route the user to a less sensitive farm use_backend sharepoint if { ssl_fc_has_crt } # check if the certificate has been provided and give access to the application default_backend webmail backend static mode http option http-server-close redirect location /certexpired.html if { ssl_c_verify 10 } ! { path /certexpired.html } # SSL error 10 means "certificate expired" redirect location /certrevoked.html if { ssl_c_verify 23 } ! { path /certrevoked.html } # SSL error 23 means "Certificate revoked" redirect location /othererrors.html unless { ssl_c_verify 0 } ! { path /othererrors.html } server srv5 192.168.10.105:80 check server srv6 192.168.10.106:80 check backend sharepoint mode http server srv1 192.168.10.101:80 check server srv2 192.168.10.102:80 check backend webmail mode http server srv3 192.168.10.103:80 check server srv4 192.168.10.104:80 check
- If the client does not provide any certificate, then HAProxy route him to the webmail.
- If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
- If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).
- If the client provides a revoked certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the revoked certificate (it’s up to the admin to write this page).
- For any other errors related to the client certificate, then HAProxy routes the user to a static server (non-sensitive) and force the users to show a page explaining there has been an error and how to contact the support (it’s up to the admin to write this page).
Phase 5: same as phase 4, but with multiple CAs, Cert error in header and some ACLs
In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err all” tells HAProxy to ignore all client Certificate.
The option “crl-file ./ca_crl.pem” tells HAProxy to check if the client has not been revoked in the Certificate Revocation List provided in argument.
The file ca.pem contains 2 CAs: ca and ca2.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate. We can also present a dedicate page to users whose certificate has been revoked.
frontend ssltests mode http bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.pem verify optional crt-ignore-err all crl-file ./ca_crl.pem use_backend static unless { ssl_c_verify 0 } # if there is an error with the certificate, then route the user to a less sensitive farm use_backend sharepoint if { ssl_fc_has_crt } # check if the certificate has been provided and give access to the application default_backend webmail backend static mode http option http-server-close acl url_expired path /certexpired.html acl url_revoked path /certrevoked.html acl url_othererrors path /othererrors.html acl cert_expired ssl_c_verify 10 acl cert_revoked ssl_c_verify 23 reqadd X-Ssl-Error: 10 if cert_expired reqadd X-Ssl-Error: 23 if cert_revoked reqadd X-Ssl-Error: other if ! cert_expired ! cert_revoked redirect location /certexpired.html if cert_expired ! url_expired redirect location /certrevoked.html if cert_revoked ! url_revoked redirect location /othererrors.html if ! cert_expired ! cert_revoked ! url_othererrors server srv5 192.168.10.105:80 check server srv6 192.168.10.106:80 check backend sharepoint mode http server srv1 192.168.10.101:80 check server srv2 192.168.10.102:80 check backend webmail mode http server srv3 192.168.10.103:80 check server srv4 192.168.10.104:80 check
- If the client does not provide any certificate, then HAProxy route him to the webmail.
- If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
- If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).
- If the client provides a revoked certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the revoked certificate (it’s up to the admin to write this page).
- For any other errors related to the client certificate, then HAProxy routes the user to a static server (non-sensitive) and force the users to show a page explaining there has been an error and how to contact the support (it’s up to the admin to write this page).
Coming soon…
Later, we’ll improve HAProxy client certificate management with:
- client certificate information in HTTP header
- ACLs to match the client information provided in the certificate and take classic decision (routing, blocking, etc….)
- Persistence based on the information provided by the certificate (stick tables)
- Ability to use a client certificate to get connected (and authenticated) on a server
Don’t hesitate to send your us your wishes!
Related links
- OpenSSL verify status code
- Client side certificate auth in Nginx (brilliant article by the way)
- HAProxy Technologies Github scripts related to this article
- How to get SSL with HAProxy getting rid of stunnel, stud, nginx or pound
- Enhanced SSL load-balancing with Server Name Indication (SNI) TLS extension
- Maintain affinity based on SSL session ID
- Benchmarking SSL performance
The new SSL features are fantastic (and so are the new ACLs, like request rates)! I think it would be very useful to be able to actually get the SSL error code — instead of just putting X-Ssl-Error: Other in the request, either stuff the actual error code in, or perhaps append as a parameter to the URI. This would allow the use of a dynamic web page for all SSL errors with content related to the error that actually occurred. Of course, the admin can still setup a default page, but access to the error code would add a lot of flexibility to SSL error handling.
Hi – can you confirm the URL below is the correct one to use for cloning HAProxy-1.5-dev13?
It seems to be hanging this morning… I am not sure if it is a git server issue or an incorrect URL being used. Thanks!
git clone http://git.1wt.eu/git/haproxy.git
Hi – git server must have been slow/offline the past hour… but all is ok now. Clone command works fine now. Thanks.
Small fix to post:
Baptiste, in phases 4 and 5 you are talking about crlfile option but forgot to add the option to configs.
Hi Ilya,
You’re absolutely right, thanks for reporting.
Article updated.
cheers
Thanks. I’m looking for similar solution for my application. I need manage client certificates from a DB of certificates and then store a session to DB.
Is there any solution for these needs?
Hi,
not with HAProxy.
Baptiste
Unfortunately this doesn’t work for me with version 1.5-dev18, I can’t even get Phase 1 of this example to work. I always get:
140501283583648:error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca:s3_pkt.c:1248:SSL alert number 48
140501283583648:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
Hi,
Please send a mail to HAProxy ML with your configuration and the commands you used to test your configuration.
Baptiste
Hi Baptiste,
I did send an email to the mailing list but unfortunately did not receive any replies.
Tom
Hi,
Answered on the ML.
You’re issue is related to your CN names: they are the same for your CA, server and client certificates.
Please use a different one and it works.
Baptiste
Are there any plans on supporting “client certificate information in HTTP header”? This is what currently stops me from switching from nginx to haproxy.
Hi Nick,
This has been developped.
You can install haproxy-1.5-dev18, the git version…
It’s stable and have the required features.
I’ll write an article on the blog soon, so you’ll be able to follow instructions.
Stay tuned.
Baptiste
Hi Nick,
Please find the article here: http://blog.exceliance.fr/2013/06/13/client-certificate-information-in-http-headers-and-logs/
Baptiste
I’m wondering if the article you mentioned to Nick will be ready soon? I have a setup where I need the certificate that the client provides in the request to HAProxy to be forwarded to the backend application servers, and a guide on how to do so would be very useful. At the moment I’ve only figured out how to get HAProxy to send its own certificate information to the backend, but that is not sufficient. Thanks, and great articles!
Hi Mike,
Please find the article here: http://blog.exceliance.fr/2013/06/13/client-certificate-information-in-http-headers-and-logs/
Baptiste
Hello Baptiste,
thank you for this great and very helpful article. At the end of this article you announced a follow-up article about the “Ability to use a client certificate to get connected (and authenticated) on a Server”. What I need is the possibility to use a client certificate (stored on haproxy) which haproxy uses for client-based authentication on a remote http(s) server. Is there a chance to set this up with haproxy version 1.5 ?
Thank’s again for your great articles
Best regards
Lothar
Hello Baptiste,
I’m using this configuration to accept client certificates successfully, but now I have a need to make the client certificate mandatory only on certain paths of a web site. Is this allowed by haproxy?
thanks for this information ! Keep it this good work !
I just want to point some issues on this post;
unless { ssl_c_verify 0 } -> if if { ssl_c_verify ne 0 }
if { ssl_c_verify 10 } -> if { ssl_c_verify eq 10 }
if { ssl_c_verify 23} -> if { ssl_c_verify eq 23}