HAProxy and SSL

The history of SSL in HAProxy is very short: around one month ago, we announced the ability for HAProxy to offload SSL from the servers.
HAProxy SSL stack comes with some advanced features like TLS extension SNI.

Well, since yesterday afternoon (Tuesday the 2nd), HAProxy can also offload the client certificate management from the server, with some advanced features. This is the purpose of today’s article.
Again, all the dev is provided by HAProxy Technologies.
For the people using the ALOHA Load-Balancer, these features will be included in the next release without GUI integration (which will come later).
Concerning HAProxy, just git clone the latest version or wait for HAProxy-1.5-dev13. When compiling, don’t forget the USE_OPENSSL=yes flag.

Introduction

Why client certificates?

The main purpose of using client-side certificates is to increase the level of authentication.
Since HAProxy is often in front of web platform, it is the right place to do this authentication. That way, it could do all the certificate checking before allowing the user to pass through. Then it can process SSL on behalf of server and apply any standard features.
The main purpose of the article is to introduce the new HAProxy features related to SSL client certificates.
Basically, we’ll see how to protect access to our application with client-side certificates and how to properly redirect users to the right page when there is an issue with their certificates.

SSL Client certificate generation: thanks nginx!

Well, we’ll have to create a CA, a server certificate and clients certificates!
Nathan, a nginx user, has written a very nice and well documented article on how to generate a CA and client certificate here: http://blog.nategood.com/client-side-certificate-authentication-in-ngi.
So I won’t rewrite all the procedure here, just follow Nathan instructions to create your own CA and generate a few client certificates. All you need is available here:
https://github.com/exceliance/haproxy/blob/master/blog/ssl_client_certificate_management_at_application_level/

Other stuff

During this article, we’ll use a few client certificates: client1, client2 and client_expired (whose certificate has … expired!).
On the githbu link above, you’ll find as well how to generate the PEM file required by HAProxy.

Phase 1: Client Certificate mandatory

In the configuration below, only users with a client certificate are allowed to get connected on the application. This is achieved by the keywords “verify required“.

frontend ft_ssltests
 mode http
 bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify required
 default_backend bk_ssltests

backend ssltests
 mode http
 server s1 192.168.10.101:80 check
 server s2 192.168.10.102:80 check

If the client does not provide any certificate, then HAProxy would shut the connection during the SSL handshake. It’s up the the user’s software to report the right error…
Testing:

• Connection with a certificate is allowed:

  $ openssl s_client -connect 192.168.10.1:443 -cert ./client1.crt -key ./client1.key

• Connection without a certificate is refused:

  $ openssl s_client -connect 192.168.10.1:443
  [...]ssl handshake failure[...]

• Connection with an expired certificate is refused too:

  $ openssl s_client -connect 192.168.10.1:443 -cert ./client_expired.crt -key ./client_expired.key 
  [...]ssl handshake failure[...]

Phase 2: Client Certificate optional

In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
We can redirect users to different farm, based on the presence of the certificate or not:

frontend ssltests
 mode http
 bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional
 use_backend sharepoint if { ssl_fc_has_crt }     # check if the certificate has been provided and give access to the application
 default_backend webmail

backend sharepoint
 mode http
 server srv1 192.168.10.101:80 check
 server srv2 192.168.10.102:80 check

backend webmail
 mode http
 server srv3 192.168.10.103:80 check
 server srv4 192.168.10.104:80 check

• If the client does not provide any certificate, then HAProxy route him to the webmail.
• If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
• If the client provides an expired certificate, then HAProxy refuses the connection like in the phase 1

Phase 3: Client Certificate optional and managing expired certificates

In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err 10” tells HAProxy to ignore Certificate errors 10 which actually matches the expired certificate.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate

frontend ssltests
 mode http
 bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional crt-ignore-err 10
 use_backend static if { ssl_c_verify 10 }  # if the certificate has expired, route the user to a less sensitive server to print an help page
 use_backend sharepoint if { ssl_fc_has_crt }        # check if the certificate has been provided and give access to the application
 default_backend webmail

backend static
 mode http
 option http-server-close
 redirect location /certexpired.html if { ssl_c_verify 10 } ! { path /certexpired.html }
 server srv5 192.168.10.105:80 check
 server srv6 192.168.10.106:80 check

backend sharepoint
 mode http
 server srv1 192.168.10.101:80 check
 server srv2 192.168.10.102:80 check

backend webmail
 mode http
 server srv3 192.168.10.103:80 check
 server srv4 192.168.10.104:80 check

• If the client does not provide any certificate, then HAProxy route him to the webmail.
• If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
• If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).

Phase 4: Client Certificate optional, managing expired certificates and a revocation list

In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err all” tells HAProxy to ignore any client Certificate error.
The option “crl-file ./ca_crl.pem” tells HAProxy to check if the client has not been revoked in the Certificate Revocation List provided in argument.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate. We can also present a dedicate page to users whose certificate has been revoked.

frontend ssltests
 mode http
 bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.crt verify optional crt-ignore-err all crl-file ./ca_crl.pem
 use_backend static unless { ssl_c_verify 0 }  # if there is an error with the certificate, then route the user to a less sensitive farm
 use_backend sharepoint if { ssl_fc_has_crt }           # check if the certificate has been provided and give access to the application
 default_backend webmail

backend static
 mode http
 option http-server-close
 redirect location /certexpired.html if { ssl_c_verify 10 } ! { path /certexpired.html } # SSL error 10 means "certificate expired"
 redirect location /certrevoked.html if { ssl_c_verify 23 } ! { path /certrevoked.html } # SSL error 23 means "Certificate revoked"
redirect location /othererrors.html unless { ssl_c_verify 0 } ! { path /othererrors.html }
 server srv5 192.168.10.105:80 check
 server srv6 192.168.10.106:80 check

backend sharepoint
 mode http
 server srv1 192.168.10.101:80 check
 server srv2 192.168.10.102:80 check

backend webmail
 mode http
 server srv3 192.168.10.103:80 check
 server srv4 192.168.10.104:80 check

• If the client does not provide any certificate, then HAProxy route him to the webmail.
• If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
• If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).
• If the client provides a revoked certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the revoked certificate (it’s up to the admin to write this page).
• For any other errors related to the client certificate, then HAProxy routes the user to a static server (non-sensitive) and force the users to show a page explaining there has been an error and how to contact the support (it’s up to the admin to write this page).

Phase 5: same as phase 4, but with multiple CAs, Cert error in header and some ACLs

In the configuration below, all users, those with and those without the certificate are allowed to get connected. This is achieved by the keyword “verify optional“.
The option “crt-ignore-err all” tells HAProxy to ignore all client Certificate.
The option “crl-file ./ca_crl.pem” tells HAProxy to check if the client has not been revoked in the Certificate Revocation List provided in argument.
The file ca.pem contains 2 CAs: ca and ca2.
We can redirect users to different farm, based on the presence of the certificate or not and we can propose a dedicated page for users whose certificate has expired with a procedure on how to renew or ask for a new certificate. We can also present a dedicate page to users whose certificate has been revoked.

frontend ssltests
 mode http
 bind 192.168.10.1:443 ssl crt ./server.pem ca-file ./ca.pem verify optional crt-ignore-err all crl-file ./ca_crl.pem
 use_backend static unless { ssl_c_verify 0 }  # if there is an error with the certificate, then route the user to a less sensitive farm
 use_backend sharepoint if { ssl_fc_has_crt }           # check if the certificate has been provided and give access to the application
 default_backend webmail

backend static
 mode http
 option http-server-close
 acl url_expired path /certexpired.html
 acl url_revoked path /certrevoked.html
 acl url_othererrors path /othererrors.html
 acl cert_expired ssl_c_verify 10
 acl cert_revoked ssl_c_verify 23
 reqadd X-Ssl-Error: 10 if cert_expired
 reqadd X-Ssl-Error: 23 if cert_revoked
 reqadd X-Ssl-Error: other if ! cert_expired ! cert_revoked
 redirect location /certexpired.html if cert_expired ! url_expired
 redirect location /certrevoked.html if cert_revoked ! url_revoked
 redirect location /othererrors.html if ! cert_expired ! cert_revoked ! url_othererrors
 server srv5 192.168.10.105:80 check
 server srv6 192.168.10.106:80 check

backend sharepoint
 mode http
 server srv1 192.168.10.101:80 check
 server srv2 192.168.10.102:80 check

backend webmail
 mode http
 server srv3 192.168.10.103:80 check
 server srv4 192.168.10.104:80 check

• If the client does not provide any certificate, then HAProxy route him to the webmail.
• If the client provides a certificate, then HAProxy routes him to the application (sharepoint in our example)
• If the client provides an expired certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the expired certificate and how to renew it (it’s up to the admin to write this page).
• If the client provides a revoked certificate, then HAProxy routes him to a static server (non-sensitive) and force the users to show the page which provides the explanation about the revoked certificate (it’s up to the admin to write this page).
• For any other errors related to the client certificate, then HAProxy routes the user to a static server (non-sensitive) and force the users to show a page explaining there has been an error and how to contact the support (it’s up to the admin to write this page).

Coming soon…

Later, we’ll improve HAProxy client certificate management with:

• client certificate information in HTTP header
• ACLs to match the client information provided in the certificate and take classic decision (routing, blocking, etc….)
• Persistence based on the information provided by the certificate (stick tables)
• Ability to use a client certificate to get connected (and authenticated) on a server

Don’t hesitate to send your us your wishes!

Related links

• OpenSSL verify status code
• Client side certificate auth in Nginx (brilliant article by the way)
• HAProxy Technologies Github scripts related to this article
• How to get SSL with HAProxy getting rid of stunnel, stud, nginx or pound
• Enhanced SSL load-balancing with Server Name Indication (SNI) TLS extension
• Maintain affinity based on SSL session ID
• Benchmarking SSL performance

Links

• HAProxy Technologies
• Aloha load balancer: HAProxy based LB appliance
• HAPEE: HAProxy Enterprise