It’s almost wild how two years have passed already since we first announced the HAProxy Ingress Controller for Kubernetes. Note that this project is different from the jcmoraisjr/haproxy-ingress HAProxy ingress controller project on GitHub. Our project is also an open-source project, but it is overseen by engineers at HAProxy Technologies, which has allowed us to streamline its release cycle. It seems like a round of celebrations are in order not just for our own project, but for the open-source community as a whole. This year, Linux is celebrating 30 years of existence and HAProxy is 20 years old. Yes, 20 years on and HAProxy still remains the most performant load balancer in the world! With only two years chalked up for the ingress controller, we can’t compare, but there are many things we can learn from those giants, and I can only hope to have such a great journey.
The first steps are done and we’ve managed to reach 10 million community downloads!
total number of downloads during two years
The idea for the ingress controller began simply as a way to help our community deploy and configure HAProxy even in an environment that was very different from a traditional datacenter. The same idea catalyzed the effort to create a RESTful API for HAProxy, which would allow users to integrate the load balancer within many other types of environments and processes. Having a very similar purpose, the ingress controller and the HAProxy Dataplane API were built simultaneously and with shared libraries.
Shared Libraries
Those libraries were created as separate projects so that we or others in the community could reuse them to build additional tools in the future. The two libraries are named config-parser and client-native.
config-parser
Config-parser, which you can learn more about on its GitHub page, is a Go package that reads and modifies HAProxy configuration files. Our most important goal was to make it as simple as possible. Regardless of that, it can still read complete configurations that contain more than 100k lines. They do exist!
The main task of the parser is to read the entire configuration and store every field in memory, even if some of those fields don’t have a sub-parser yet that allows us to edit them. HAProxy has a lot of options, so we prioritize making the most popular options editable so that we can expose them in the ingress controller as Kubernetes annotations. Another requirement is to preserve everything in order after modifying the configuration, as HAProxy expects, like ordering acl lines to appear before use_backend lines.
client-native
If the config-parser library is the vehicle for reading and modifying the HAProxy configuration file, then the client-native library, which also has a GitHub page, is the driver.
It is a Go package that allows communication with HAProxy, both for sending configuration file changes and executing commands against HAProxy’s embedded Runtime API. There is also a built-in OpenAPI 2.0 specification (3.0 compatible) that you can use to generate native Go models, complete with validation functions, for use in your client applications. Both the HAProxy Kubernetes Ingress Controller and the Dataplane API use it to communicate with HAProxy. This package interacts with config-parser to read and save configurations.
One of its main features, besides runtime changes, is support for batching configuration changes and validation checks into a transaction, which safeguards your load balancer from enacting an invalid or partial set of instructions. There is also an option to do extra external validation if needed.
Both packages allow building robust applications, and we can reuse the same logic and be consistent across multiple products.
HAProxy libraries and projects
The Go Programming Language
Why Go ?
Selecting the right programming language is very opinionated, and many teams and developers often have different opinions about which language is great for what. So, my response will be very simple: Why not? 🙂
On a more serious note, like any other programming language it has its pros and cons, but Go ticks a lot of boxes. It’s fast and easy to understand. It has great support and an active community. It builds a single binary with no dependencies (in general), and every new release brings a tiny bit more performance and optimization. Go also makes it simple to build for multiple platforms. There are many more great reasons. Did I mention Docker and Kubernetes are built with Go?
Just so that I can complain a bit, one major issue with Go had been the lack of official dependency management tools. At the time of creating the projects, we already knew that Go modules would become a standard part of the language. So, we made the decision to avoid dependency management tools that existed and rely on modules. The last package that we needed, client-go, added support for Go modules just in time, and it happened to be the most important one, used to communicate with Kubernetes. Some of our design decisions, especially during the early stages of the ingress controller, were impacted by that anticipated major change in the Go language. This sometimes required some creativity, but with careful planning everything was ready for release.
Helm Charts
Soon after the Kubernetes ingress controller was released, we added support for Helm charts. You’ll find more information about them in our Kubernetes ingress controller installation documentation and on the GitHub page. Helm charts in general help with defining, installing and upgrading applications in Kubernetes.
There are two charts. One allows users to install the ingress controller and another installs HAProxy as a traditional, but containerized, load balancer in Kubernetes.
Customize, customize, customize…
You can customize the ingress controller in multiple ways, and customization is fine grained.
Most general customization can be done by adding annotations to a ConfigMap resource, which affects all services. Additional tuning is available at the Ingress and Service level so the controller can work in a very specific way for your pods. This kind of setup allows users to fully tailor the behaviour of the controller for every service they operate.
Recently, we’ve added config-snippets. Users with more advanced knowledge of the underlying HAProxy syntax can define snippets of HAProxy markup that apply to global, frontend and backend parts of the configuration. This allow you to go above and beyond the functionality exposed via annotations and wield all of the power that HAProxy provides.
On top of that, support for defining a secondary configuration file was also added, allowing access to config sections like resolvers, cache, and so on.
Expanding Platforms
The number of platform that the controller supports continues to expand. Since version 1.6.1, with every new release there are automatic builds of binaries for 12 different platforms. This year, multi-arch support was added, and the latest releases have multi-arch Docker images.
Another recent update, called external mode, which you can read about in a previous blog post, also pushes the controller to new frontiers. From the very start, the option to run the ingress controller outside of a Kubernetes cluster existed. However, we used it only for debugging and developing purposes. This year, we exposed that feature in a way that makes sense for a production environment. It has proven to be quite an interesting and popular way of running an ingress controller.
What’s Next…
As always, there is room for improvement and there are reasons to add new features. Lately, there is more and more demand to further customize and support new capabilities. Both Kubernetes and HAProxy have added options that the community is using. With so many options that HAProxy provides in general, we have lots of exciting work ahead. The same goes for Kubernetes features too.
Through GitHub issues, and a dedicated Slack channel on the HAProxy Slack workspace, many interesting ideas are being proposed and discussed. This allows us to have a better understanding of some of the hot features that are most anticipated.
Also, we are seeing more and more contributions from the community. From only one contributor at the start, we can now say that numbers are growing (27 currently on the ingress controller itself, not counting all of the other projects that it integrates with) and this is one of the biggest strengths of open-source projects in general.
At the same time, the team at HAProxy Technologies that works on the ingress controller is growing too. As a team lead, I can honestly say that it’s a privilege and pleasure to work with such a team!
