HAProxy Enterprise Documentation 2.5r1

Consul

Consul operates as a service mesh when you enable its Connect mode. In this mode, Consul agents integrate with HAProxy Enterprise to form an interconnected web of proxies. Whenever one of your services needs to call another, their communication is relayed through the web, or mesh, with HAProxy Enterprise nodes passing messages between all services.

An HAProxy Enterprise node exists next to each of your services on both the caller and callee end. When a caller makes a request, they direct it to localhost where HAProxy Enterprise is listening. HAProxy Enterprise then relays it transparently to the remote callee. From the caller's perspective, all services appear to be local, which simplifies the service's configuration.

Kubernetes

This section describes how to deploy the Consul service mesh with HAProxy Enterprise in Kubernetes.

Deploy the Consul servers

Consul agents running in server mode watch over the cluster and send service discovery information to each Consul client in the service mesh.

Deploy the Consul server nodes. In Kubernetes, you can install the Consul Helm chart.
```
$ helm repo add hashicorp https://helm.releases.hashicorp.com
$ helm repo update
$ helm install consul hashicorp/consul --set global.name=consul --set connect=true
```
Note

If you are using a single-node Kubernetes cluster, such as minikube, then set the server.replicas and server.bootstrapExpect flags, as described in the guide Consul Service Discovery and Mesh on Minikube.
```
--set server.replicas=1
--set server.bootstrapExpect=1
```

Create a file named pod-reader-role.yaml and add the following contents to it.

This creates a Role and RoleBinding resource in your Kubernetes cluster that grant permissions to the Consul agents to read pod labels.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: system:serviceaccount:default:default
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Deploy it with kubectl apply:
```
$ kubectl apply -f pod-reader-role.yaml
```
Optional: The Helm chart creates a Kubernetes service named consul-server that exposes a web dashboard on port 8500. To make it available outside of the Kubernetes cluster, you can forward the port via the HAProxy Enterprise Kubernetes Ingress Controller.
1. Deploy the HAProxy Enterprise Kubernetes Ingress Controller into your Kubernetes cluster.
2. Create a file named consul-server-ingress.yaml that defines an Ingress resource for the Consul service.
  
  In this example, we define a host-based rule that routes all requests for consul.test.local to the consul-server service at port 8500.
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: consul-server-ingress
spec:
  rules:
    - host: consul.test.local
      http:
        paths:
          - path: "/"
            pathType: Prefix
            backend:
              service:
                name: consul-server
                port:
                  number: 8500
```
3. Deploy it using kubectl apply:
```
$ kubectl apply -f consul-server-ingress.yaml
```
4. Add an entry to your system's hosts file that maps the consul.test.local hostname to the IP address of your Kubernetes cluster. If you are using minikube, you can get the IP address of the node with minikube ip. Below is an example hosts file:
```
192.168.99.125  consul.test.local
```
5. Use kubectl get service to check which port the ingress controller has mapped to port 80. In the example below, port 80 is mapped to port 30624.
```
$ minikube ip
192.168.99.120

$ kubectl get service kubernetes-ingress
NAME                 TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                                     AGE
kubernetes-ingress   NodePort   10.110.104.60   <none>        80:30624/TCP,443:31147/TCP,1024:31940/TCP   7m40s
```
  Open a browser window and go to the consul.test.local address at that port, e.g. consul.test.local:30624.

Deploy your application

For each service that you want to include in the service mesh, you must deploy two extra containers into the same pod.

container 1: your application
container 2: Consul agent, consul
container 3: HAProxy-Consul connector, hapee-plus-registry.haproxy.com/hapee-consul-connect

The three containers (application, Consul, Consul-HAProxy Enterprise connector) are defined inside a single pod.

Use kubectl create secret to store your credentials for the private HAProxy Docker registry, replacing <KEY> with your HAProxy Enterprise license key. You will pull the hapee-consul-connect container image from this registry.
```
$ kubectl create secret docker-registry regcred \
    --namespace=default \
    --docker-server=hapee-plus-registry.haproxy.com \
    --docker-username=<KEY> \
    --docker-password=<KEY>
```

Add the haproxy-enterprise-consul and consul containers to each of your Kubernetes Deployment manifests. In the example below, we deploy these two containers inside the same pod as a service named news-service.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: news-service
 labels:
   app: news-service
spec:
 replicas: 1
 selector:
   matchLabels:
     app: news-service
 template:
   metadata:
     labels:
       app: news-service
   spec:
     imagePullSecrets:
     - name: regcred
     containers:
     - name: news-service
       image: jmalloc/echo-server
     - name: haproxy-enterprise-consul
       image: hapee-plus-registry.haproxy.com/hapee-consul-connect
       args:
       - -sidecar-for=news-service
       - -enable-intentions
     - name: consul
       image: consul
       env:
             - name: CONSUL_LOCAL_CONFIG
               value: '{
                 "service": {
                   "name": "news-service",
                   "port": 80,
                   "connect": {
                     "sidecar_service": {}
                   }
                 }
             }'
       args: ["agent", "-bind=0.0.0.0", "-retry-join=provider=k8s label_selector=\"app=consul\""]

Note the following arguments for the haproxy-enterprise-consul container:

Argument	Description
`-sidecar-for news-service`	Indicates the name of the service for which to create an HAProxy Enterprise proxy.
`-enable-intentions`	Enables Consul intentions, which HAProxy Enterprise enforces.

Note the following arguments for the consul container:

Argument	Description
`agent`	Runs the Consul agent.
`-bind=0.0.0.0`	The address that should be bound to for internal cluster communications.
`-retry-join=provider=k8s label_selector="app=consul"`	Similar to `-join`, which specifies the address of another agent to join upon starting up (typically one of the Consul server agents), but allows retrying a join until it is successful. In Kubernetes, you set this to provider=k8s and then include a label selector for finding the Consul servers. The Consul Helm chart adds the label app=consul to the Consul server pods.

We've registered the news-service with the Consul service mesh by setting an environment variable named CONSUL_LOCAL_CONFIG in the Consul container. This defines the Consul configuration and registraton for the service. It indicates that the service receives requests on port 80.

{
  "service": {
    "name": "news-service",
    "port": 80,
    "connect": {
      "sidecar_service": {}
  }
}

Deploy it with kubectl apply:
```
$ kubectl apply -f news-service.yaml
```

Deploy a second application that calls the other

The news-service from the previous section is published to the service mesh where other services within the mesh can call it. To define a service that calls another, add a proxy section to the connect.sidecar_service section of the Consul container's configuration.

In the example below, the service named app-ui adds the news-service as an upstream service, which makes it available at localhost at port 3000 inside the pod.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: app-ui
 labels:
   app: app-ui
spec:
   replicas: 1
   selector:
     matchLabels:
       app: app-ui
   template:
     metadata:
       labels:
         app: app-ui
     spec:
       containers:
       - name: app-ui
         image: jmalloc/echo-server
       - name: haproxy-enterprise-consul
         image: hapee-plus-registry.haproxy.com/hapee-consul-connect
         args:
         - -sidecar-for=app-ui
         - -enable-intentions
       - name: consul
         image: consul
         env:
         - name: CONSUL_LOCAL_CONFIG
           value: '{
               "service": {
                 "name": "app-ui",
                 "port": 80,
                 "connect": {
                   "sidecar_service": {
                       "proxy": {
                           "upstreams": [
                               {
                                 "destination_name": "news-service",
                                 "local_bind_port": 3000
                               }
                           ]
                       }
                   }
                 }
               }
             }'
         args: ["agent", "-bind=0.0.0.0", "-retry-join=provider=k8s label_selector=\"app=consul\""]

Note that we set the environment variable named CONSUL_LOCAL_CONFIG in the Consul container to register this service with the service mesh. It declares that it has an upstream dependency on the news-service service.

Enable Consul ACLs

In Consul, ACLs are a security measure that requires Consul agents to present an authentication token before they can join the cluster or call API methods.

When installing Consul, set the global.acls.manageSystemACLs flag to true to enable ACLs.

$ helm install consul hashicorp/consul \
    --set global.name=consul \
    --set connect=true \
    --set global.acls.manageSystemACLs=true

Use kubectl get secret to get the auto-generated bootstrap token, which is base64 encoded:

$ sudo apt install jq
$ kubectl get secret consul-bootstrap-acl-token \
    -o json | jq -r '.data.token' | base64 -d

8f1c8c5e-d0fb-82ff-06f4-a4418be245dc

Use this token to log into the Consul web UI.

In the Consul web UI, go to ACL > Policies and select the client-token row. Change the policy's value so that the service_prefix section has a policy of write:
```
node_prefix "" {
  policy = "write"
}
service_prefix "" {
  policy = "write"
}
```
Go back to the ACL screen and select the client-token row. Copy this token value (e.g. f62a3058-e139-7e27-75a0-f47df9e2e4bd).

For each of your services, update your Deployment manifest so that the haproxy-enterprise-consul container includes the -token argument, set to the client-token value.

- name: haproxy-enterprise-consul
  image: hapee-plus-registry.haproxy.com/hapee-consul-connect
  args:
  args:
  - -sidecar-for=app-ui
  - -enable-intentions
  - -token=f62a3058-e139-7e27-75a0-f47df9e2e4bd

Update the consul container's configuration to include an acl section where you will specify the same client-token value. Also, set primary_datacenter to dc1 (or to the value you've set for your primary datacenter, if you have changed it).

- name: consul
  image: consul
  env:
    - name: CONSUL_LOCAL_CONFIG
      value: '{
        "primary_datacenter": "dc1",
        "acl": {
          "enabled": true,
          "default_policy": "allow",
          "down_policy": "extend-cache",
          "tokens": {
            "default": "f62a3058-e139-7e27-75a0-f47df9e2e4bd"
          }
        },
        "service": {
          "name": "news-service",
          "port": 80,
          "connect": {
            "sidecar_service": {}
          }
        }
      }'

Next up

Protocols