HAProxy Enterprise Documentation 2.2r1

SNMP

Simple Network Management Protocol (SNMP) offers a way to collect information about network devices. Having been around for decades, you'll find it in many different types of devices such as routers, switches, servers, and printers. The HAProxy Enterprise SNMP module enables you to collect metrics from the load balancer. The module provides read access to load balancer statistics.

This guide demonstrates the setup using SNMPv3.

Install the management software

The SNMP management software is not, strictly speaking, required, but it does provide tools for creating SNMPv3 user accounts and testing that the setup works by making SNMP queries.

  1. Install the SNMP management software package:

    $ # On Debian/Ubuntu
    $ sudo apt update
    $ sudo apt install snmp
    
    $ # On RedHat
    $ sudo yum install net-snmp-utils
  2. On Debian and Ubuntu, perform these additional steps:

    1. Configure the management software to load Management Information Bases (MIBs) by editing the file /etc/snmp/snmp.conf and commenting out the mibs line so that it looks like this:

      # mibs :
    2. Append the path to the HAProxy Enterprise MIB directory, /opt/hapee-extras/misc, to the end of the mibdirs line, and uncomment it if it is commented out:

      mibdirs /usr/share/snmp/mibs:/usr/share/snmp/mibs/iana:/usr/share/snmp/mibs/ietf:/opt/hapee-extras/misc
    3. Add the non-free repository to the existing line in /etc/apt/sources.list, which will allow you to install the snmp-mibs-downloader package:

      deb https://deb.debian.org/debian bullseye main non-free
    4. Add the MIB files by installing the snmp-mibs-downloader package:

      $ sudo apt update
      $ sudo apt install snmp-mibs-downloader

Install the agent software

An SNMP agent is software that runs on the monitored device, which in this case is the load balancer. It returns metrics when queried by the management software.

  1. Install the SNMP agent software package:

    $ # On Debian/Ubuntu
    $ sudo apt update
    $ sudo apt install snmpd
    
    $ # On RedHat
    $ sudo yum install net-snmp
  2. Enable the agent service:

    $ sudo systemctl enable snmpd
  3. Stop the agent service:

    $ sudo systemctl stop snmpd
  4. Edit the agent configuration file, /etc/snmp/snmpd.conf, and make the following changes:

    • Add the following createUser and rwuser lines to the end of the file to create an initial user account with read-write access. Note that this applies to SNMPv3 only, since older versions of the protocol do not require user accounts and instead use a community string for authentication. This initial user account will become the template from which we will create other user accounts.

    createUser initial SHA setup_passphrase AES setup_passphrase
    rwuser initial
    • Add another rwuser line to prepare a second user account with read-write access. Below, we define permissions for a user named myuser:

    rwuser myuser
    • Add the following pass_persist line to the end of the file. It configures the SNMP agent service to pass requests through to the HAProxy Enterprise SNMP module:

    pass_persist  .1.3.6.1.4.1.23263.4.3.1.3 /opt/hapee-extras/bin/hapee-snmp-lb
    • On Debian and Ubuntu, to have the agent service listen on all interfaces instead of 127.0.0.1, comment out the line agentAddress udp:127.0.0.1:161 if it exists and add the line agentAddress udp:161,udp6:[::1]:161.

    # Listen for connections from the local system only
    # agentAddress  udp:127.0.0.1:161
    
    # Listen for connections on all interfaces (both IPv4 *and* IPv6)
    agentAddress udp:161,udp6:[::1]:161
  5. Start the agent service:

    $ sudo systemctl start snmpd
  6. Using the initial user's username and passphrase, create a new user account that you'll use to make SNMP requests. This user will inherit settings from the initial user account. Below, we create a user named myuser:

    $ snmpusm -v3 -n "" -u initial -a SHA -A setup_passphrase -x AES -X setup_passphrase -l authPriv 127.0.0.1 create myuser initial
    User successfully created.
  7. The new user account inherited the passphrase from the initial user account. Use the snmpusm command again to change the user's passphrase. Below, we change the myuser user account's passphrase to mypassword. In a production environment, be sure to use a strong password.

    $ snmpusm -v3 -n "" -u myuser -a SHA -A setup_passphrase -x AES -X setup_passphrase -l authPriv 127.0.0.1 passwd setup_passphrase mypassword
    SNMPv3 Key(s) successfully changed.
  8. To verify that the user you created can make requests to the agent software, you can use the management software's snmpget command to fetch some metrics. Try the following:

    $ snmpget -v3 -n "" -u myuser -a SHA -A mypassword -x AES -X mypassword -l authPriv 127.0.0.1 sysUpTime.0
    DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (34926) 0:05:49.26

Install the HAProxy Enterprise SNMP module

  1. Install the HAProxy Enterprise SNMP module:

    $ # On Debian/Ubuntu
    $ sudo apt-get install hapee-extras-snmp-lb
    $ # On CentOS/RedHat/Oracle/Photon OS
    $ sudo yum install hapee-extras-snmp-lb
    $ # On SUSE
    $ sudo zypper install hapee-extras-snmp-lb
    $ # On FreeBSD
    $ sudo pkg install hapee-extras-snmp-lb
  2. In the global section of your load balancer configuration, add a stats socket line that points to /var/run/hapee-extras/hapee-lb.sock. The SNMP module will connect to this socket to retrieve metrics data. Note that your configuration will likely come with a stats socket line that's different from this. You can add a second stats socket line to support this feature:

    global
       stats socket /var/run/hapee-extras/hapee-lb.sock user hapee-lb group hapee mode 660 level user

    On Debian and Ubuntu, set the group parameter on this line to Debian-snmp instead of hapee. That is the group in which the SNMP service runs.

  3. Optional: Add id directives to your frontend and backend sections. The id directive's value is an integer that indicates that section's unique identifier, which will appear in the SNMP output, making it easier to tell one frontend or backend from another. Otherwise, an ID will be set for you. The IDs for a frontend and backend do not need to relate in any way.

    frontend www
      id 1
    
    backend webservers
      id 10
    
    backend dbservers
      id 20
  4. Restart the HAProxy Enterprise service:

    $ sudo systemctl restart hapee-2.2-lb
  5. To verify that requests for load balancer metrics get passed to the module, use the snmpwalk command to return some data:

    $ # On Debian/Ubuntu
    $ snmpwalk -v3 -u myuser -a SHA -A mypassword -x AES -X mypassword -l authPriv 127.0.0.1 EXCELIANCE-MIB::lbStats
    
    $ # On RedHat
    $ snmpwalk -v3 -u myuser -a SHA -A mypassword -x AES -X mypassword -l authPriv -M /usr/share/snmp/mibs:/opt/hapee-extras/misc -m EXCELIANCE-MIB 127.0.0.1 EXCELIANCE-MIB::lbStats

    Note that EXCELIANCE is the company acquired by HAProxy Technologies.

    EXCELIANCE-MIB::lbProcessID.1 = INTEGER: 1
    EXCELIANCE-MIB::lbProcessVersion.1 = STRING: "2.7.0-1.0.0-301.904"
    EXCELIANCE-MIB::lbProcessReleaseDate.1 = STRING: "2023/09/07"
    EXCELIANCE-MIB::lbProcessNbProc.1 = INTEGER: 1
    EXCELIANCE-MIB::lbProcessProductName.1 = STRING: "hapee-lb"
    EXCELIANCE-MIB::lbProcessSystemPID.1 = INTEGER: 9494
    EXCELIANCE-MIB::lbProcessUptime.1 = STRING: "0d 0h21m29s"
  6. Configure your SNMP monitoring software to collect metrics from the HAProxy Enterprise server by using the SNMPv3 protocol with the username and passphrase you set.

How it works

The SNMP stack is split into two main components:

  • The Operating system SNMP agent service, called snmpd. It listens on a network interface on port 161/UDP and handles SNMP requests from clients.

  • The SNMP module for HAProxy Enterprise, which collects data from the load balancer for snmpd.

The diagram below illustrates how the SNMP agent service monitors the system's network, system disks, and HAProxy Enterprise:

HAProxy Enterprise SNMP service

Troubleshooting

You get the error 'No Such Instance currently exists at this OID'

This error indicates that although the object at the given ID (the metric) was found on this server, no value was returned for it.

  • Try disabling SE Linux, which can interfere with the SNMP service.

  • Check that you are using the right object ID (OID). You can either use the human readable names or the numeric names. The snmptranslate command shows you the OIDs that are defined in the MIB file /opt/hapee-extras/misc/HAPEE.mib.

    The name of the MIB is EXCELIANCE-MIB, from the company Exceliance that HAProxy Technologies acquired.

    $ # On Debian/Ubuntu
    $ snmptranslate -Pu -Tz -m EXCELIANCE-MIB
    
    $ # On RedHat
    $ snmptranslate -Pu -Tz -M /usr/share/snmp/mibs:/opt/hapee-extras/misc -m EXCELIANCE-MIB
    "hapee"                 "1.3.6.1.4.1.23263.4.3"
    "lbcompat1"                     "1.3.6.1.4.1.23263.4.3.1"
    "lbProductInfo"                 "1.3.6.1.4.1.23263.4.3.1.1"
    "lbStats"                       "1.3.6.1.4.1.23263.4.3.1.3"
    "lbProcessTable"                        "1.3.6.1.4.1.23263.4.3.1.3.1"
    "lbProcessTableEntry"                   "1.3.6.1.4.1.23263.4.3.1.3.1.1"
    "lbProcessID"                   "1.3.6.1.4.1.23263.4.3.1.3.1.1.1"
  • Check that the pass_persist is working. To debug the pass_persist line in the agent configuration file, use the following commands to run the agent in debug mode:

    $ sudo systemctl stop snmpd
    $ sudo snmpd -f -Lo -Ducd-snmp/pass

    After invoking snmpget or snmpwalk, the debugger should show output like the following:

    getnext
    .1.3.6.1.4.1.23263.4.3.1.3.4.1.27.1.5.3
    Connection from UDP: [127.0.0.1]:52720->[127.0.0.1]:161
    ucd-snmp/pass_persist: open_persist_pipe(1,'/opt/hapee-extras/bin/hapee-snmp-lb') recurse=0
    ucd-snmp/pass_persist: persistpass-sending:

    When finished, stop the snmpd command and start the snmpd service.

  • Check that your stats socket line is correct in the load balancer configuration file. It should use /var/run/hapee-extras/hapee-lb.sock (you will probably need to add this as a new line in your configuration).

    To verify that requests are passing to the stats socket, you can place a proxy in front of the socket and then monitor the traffic. In the following example, we use socat as a proxy. It shows that the socket is being queried and is returning data:

    $ sudo systemctl restart hapee-2.7-lb
    $ sudo apt install socat
    $ sudo mv /var/run/hapee-extras/hapee-lb.sock /var/run/hapee-extras/hapee-lb.original
    $ sudo socat -t100 -x -v UNIX-LISTEN:/var/run/hapee-extras/hapee-lb.sock,mode=777,reuseaddr,fork UNIX-CONNECT:/var/run/hapee-extras/hapee-lb.original
    > 2023/09/21 21:30:29.555027  length=10 from=0 to=9
     73 68 6f 77 20 69 6e 66 6f 0a                    show info.
    --
    < 2023/09/21 21:30:29.583831  length=1158 from=0 to=1157
     4e 61 6d 65 3a 20 68 61 70 65 65 2d 6c 62 0a     Name: hapee-lb.
     56 65 72 73 69 6f 6e 3a 20 32 2e 37 2e 30 2d 31  Version: 2.7.0-1
     2e 30 2e 30 2d 33 30 31 2e 39 30 34 0a           .0.0-301.904.
     52 65 6c 65 61 73 65 5f 64 61 74 65 3a 20 32 30  Release_date: 20
     32 33 2f 30 39 2f 30 37 0a                       23/09/07.
     4e 62 74 68 72 65 61 64 3a 20 32 0a              Nbthread: 2.
     4e 62 70 72 6f 63 3a 20 31 0a                    Nbproc: 1.
     50 72 6f 63 65 73 73 5f 6e 75 6d 3a 20 31 0a     Process_num: 1.
     50 69 64 3a 20 39 36 32 37 0a                    Pid: 9627.
     55 70 74 69 6d 65 3a 20 30 64 20 30 68 30 30 6d  Uptime: 0d 0h00m
     32 32 73 0a                                      22s.

    When finished, restart the HAProxy Enterprise service to restore the original socket.


Next up

Monitoring