Troubleshooting
Network troubleshooting
This page describes tools you can use to diagnose network issues.
If you change the configuration, don’t forget to save it. To make your changes persistent after a reboot, click the Setup tab. Then click Save under Configuration.
ARP diagnostics Jump to heading
This troubleshooting method uses Address Resolution Protocol (ARP) to check MAC / IP address associations.
From the Web user interface (web UI) Jump to heading
-
Open the DiagTools tab.
-
From the Select list, choose
arp who-has (interface, source*, destination)
. -
Enter the following parameters:
-
interface
: the interface where to send the ARP packet -
source*
: (optional) source IP -
destination
: the destination IP from which to get the MAC address
-
-
Click on Run.
From the command line interface (CLI) Jump to heading
-
Get root rights by typing
root
. -
Run
arping -c 5 -I <interface> [-s <source>] <destination IP>
.outputtextARPING 10.0.32.10 from 10.0.32.11 eth0Unicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.636msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.608msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.650msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.666msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.612msoutputtextARPING 10.0.32.10 from 10.0.32.11 eth0Unicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.636msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.608msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.650msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.666msUnicast reply from 10.0.32.10 [00:0D:C5:36:C6:72] 0.612ms
Analyze output Jump to heading
In the example above, the MAC address is associated to the IP 10.0.32.10
.
Tips Jump to heading
Duplicated IPs Jump to heading
ARP WHO-HAS packets are useful to fix duplicated IPs issues.
When you can reach a server or a VIP on the network, you can use ARP WHO-HAS to check if the MAC address associated with the IP address is the one expected or not:
-
If expected, the problem could be on a upper layer.
-
If not expected, then the MAC address of the server has the duplicated IP.
Packet capture Jump to heading
The diagnostic tool capture
captures network traffic to retrieve the following information:
-
Source MAC and IP address
-
VRID
-
VRRP priority
From the Web user interface (web UI) Jump to heading
-
Open the DiagTools tab.
-
From Select list, choose
capture (interface*, filter*, ignored)
. -
Enter the following parameters:
-
interface
: Capture on a single interface. Default capture on all interfaces -
filter
: (optional) which packets to capture. By default, all packets are captured.For example:
-
A protocol name:
vrrp
,icmp
-
A specific IP address: host
<ip>
-
A specific TCP or UDP port: port
<port>
-
-
-
Click Run.
From the command line interface (CLI) Jump to heading
-
Get root rights by typing
root
. -
Run
tcpdump -vvvenns0 -c 5 [-i <interface>] <filter>
. Note that the output below is purposely truncated.outputtexttcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes04:59:41.088388 00:15:5d:75:2e:1b > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 106: (tos 0x0, ttl 255, id 879, offset0 flags [none], [...]04:59:41.157687 00:15:5d:75:2e:45 > 00:15:5d:75:2e:1b, ethertype ARP (0x0806), length 42: arp who-has 10.0.0.190(00:15:5d:75:2e:1b) tell 10.0.0.18704:59:41.157723 00:15:5d:75:2e:1b > 00:15:5d:75:2e:45, ethertype ARP (0x0806), length 42: arp reply 10.0.0.190 is-at00:15:5d:75:2e:1b04:59:41.306124 00:15:5d:75:2e:1b > 00:15:5d:75:2e:45, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 48464,offset 0, flags [DF], [...]04:59:41.307063 00:15:5d:75:2e:45 > 00:15:5d:75:2e:1b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6419,offset 0, flags [DF], [...]5 packets captured0 packets received by filter0 packets dropped by kerneloutputtexttcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes04:59:41.088388 00:15:5d:75:2e:1b > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 106: (tos 0x0, ttl 255, id 879, offset0 flags [none], [...]04:59:41.157687 00:15:5d:75:2e:45 > 00:15:5d:75:2e:1b, ethertype ARP (0x0806), length 42: arp who-has 10.0.0.190(00:15:5d:75:2e:1b) tell 10.0.0.18704:59:41.157723 00:15:5d:75:2e:1b > 00:15:5d:75:2e:45, ethertype ARP (0x0806), length 42: arp reply 10.0.0.190 is-at00:15:5d:75:2e:1b04:59:41.306124 00:15:5d:75:2e:1b > 00:15:5d:75:2e:45, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 48464,offset 0, flags [DF], [...]04:59:41.307063 00:15:5d:75:2e:45 > 00:15:5d:75:2e:1b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 6419,offset 0, flags [DF], [...]5 packets captured0 packets received by filter0 packets dropped by kernel
Tips Jump to heading
Capture VRRP traffic Jump to heading
To capture VRRP traffic, use vrrp
as <filter>
and choose an interface
outputtext
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes14:12:27.936839 00:0c:49:65:47:0e > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 30, prio 35, ... , addrs: ...14:12:28.250381 00:0c:59:20:c7:f6 > ... , proto VRRP (112), 10.9.104.241 > 224.0.0.18, ... , vrid 166, prio 133, ... , addrs: ...14:12:28.460930 00:0d:b9:18:b9:74 > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 55, prio 70, ... , addrs: ...14:12:28.939831 00:0c:49:65:47:0e > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 30, prio 35, ... , addrs: ...14:12:28.250381 00:0c:59:20:c7:f6 > ... , proto VRRP (112), 10.9.104.241 > 224.0.0.18, ... , vrid 166, prio 133, ... , addrs: ...5 packets captured0 packets received by filter0 packets dropped by kernel
outputtext
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes14:12:27.936839 00:0c:49:65:47:0e > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 30, prio 35, ... , addrs: ...14:12:28.250381 00:0c:59:20:c7:f6 > ... , proto VRRP (112), 10.9.104.241 > 224.0.0.18, ... , vrid 166, prio 133, ... , addrs: ...14:12:28.460930 00:0d:b9:18:b9:74 > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 55, prio 70, ... , addrs: ...14:12:28.939831 00:0c:49:65:47:0e > ... , proto VRRP (112), 10.9.104.253 > 224.0.0.18, ... , vrid 30, prio 35, ... , addrs: ...14:12:28.250381 00:0c:59:20:c7:f6 > ... , proto VRRP (112), 10.9.104.241 > 224.0.0.18, ... , vrid 166, prio 133, ... , addrs: ...5 packets captured0 packets received by filter0 packets dropped by kernel
Ping Jump to heading
Pinging ensures that an IP address is reachable through the network.
From the Web user interface (web UI) Jump to heading
-
Open the DiagTools tab.
-
From the Select list, choose
ping (source*, destination, ignored)
. -
Enter the following parameters:
source*
: (optional) source IP to usedestination
: the destination IP to reach
-
Click Run.
From the command line interface (CLI) Jump to heading
- Get root rights by typing
root
. - Run
ping -n -c 5 [-s <source>] <destination>
.
Successful Jump to heading
outputtext
PING 10.0.33.101 (10.0.33.101) 56(84) bytes of data.64 bytes from 10.0.33.101: icmp_seq=1 ttl=64 time=0.051 ms64 bytes from 10.0.33.101: icmp_seq=2 ttl=64 time=0.030 ms64 bytes from 10.0.33.101: icmp_seq=3 ttl=64 time=0.024 ms64 bytes from 10.0.33.101: icmp_seq=4 ttl=64 time=0.030 ms64 bytes from 10.0.33.101: icmp_seq=5 ttl=64 time=0.036 ms--- 10.0.33.101 ping statistics ---5 packets transmitted, 5 received, 0% packet loss, time 4008msrtt min/avg/max/mdev = 0.024/0.034/0.051/0.009 ms
outputtext
PING 10.0.33.101 (10.0.33.101) 56(84) bytes of data.64 bytes from 10.0.33.101: icmp_seq=1 ttl=64 time=0.051 ms64 bytes from 10.0.33.101: icmp_seq=2 ttl=64 time=0.030 ms64 bytes from 10.0.33.101: icmp_seq=3 ttl=64 time=0.024 ms64 bytes from 10.0.33.101: icmp_seq=4 ttl=64 time=0.030 ms64 bytes from 10.0.33.101: icmp_seq=5 ttl=64 time=0.036 ms--- 10.0.33.101 ping statistics ---5 packets transmitted, 5 received, 0% packet loss, time 4008msrtt min/avg/max/mdev = 0.024/0.034/0.051/0.009 ms
Unsuccessful Jump to heading
outputtext
PING 10.0.33.101 (10.0.33.101) 56(84) bytes of data.--- 10.0.33.101 ping statistics ---5 packets transmitted, 0 received, 100% packet loss, time 3999ms
outputtext
PING 10.0.33.101 (10.0.33.101) 56(84) bytes of data.--- 10.0.33.101 ping statistics ---5 packets transmitted, 0 received, 100% packet loss, time 3999ms
Routing diagnostics Jump to heading
This diagnostic tool prints HAProxy ALOHA routing table.
From the Web user interface (web UI) Jump to heading
-
Open the DiagTools tab.
-
From the Select list, choose advanced
routes (ignored, ignored, ignored)
. -
Optionally, enter the parameter:
interface
: (optional) limit output to routes associated to this interface
-
Click on Run.
From the command line interface (CLI) Jump to heading
-
Get root rights by typing
root
. -
Run
ip route list
.
Single default gateway Jump to heading
outputtext
default via 10.0.0.161 dev eth010.0.0.160/27 dev eth0 proto kernel scope link src 10.0.0.190127.255.255.1 dev sync proto kernel scope link src 127.255.255.2192.168.10.0/24 dev eth2 proto kernel scope link src 192.168.10.3192.168.13.0/24 dev eth3 proto kernel scope link src 192.168.13.3192.168.14.0/24 dev eth1 proto kernel scope link src 192.168.14.3192.168.15.0/24 via 192.168.10.1 dev eth2
outputtext
default via 10.0.0.161 dev eth010.0.0.160/27 dev eth0 proto kernel scope link src 10.0.0.190127.255.255.1 dev sync proto kernel scope link src 127.255.255.2192.168.10.0/24 dev eth2 proto kernel scope link src 192.168.10.3192.168.13.0/24 dev eth3 proto kernel scope link src 192.168.13.3192.168.14.0/24 dev eth1 proto kernel scope link src 192.168.14.3192.168.15.0/24 via 192.168.10.1 dev eth2
Multiple default gateways Jump to heading
outputtext
default via 10.0.0.161 dev eth0default via 192.168.14.1 dev eth1 metric 1default via 192.168.10.1 dev eth2 metric 2default via 192.168.13.1 dev eth3 metric 310.0.0.160/27 dev eth0 proto kernel scope link src 10.0.0.190127.255.255.1 dev sync proto kernel scope link src 127.255.255.2192.168.10.0/24 dev eth2 proto kernel scope link src 192.168.10.3192.168.13.0/24 dev eth3 proto kernel scope link src 192.168.13.3192.168.14.0/24 dev eth1 proto kernel scope link src 192.168.14.3192.168.15.0/24 via 192.168.10.1 dev eth2
outputtext
default via 10.0.0.161 dev eth0default via 192.168.14.1 dev eth1 metric 1default via 192.168.10.1 dev eth2 metric 2default via 192.168.13.1 dev eth3 metric 310.0.0.160/27 dev eth0 proto kernel scope link src 10.0.0.190127.255.255.1 dev sync proto kernel scope link src 127.255.255.2192.168.10.0/24 dev eth2 proto kernel scope link src 192.168.10.3192.168.13.0/24 dev eth3 proto kernel scope link src 192.168.13.3192.168.14.0/24 dev eth1 proto kernel scope link src 192.168.14.3192.168.15.0/24 via 192.168.10.1 dev eth2
Do you have any suggestions on how we can improve the content of this page?