A Linux host that has many Ethernet interfaces on the same subnet is subject to the ARP Flux problem.
The ARP Flux problem occurs when a host replies to ARP requests for interfaces on the same subnet, from any interface on that same subnet. In most cases, this behavior is not a problem. However, in specific cases, ARP Flux generates unexpected behavior of applications due to incorrect mapping between IPv4 addresses and MAC addresses.
The ARP Protocol
The Address Resolution Protocol (ARP) is used to translate an IPv4 address into a MAC address, allowing Layer 2 communications to occur (e.g Ethernet and WiFi). When a source host has to send an IPv4 packet to a destination host on the same subnet, it executes an ARP request to get the MAC address of the destination host. The source host will send an ARP request to the Ethernet broadcast address ff:ff:ff:ff:ff:ff. The ARP broadcasted request will reach all hosts on the subnet. Only the destination host will answer to the ARP request with the MAC address associated with its IPv4 address.
Once the source has obtained the destination MAC address from the destination host, it encapsulates the IPv4 packet into a Layer 2 frame, and sends it to the destination host.
ARP Flux
As mentioned at the beginning of this article, the problem with ARP Flux can occur when a host with two interfaces, on the same subnet, reply to ARP requests for interfaces on the same subnet, from any interface on that same subnet.
Let’s imagine, for example, that an application running between Host A and Host B must go through Host B’s eth1 interface, due to firewall rules or other reasons. ARP Flux may cause Host B to reply to Host A’s ARP requests with either the MAC address of eth0 or eth1. As you can from the above picture, what happens in this scenario, traffic from Host A to Host B’s eth1 interface will be instead processed by its eth0 interface. This non deterministic behavior may cause such application running between the hosts not to function properly. If you want to learn more about Linux implementation of the ARP protocol, you can read this article.
Avoiding ARP Flux
Starting from kernel 2.6.4, the default behavior of the Linux kernel can be corrected with the two following commands:
$ sysctl -w net.ipv4.conf.all.arp_announce=1
$ sysctl -w net.ipv4.conf.all.arp_ignore=2
According to the ARP Configuration Guide, these two parameters cause the Linux Kernel to implement the following behavior:
arp_announce = 1 – Try to avoid local addresses that are not in the target’s subnet for this interface. Target hosts reachable via this interface need the receiving interface to have the source IP address included in ARP requests as part of their logical network configuration. When generating the request, we will check all of the subnets that include the target IP and will preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for level 2.
arp_ignore = 2 – Reply only if the target IP address is local address configured on the incoming interface and both with the sender’s IP address are part from same subnet on this interface.
If you want to permanently save these parameters upon reboot, add the two statements in your host’s /etc/sysctl.conf configuration file.
