multi-homed systems stop answering ARP on local addresses w/ifconfig aliases

Mon Apr 26 13:52:37 UTC 2010

On Sun, May 17, 2009 at 4:35 PM, Steven Hartland
<killing at multiplay.co.uk <http://lists.freebsd.org/mailman/listinfo/freebsd-net>> wrote:
>/ Silly question but something else on the network isn't doing a arp spoof
/>/ attack is it?
/>/
/
No, there isn't any ARP at all on that address on the network when
this is a problem, verified with tcpdump. That also shouldn't impact
the system's ability to talk to its own IPs.

thanks for the response though!

>/ ----- Original Message ----- From: "Chris Buechler"
/>/ <freebsd at chrisbuechler.com <http://lists.freebsd.org/mailman/listinfo/freebsd-net>>
/>/ To: <net at freebsd.org <http://lists.freebsd.org/mailman/listinfo/freebsd-net>>
/>/ Sent: Sunday, May 17, 2009 9:08 PM
/>/ Subject: multi-homed systems stop answering ARP on local addresses
/>/ w/ifconfig aliases
/>/
/>/
/>>/ There seems to be a regression between 6.x and 7.0 and 7.1 related to
/>>/ ifconfig aliases on multi-homed hosts. Not sure on anything newer than 7.1
/>>/ (this is pfSense, we're just starting to test 7.2 builds). For periods of
/>>/ time, the system will stop answering ARP on some of its own addresses and
/>>/ hence anything on that network completely stops functioning. The same setup
/>>/ worked fine on 6.2.
/>>/
/>>/ The particular system illustrated here is a router on part of an ISP's
/>>/ network. IPs are all public, in the info provided here they've been replaced
/>>/ with 10. IPs. The subnets on the inside interfaces are routed to the outside
/>>/ interface. When this problem occurs, the IPs assigned locally on the system
/>>/ will still respond from the Internet, but the system itself loses all
/>>/ connectivity with that subnet and nothing on that subnet can communicate
/>>/ with the host due to the lack of ARP. That makes some sense, I presume when
/>>/ routing to a locally assigned address via another interface, the system
/>>/ doesn't need ARP on the address to respond. But while it still responds from
/>>/ the Internet, even the host itself can't initiate a ping to that IP. It
/>>/ behaves the same whether pf is enabled or disabled.
/>>/
/>>/ I see two similar issues in the past, one with a PR:
/>>/ http://www.freebsd.org/cgi/query-pr.cgi?pr=121437&cat= <http://www.freebsd.org/cgi/query-pr.cgi?pr=121437&cat=>
/>>/ that's exactly the same issue, it's not limited to VLANs, any multi-homed
/>>/ host is affected.
/>>/
/>>/ And another:
/>>/ http://thread.gmane.org/gmane.os.freebsd.stable/57125
/>>/
/>>/ fxp0 is the outside interface. It doesn't make any difference whether the
/>>/ ifconfig aliases are on the em0 or fxp1 interfaces, they both behave the
/>>/ same if they have any ifconfig aliases assigned.
/>>/
/>>/ # ifconfig
/>>/ fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
/>>/       options=8<VLAN_MTU>
/>>/       ether 00:90:27:86:8b:9d
/>>/       inet6 fe80::290:27ff:fe86:8b9d%fxp0 prefixlen 64 scopeid 0x1
/>>/       inet 10.11.185.146 netmask 0xfffffff8 broadcast 10.11.185.151
/>>/       media: Ethernet 100baseTX <full-duplex>
/>>/       status: active
/>>/ em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
/>>/       options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
/>>/       ether 00:11:43:2c:62:03
/>>/       inet 10.10.0.1 netmask 0xffffff00 broadcast 10.10.0.255
/>>/       inet6 fe80::211:43ff:fe2c:6203%em0 prefixlen 64 scopeid 0x2
/>>/       inet 10.13.40.1 netmask 0xffffff00 broadcast 10.13.40.255
/>>/       inet 10.13.41.1 netmask 0xffffff00 broadcast 10.13.41.255
/>>/       inet 10.13.42.1 netmask 0xffffff00 broadcast 10.13.42.255
/>>/       inet 10.13.43.1 netmask 0xffffff00 broadcast 10.13.43.255
/>>/       inet 10.13.44.1 netmask 0xffffff00 broadcast 10.13.44.255
/>>/       inet 10.13.45.1 netmask 0xffffff00 broadcast 10.13.45.255
/>>/       inet 10.13.46.1 netmask 0xffffff00 broadcast 10.13.46.255
/>>/       inet 10.13.47.1 netmask 0xffffff00 broadcast 10.13.47.255
/>>/       media: Ethernet autoselect (100baseTX <full-duplex>)
/>>/       status: active
/>>/ fxp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
/>>/       options=8<VLAN_MTU>
/>>/       ether 00:d0:b7:5d:25:9f
/>>/       inet 10.1.242.1 netmask 0xffffff00 broadcast 10.1.242.255
/>>/       inet6 fe80::2d0:b7ff:fe5d:259f%fxp1 prefixlen 64 scopeid 0x3
/>>/       inet 10.1.243.1 netmask 0xffffff00 broadcast 10.1.243.255
/>>/       media: Ethernet autoselect (100baseTX <full-duplex>)
/>>/       status: active
/>>/
/>>/
/>>/
/>>/ When the problem is occurring, you can't even ping the affected locally
/>>/ assigned addresses from the box itself:
/>>/ # ping 10.10.0.1
/>>/ PING 10.10.0.1 (10.10.0.1): 56 data bytes
/>>/ ping: sendto: Network is unreachable
/>>/ ping: sendto: Network is unreachable
/>>/ ping: sendto: Network is unreachable
/>>/
/>>/ And when trying to ping something on one of the affected attached subnets,
/>>/ you get:
/>>/ # ping 10.10.0.30
/>>/ PING 10.10.0.30 (10.10.0.30): 56 data bytes
/>>/ ping: sendto: Invalid argument
/>>/ ping: sendto: Invalid argument
/>>/
/>>/
/>>/ In the logs, you get a flood of these messages:
/>>/ May 14 02:55:12    kernel: arpresolve: can't allocate route for 10.10.0.1
/>>/ May 14 02:55:12    kernel: arplookup 10.10.0.1 failed: host is not on
/>>/ local network
/>>/ May 14 02:55:12    kernel: arpresolve: can't allocate route for 10.10.0.1
/>>/ May 14 02:55:12    kernel: arplookup 10.10.0.1 failed: host is not on
/>>/ local network
/>>/
/>>/
/>>/ It happens both with the primary IP assigned to the interface, and the
/>>/ aliases assigned, but not all at once. Some of the addresses will continue
/>>/ to work when others are failing. Somehow it thinks IPs that are locally
/>>/ assigned are not on a local network... after a couple minutes, it just
/>>/ starts working again without making any changes or even touching the system.
/>>/
/>>/ If I can provide any additional information, please let me know.
/>>/
/>>/ thanks,
/>>/ Chris
/>>/
//

//The same thing happened to me. FreeBSD 8.0-RELEASE, bge0-bge4, bge0 configured with two subnets.
ARP for primary subnet dissappear randomly (~once a day).

Do you have any resolution for this?

Best Regards,
Alexander Derevyanko
/