[Bug 211872] IPv6 UDP traffic sometimes sent using wrong mac address

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211872

            Bug ID: 211872
           Summary: IPv6 UDP traffic sometimes sent using wrong mac
                    address
           Product: Base System
           Version: 11.0-RC1
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: mandrews at bit0.com
                CC: freebsd-amd64 at FreeBSD.org
                CC: freebsd-amd64 at FreeBSD.org

Outbound IPv6 UDP traffic sometimes appears to put packets on the wire with the
destination MAC address of a completely different system.  "ndp -a" shows
correct MAC addresses.

Example:

Two nameservers, trying to query each other, on IPv6 private addresses
(fc00::/10)

fdfa::fafa:d53a, mac 00:25:90:57:21:b3
fdfa::fafa:d53b, mac 00:25:90:38:6f:fa

These are lagg0 interfaces aggregating two Intel em nics.

On each machine, run:  tcpdump -e -n net fdfa::/16 and port 53

On machine 2, run "host -t a www.fark.com fdfa::fafa:d53a" and it will timeout,
with its own tcpdump showing:
12:46:22.835604 00:25:90:38:6f:fa > 00:25:90:57:21:b3, ethertype IPv6 (0x86dd),
length 92: fdfa::fafa:d53b.15484 > fdfa::fafa:d53a.53: 47157+ A? www.fark.com.
(30)
12:46:27.836716 00:25:90:38:6f:fa > 00:25:90:57:21:b3, ethertype IPv6 (0x86dd),
length 92: fdfa::fafa:d53b.39323 > fdfa::fafa:d53a.53: 47157+ A? www.fark.com.
(30)

and machine 1's tcpdump showing a wrong MAC address used in the replies:
12:46:22.835118 00:25:90:38:6f:fa > 00:25:90:57:21:b3, ethertype IPv6 (0x86dd),
length 92: fdfa::fafa:d53b.15484 > fdfa::fafa:d53a.53: 47157+ A? www.fark.com.
(30)
12:46:22.835272 00:25:90:57:21:b3 > 00:25:90:23:be:bc, ethertype IPv6 (0x86dd),
length 232: fdfa::fafa:d53a.53 > fdfa::fafa:d53b.15484: 47157* 1/2/4 A
64.191.171.200 (170)
12:46:27.836186 00:25:90:38:6f:fa > 00:25:90:57:21:b3, ethertype IPv6 (0x86dd),
length 92: fdfa::fafa:d53b.39323 > fdfa::fafa:d53a.53: 47157+ A? www.fark.com.
(30)
12:46:27.836340 00:25:90:57:21:b3 > 00:25:90:23:be:bc, ethertype IPv6 (0x86dd),
length 232: fdfa::fafa:d53a.53 > fdfa::fafa:d53b.39323: 47157* 1/2/4 A
64.191.171.200 (170)

00:50:90:23:be:bc is a totally different 3rd machine.  That's.... weird. :)

Add the -T flag to the "host" command to force TCP, and it works.

Flip the queries around so that machine 1 queries machine 2, and it works.

I'm seeing similar issues with other similar machines (and similar hardware,
all Supermicro machines with Intel em nics of varying vintage) that have been
updated to 11.0-RC1.  All ware fine on 10.3-RELEASE.  The only em-specific
tweak I've got is "-tso", but turning tso back on doesn't change behavior any.

Another weird and possibly related issue: "ndp -c" fails on machine 1 -- it
deletes one or two entries and then stops with "ndp: writing to routing socket:
Operation not permitted".  On machine 2, "ndp -c" completes with no problems.

I haven't yet tried dropping the lagg interface and using the em interfaces
directly.

-- 
You are receiving this mail because:
You are on the CC list for the bug.