ARP problem with 6.2-STABLE Intel PRO/1000 NIC, latest em driver

[Jack Vogel]

On 3/4/07, Duane Whitty <duane at dwlabs.ca> wrote:
> On Sun, Mar 04, 2007 at 11:30:07PM -0700, Mark Costlow wrote:
> > The Machine:
> >
> > I have a dual Xeon 5130 machine, Supermicro motherboard, with
> > the 82563EB NIC.  From dmesg:
> >
> > CPU: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz (2000.08-MHz 686-class CPU)
> > cpu0: <ACPI CPU> on acpi0
> > em0: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port 0x2000-0x201f mem 0xda000000-0xda01ffff irq 18 at device 0.0 on pci4
> >
> > The machine has 4G RAM and a 3ware 9000 series RAID controller with 2 drives.
> >
> > pciconf -l says:
> >
> > em0 at pci4:0:0:   class=0x020000 card=0x000015d9 chip=0x10968086 rev=0x01 hdr=0x00
> > em1 at pci4:0:1:   class=0x020000 card=0x000015d9 chip=0x10968086 rev=0x01 hdr=0x00
> >
> >
> > The symptom:
> >
> > The machine boots OK, but can only intermittently make netork connections.
> > Eventually determined that it seems to only see a few ARP packets, so
> > it's falling out of other machines' ARP tables, and is often unable to
> > see the replies to its own ARP requests.  It does see SOME ARPs
> > though.  When it is able to communicate with another machine, it
> > does not appear to drop any packets between them (e.g. I scp'd a 500M file
> > at 300Mbps to this machine).
> >
> > When I run "tcpdump -n arp" I see a few ARPs, but not many.  In a 1-minute
> > period, I saw 3 ARP who-has/reply packets.  On a different machine on
> > the same ethernet switch, I saw 225 who-has/reply packets in the same
> > 1-minute period.
> >
> > I've tried different cables, and a different switch.  I started with
> > 6.2-RELEASE, and then went to 6.2-STABLE on 3/3/07 to get the latest
> > em driver fixes.  I've used SMP and GENERIC kernels.  I get the same
> > results in all cases.
> >
> > There are no firewall rules installed.
> >
> > I plugged in a USB ethernet adapter (realtek), and it works straight away.
> > "tcpdump -n arp" sees the same noise as other machines on that LAN.
> >
>
> Sounds like it could be bad hardware.  Can you swap nics?

No he can't these are LOMs (on the motherboard).

> > I read through the recent threads on the em driver, but didn't see any
> > reported symptoms like this.  Has anyone seen anything like this?  Got
> > any hints for me?  Am I doing something stupid?  Did I leave out any
> > useful information about my configuration?
> >
>
> Maybe more of your dmesg might help as it could show interrrupt issues
> that perhaps others could help diagnose

Yes, agreed, this might be revealing.

Jack