ARP problem with 6.2-STABLE Intel PRO/1000 NIC, latest em driver

[Jack Vogel]

On 3/4/07, Mark Costlow <cheeks at swcp.com> wrote:
> The Machine:
>
> I have a dual Xeon 5130 machine, Supermicro motherboard, with
> the 82563EB NIC.  From dmesg:
>
> CPU: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz (2000.08-MHz 686-class CPU)
> cpu0: <ACPI CPU> on acpi0
> em0: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port 0x2000-0x201f mem 0xda000000-0xda01ffff irq 18 at device 0.0 on pci4
>
> The machine has 4G RAM and a 3ware 9000 series RAID controller with 2 drives.
>
> pciconf -l says:
>
> em0 at pci4:0:0:   class=0x020000 card=0x000015d9 chip=0x10968086 rev=0x01 hdr=0x00
> em1 at pci4:0:1:   class=0x020000 card=0x000015d9 chip=0x10968086 rev=0x01 hdr=0x00
>
>
> The symptom:
>
> The machine boots OK, but can only intermittently make netork connections.
> Eventually determined that it seems to only see a few ARP packets, so
> it's falling out of other machines' ARP tables, and is often unable to
> see the replies to its own ARP requests.  It does see SOME ARPs
> though.  When it is able to communicate with another machine, it
> does not appear to drop any packets between them (e.g. I scp'd a 500M file
> at 300Mbps to this machine).
>
> When I run "tcpdump -n arp" I see a few ARPs, but not many.  In a 1-minute
> period, I saw 3 ARP who-has/reply packets.  On a different machine on
> the same ethernet switch, I saw 225 who-has/reply packets in the same
> 1-minute period.
>
> I've tried different cables, and a different switch.  I started with
> 6.2-RELEASE, and then went to 6.2-STABLE on 3/3/07 to get the latest
> em driver fixes.  I've used SMP and GENERIC kernels.  I get the same
> results in all cases.
>
> There are no firewall rules installed.
>
> I plugged in a USB ethernet adapter (realtek), and it works straight away.
> "tcpdump -n arp" sees the same noise as other machines on that LAN.
>
> I read through the recent threads on the em driver, but didn't see any
> reported symptoms like this.  Has anyone seen anything like this?  Got
> any hints for me?  Am I doing something stupid?  Did I leave out any
> useful information about my configuration?

These are one of our latest NICs, I have had no trouble with these
but I'm used to using them on an Intel design, not SuperMicro.

First question, do you get the same behavior on both ports?
My first guess is that this is a BIOS/management problem.

Double check SM website and see if there's any support updates
to firmware for the system.

I will check at work tomorrow morning to see if anyone else has
heard of this.

Cheers,

Jack