em 6.6.6 - watchdog timeout

[Jack Vogel]

On 10/21/07, Mike Tancsa <mike at sentex.net> wrote:
> At 12:10 AM 10/21/2007, Mike Andrews wrote:
>
> >I haven't tried the 6.6.6 driver on mine yet, though, so this could
> >be something totally different.  I was going to bump one of them
> >from RELENG_6 to RELENG_7 as a test soon.
>
> I see this problem running RELENG_6, which has the 6.6.6 driver. I
> forget the exact supermicro model #
>
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz (2402.50-MHz
> 686-class CPU)
>    Origin = "GenuineIntel"  Id = 0x6f6  Stepping = 6
>    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>    Features2=0xe3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
>    AMD Features=0x20100000<NX,LM>
>    AMD Features2=0x1<LAHF>
>    Cores per package: 2
> real memory  = 2144329728 (2044 MB)
> avail memory = 2092859392 (1995 MB)
> ACPI APIC Table: <INTEL  S3000AHX>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>   cpu0 (BSP): APIC ID:  0
>   cpu1 (AP): APIC ID:  1
> ioapic0: Changing APIC ID to 5
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 30-53 on motherboard
> kbd1 at kbdmux0
> acpi0: <INTEL S3000AHX> on motherboard
> em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port
> 0x3020-0x303f mem 0x88260000-0x8827ffff,0x88240000-0x8825ffff irq 16
> at device 0.0 on pci1
> em0: Ethernet address: 00:15:17:12:f6:04
> em0: [FAST]
> em1: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port
> 0x3000-0x301f mem 0x88220000-0x8823ffff,0x88200000-0x8821ffff irq 17
> at device 0.1 on pci1
> em1: Ethernet address: 00:15:17:12:f6:05
> em1: [FAST]
> em2: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port
> 0x2000-0x201f mem 0x88180000-0x8819ffff,0x88100000-0x8817ffff irq 17
> at device 0.0 on pci5
> em2: Ethernet address: 00:15:17:29:6f:ef
> em2: [FAST]
> em3: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port
> 0x1100-0x113f mem 0x88020000-0x8803ffff,0x88000000-0x8801ffff irq 17
> at device 5.0 on pci6
> em3: Ethernet address: 00:15:17:29:6f:f0
> em3: [FAST]
>
> I already ran the dos util to "fix" the eeprom, but no difference.

I would like you all to try using MSI interrupts, watchdogs don't happen
when I do this. If you have hardware that has a system issue with MSI
then ignore this, but these SuperMicros systems should be fine.

First, you must enable it on the system:

    sysctl hw.pci.enable_msi=1

Then you must reload the driver. If you use em static in the kernel you
will have to change the loader.conf to enable msi on boot.

I am going to add a display that will tell you when an adapter uses MSI or MSI/X
next time I check in code.

Not only does this solve my watchdog problems, I also find on the UDP_STREAM
test of netperf that I get better performance when using MSI.

Let me know how it works if you try this.

Cheers,

Jack