em0 issues with 4.10 + SMP

[Ross Kramer]

On Fri, 2005-01-14 at 13:11 -0500, John Baldwin wrote:
> On Friday 14 January 2005 12:57 pm, Ross Kramer wrote:
> > Is there any known issues and solutions regarding this? My system specs
> > are as follows:
> >
> > Supermicro Superserver 6013A-T
> > Onboard Intel Pro 1000 (em) + Intel Pro 100 (fxp)
> > Dual 3.2 Ghz Xeons
> > 3ware 8006 S/ATA controller card with two 250gig drives in a mirrored
> > config
> >
> > When I try to rebuild the 4.10 kernel for SMP, the system boots fine,
> > but it looks like em0 is continuously resetting itself, repeating:
> > "em0: Link is up 100 Mbps Full Duplex"
> > over and over again. I've tried with the driver version that came with
> > 4.10 (1.7.25) and I tried upgrading to Intel's latest (1.7.45), both
> > with the same result. It boots and runs fine with no problems when
> > booted with the generic kernel. I've tried both the em interface and the
> > fxp interface with SMP, and they both do the interface reset dance. I'am
> > about at the end of my rope here, anybody have an idea?
> >
> > My boot messages:
> > Jan 14 11:28:43 fbsdweb02 /kernel: Copyright (c) 1992-2004 The FreeBSD
> > Project.
> > Jan 14 11:28:43 fbsdweb02 /kernel: Copyright (c) 1979, 1980, 1983, 1986,
> > 1988, 1989, 1991, 1992, 1993, 1994
> > Jan 14 11:28:43 fbsdweb02 /kernel: The Regents of the University of
> > California. All rights reserved.
> > Jan 14 11:28:43 fbsdweb02 /kernel: FreeBSD 4.10-RELEASE #0: Fri Jan 14
> > 11:23:18 EST 2005
> > Jan 14 11:28:43 fbsdweb02 /kernel:
> > root at fbsdweb02.lab.compete.com:/usr/src/sys/compile/CMPKERNEL
> > Jan 14 11:28:43 fbsdweb02 /kernel: Timecounter "i8254"  frequency
> > 1193182 Hz
> > Jan 14 11:28:43 fbsdweb02 /kernel: CPU: Intel(R) Xeon(TM) CPU 3.20GHz
> > (3200.13-MHz 686-class CPU)
> > Jan 14 11:28:43 fbsdweb02 /kernel: Origin = "GenuineIntel"  Id = 0xf25
> > Stepping = 5
> > Jan 14 11:28:43 fbsdweb02 /kernel:
> > Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MC
> >A,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Jan 14
> > 11:28:43 fbsdweb02 /kernel: real memory  = 1073741824 (1048576K bytes)
> > Jan 14 11:28:43 fbsdweb02 /kernel: avail memory = 1041833984 (1017416K
> > bytes)
> > Jan 14 11:28:43 fbsdweb02 /kernel: Programming 24 pins in IOAPIC #0
> > Jan 14 11:28:43 fbsdweb02 /kernel: IOAPIC #0 intpin 2 -> irq 0
> > Jan 14 11:28:43 fbsdweb02 /kernel: IOAPIC #0 intpin 16 -> irq 11
> > Jan 14 11:28:43 fbsdweb02 /kernel: IOAPIC #0 intpin 17 -> irq 10
> > Jan 14 11:28:43 fbsdweb02 /kernel: IOAPIC #0 intpin 19 -> irq 5
> > Jan 14 11:28:43 fbsdweb02 /kernel: FreeBSD/SMP: Multiprocessor
> > motherboard: 2 CPUs
> > Jan 14 11:28:43 fbsdweb02 /kernel: cpu0 (BSP): apic id:  0, version:
> > 0x00050014, at 0xfee00000
> > Jan 14 11:28:43 fbsdweb02 /kernel: cpu1 (AP):  apic id:  6, version:
> > 0x00050014, at 0xfee00000
> > Jan 14 11:28:43 fbsdweb02 /kernel: io0 (APIC): apic id:  7, version:
> > 0x00178020, at 0xfec00000
> > Jan 14 11:28:43 fbsdweb02 /kernel: Preloaded elf kernel "kernel" at
> > 0xc0341000.
> > Jan 14 11:28:43 fbsdweb02 /kernel: Warning: Pentium 4 CPU: PSE disabled
> > Jan 14 11:28:43 fbsdweb02 /kernel: Pentium Pro MTRR support enabled
> > Jan 14 11:28:43 fbsdweb02 /kernel: md0: Malloc disk
> > Jan 14 11:28:43 fbsdweb02 /kernel: Using $PIR table, 10 entries at
> > 0xc00f4720
> > Jan 14 11:28:43 fbsdweb02 /kernel: npx0: <math processor> on motherboard
> > Jan 14 11:28:43 fbsdweb02 /kernel: npx0: INT 16 interface
> > Jan 14 11:28:43 fbsdweb02 /kernel: pcib0: <Host to PCI bridge> on
> > motherboard
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <PCI bus> on pcib0
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <UHCI USB controller> at 29.0
> > irq 11
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <UHCI USB controller> at 29.1
> > irq 5
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <UHCI USB controller> at 29.2
> > irq 11
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <USB controller> at 29.7 irq 5
> > Jan 14 11:28:43 fbsdweb02 /kernel: pcib1: <Intel 82801BA/BAM (ICH2) Hub
> > to PCI bridge> at device 30.0 on pci0
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci1: <PCI bus> on pcib1
> > Jan 14 11:28:43 fbsdweb02 /kernel: twe0: <3ware Storage Controller
> > driver ver. 1.40.01.001> port 0xbc00-0xbc0f mem
> > 0xfe000000-0xfe7fffff,0xfeafec00-0xfeafec0f irq 5 at device
> > 1.0 on pci1
> > Jan 14 11:28:43 fbsdweb02 /kernel: twe0: 2 ports, Firmware FE8S
> > 1.05.00.068, BIOS BE7X 1.08.00.048
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci1: <ATI Mach64-GR graphics
> > accelerator> at 4.0 irq 10
> > Jan 14 11:28:43 fbsdweb02 /kernel: fxp0: <Intel 82551 Pro/100 Ethernet>
> > port 0xb400-0xb43f mem 0xfeaa0000-0xfeabffff,0xfeafd000-0xfeafdfff irq 5
> > at device 7.0 on pci1
> > Jan 14 11:28:43 fbsdweb02 /kernel: fxp0: Ethernet address
> > 00:30:48:73:6e:a2
> > Jan 14 11:28:43 fbsdweb02 /kernel: inphy0: <i82555 10/100 media
> > interface> on miibus0
> > Jan 14 11:28:43 fbsdweb02 /kernel: inphy0:  10baseT, 10baseT-FDX,
> > 100baseTX, 100baseTX-FDX, auto
> > Jan 14 11:28:43 fbsdweb02 /kernel: em0: <Intel(R) PRO/1000 Network
> > Connection, Version - 1.7.42> port 0xb000-0xb03f mem
> > 0xfea60000-0xfea7ffff irq 5 at device 8.0 on pci1
> > Jan 14 11:28:43 fbsdweb02 /kernel: em0:  Speed:N/A  Duplex:N/A
> > Jan 14 11:28:43 fbsdweb02 /kernel: isab0: <PCI to ISA bridge
> > (vendor=8086 device=24d0)> at device 31.0 on pci0
> > Jan 14 11:28:43 fbsdweb02 /kernel: isa0: <ISA bus> on isab0
> > Jan 14 11:28:43 fbsdweb02 /kernel: atapci0: <Intel ICH5 ATA100
> > controller> port 0xfc00-0xfc0f,0-0x3,0-0x7,0-0x3,0-0x7 irq 0 at device
> > 31.1 on pci0
> > Jan 14 11:28:43 fbsdweb02 /kernel: ata0: at 0x1f0 irq 14 on atapci0
> > Jan 14 11:28:43 fbsdweb02 /kernel: ata1: at 0x170 irq 15 on atapci0
> > Jan 14 11:28:43 fbsdweb02 /kernel: pci0: <unknown card> (vendor=0x8086,
> > dev=0x24d3) at 31.3 irq 10
> > Jan 14 11:28:43 fbsdweb02 /kernel: orm0: <Option ROMs> at iomem
> > 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
> > Jan 14 11:28:43 fbsdweb02 /kernel: pmtimer0 on isa0
> > Jan 14 11:28:43 fbsdweb02 /kernel: fdc0: <NEC 72065B or clone> at port
> > 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
> > Jan 14 11:28:43 fbsdweb02 /kernel: fdc0: FIFO enabled, 8 bytes threshold
> > Jan 14 11:28:44 fbsdweb02 /kernel: fd0: <1440-KB 3.5" drive> on fdc0
> > drive 0
> > Jan 14 11:28:44 fbsdweb02 /kernel: atkbdc0: <Keyboard controller
> > (i8042)> at port 0x60,0x64 on isa0
> > Jan 14 11:28:44 fbsdweb02 /kernel: atkbd0: <AT Keyboard> flags 0x1 irq 1
> > on atkbdc0
> > Jan 14 11:28:44 fbsdweb02 /kernel: kbd0 at atkbd0
> > Jan 14 11:28:44 fbsdweb02 /kernel: vga0: <Generic ISA VGA> at port
> > 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> > Jan 14 11:28:44 fbsdweb02 /kernel: sc0: <System console> at flags 0x100
> > on isa0
> > Jan 14 11:28:44 fbsdweb02 /kernel: sc0: VGA <16 virtual consoles,
> > flags=0x300>
> > Jan 14 11:28:44 fbsdweb02 /kernel: sio0 at port 0x3f8-0x3ff irq 4 flags
> > 0x10 on isa0
> > Jan 14 11:28:44 fbsdweb02 /kernel: sio0: type 16550A
> > Jan 14 11:28:44 fbsdweb02 /kernel: sio1 at port 0x2f8-0x2ff irq 3 on
> > isa0
> > Jan 14 11:28:44 fbsdweb02 /kernel: sio1: type 16550A
> > Jan 14 11:28:44 fbsdweb02 /kernel: APIC_IO: Testing 8254 interrupt
> > delivery
> > Jan 14 11:28:44 fbsdweb02 /kernel: APIC_IO: routing 8254 via IOAPIC #0
> > intpin 2
> > Jan 14 11:28:44 fbsdweb02 /kernel: SMP: AP CPU #1 Launched!
> > Jan 14 11:28:44 fbsdweb02 /kernel: acd0: CDROM <CD-224E> at ata1-master
> > PIO4
> > Jan 14 11:28:44 fbsdweb02 /kernel: twed0: <Unit 0, TwinStor, Normal> on
> > twe0
> > Jan 14 11:28:44 fbsdweb02 /kernel: twed0: 238474MB (488395120 sectors)
> > Jan 14 11:28:44 fbsdweb02 /kernel: em0: Link is up 100 Mbps Full Duplex
> > Jan 14 11:28:44 fbsdweb02 /kernel: Mounting root from ufs:/dev/twed0s1a
> > Jan 14 11:28:44 fbsdweb02 /kernel: em0: Link is up 100 Mbps Full Duplex
> > Jan 14 11:29:20 fbsdweb02 last message repeated 6 times
> > Jan 14 11:30:40 fbsdweb02 last message repeated 2 times
> > Jan 14 11:32:41 fbsdweb02 last message repeated 6 times
> 
> Try adding 'device uhci' and 'device usb' to your kernel so that the USB 
> controller (which shares the same IRQ) will probe.  It may be that the USB 
> controller is causing an interrupt storm of sorts, though on 4.x that usually 
> results in a hard hang.
> 

I tried enabling that in the config, rebuilt the kernel, installed it,
and rebooted. em0 did the same thing again, but its ping times to the
local network were in the 8000ms range, when it wasn't returning "Host
down"... Any other suggestions?
____________________________
# Ross Kramer              #
# Systems Administrator    #
# Compete, Inc.            #
# rkramer at compete.com      #
#__________________________#

Random quote:
 
"If I were married to you, I'd put poison in your coffee." - Lady Astor
(to Winston Churchill) "If you were my wife, I'd drink it." - Winston
Churchill, in reply