kern/123347: bge1: watchdog timeout -- linkstate changed to DOWN

Fri May 2 22:10:04 UTC 2008

>Number:         123347
>Category:       kern
>Synopsis:       bge1: watchdog timeout -- linkstate changed to DOWN
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 02 22:10:03 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Adam
>Release:        FreeBSD 6.3-RELEASE
>Organization:
MHM Telecom
>Environment:
FreeBSD GBRT2.mhm.lv 6.3-RELEASE FreeBSD 6.3-RELEASE #0: Tue Apr  1 00:29:12 UTC 2008     root at GBRT2.mhm.lv:/usr/src/sys/i386/compile/GBRT2  i386
>Description:
This box works as an Internet router and I have two NICs in it, 3com 3C996-SX and 3C996-T, the later is bge1 on which I get this persistent ERROR below, the first one, bge0 is OK!

The result of this error is blackouts and instability of the Internet connections!

May  2 14:09:48 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 14:09:48 GBRT2 kernel: bge1: link state changed to DOWN
May  2 14:09:51 GBRT2 kernel: bge1: link state changed to UP
May  2 14:38:53 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 14:38:53 GBRT2 kernel: bge1: link state changed to DOWN
May  2 14:38:55 GBRT2 kernel: bge1: link state changed to UP
May  2 15:56:56 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 15:56:56 GBRT2 kernel: bge1: link state changed to DOWN
May  2 15:56:58 GBRT2 kernel: bge1: link state changed to UP
May  2 17:04:14 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 17:04:14 GBRT2 kernel: bge1: link state changed to DOWN
May  2 17:04:16 GBRT2 kernel: bge1: link state changed to UP
May  2 17:14:13 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 17:14:13 GBRT2 kernel: bge1: link state changed to DOWN
May  2 17:14:16 GBRT2 kernel: bge1: link state changed to UP
May  2 18:50:36 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 18:50:36 GBRT2 kernel: bge1: link state changed to DOWN
May  2 18:50:38 GBRT2 kernel: bge1: link state changed to UP
May  2 19:03:48 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 19:03:48 GBRT2 kernel: bge1: link state changed to DOWN
May  2 19:03:51 GBRT2 kernel: bge1: link state changed to UP
May  2 20:13:39 GBRT2 kernel: bge1: watchdog timeout -- resetting
May  2 20:13:39 GBRT2 kernel: bge1: link state changed to DOWN
May  2 20:13:41 GBRT2 kernel: bge1: link state changed to UP

Here is the output of dmesg.boot:

Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.3-RELEASE #0: Tue Apr  1 00:29:12 UTC 2008
    root at GBRT2.mhm.lv:/usr/src/sys/i386/compile/GBRT2
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Core(TM)2 Duo CPU     E6850  @ 3.00GHz (3001.18-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x6fb  Stepping = 11
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS$
  Features2=0xe3fd<SSE3,RSVD2,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
  AMD Features=0x20000000<LM>
  AMD Features2=0x1<LAHF>
  Cores per package: 2
real memory  = 2146828288 (2047 MB)
avail memory = 2095779840 (1998 MB)
ACPI APIC Table: <A_M_I_ OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
acpi0: <NEC > on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
cpu1: <ACPI CPU> on acpi0
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci6: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci4: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci4
pci5: <ACPI PCI bus> on pcib3
bge0: <Broadcom BCM5701 B5, ASIC rev. 0x105> mem 0xfeaf0000-0xfeafffff irq 24 at device 1.0 on pci5
bge0: Ethernet address: 00:0a:5e:62:38:1a
bge1: <Broadcom BCM5700 B2, ASIC rev. 0x7102> mem 0xfeae0000-0xfeaeffff irq 28 at device 2.0 on pci5
miibus0: <MII bus> on bge1
brgphy0: <BCM5401 10/100/1000baseTX PHY> on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge1: Ethernet address: 00:04:76:2f:c1:25
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib4
mskc0: <Marvell Yukon 88E8052 Gigabit Ethernet> port 0xd800-0xd8ff mem 0xfe9fc000-0xfe9fffff irq 16 at device 0.0 on pci3
msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
msk0: Ethernet address: 00:1d:60:05:00:7e
miibus1: <MII bus> on msk0
e1000phy0: <Marvell 88E1111 Gigabit PHY> on miibus1
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
mskc0: [FAST]
pcib5: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci2: <ACPI PCI bus> on pcib5
atapci0: <Marvell 88SX6145 UDMA133 controller> port 0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc807,0xc480-0xc483,0xc400-0xc40f mem 0xfe8$
ata2: <ATA channel 0> on atapci0
ata3: <ATA channel 1> on atapci0
pcib6: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib6
pci1: <display, VGA> at device 1.0 (no driver attached)
skc0: <Marvell Gigabit Ethernet> port 0xb800-0xb8ff mem 0xfe7f8000-0xfe7fbfff irq 21 at device 5.0 on pci1
skc0: Marvell Yukon Lite Gigabit Ethernet rev. (0x9)
sk0: <Marvell Semiconductor, Inc. Yukon> on skc0
sk0: Ethernet address: 00:1d:60:05:02:43
miibus2: <MII bus> on sk0
e1000phy1: <Marvell 88E1011 Gigabit PHY> on miibus2
e1000phy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0
ata0: <ATA channel 0> on atapci1
ata1: <ATA channel 1> on atapci1
atapci2: <Intel ICH7 SATA300 controller> port 0xec00-0xec07,0xe880-0xe883,0xe800-0xe807,0xe480-0xe483,0xe400-0xe40f mem 0xfebffc00-$
ata4: <ATA channel 0> on atapci2
ata5: <ATA channel 1> on atapci2
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, rule-based forwarding enabled, default to accept, logging limited to 10 packets/entry by$
ad0: 76319MB <WDC WD800JB-22JJC0 05.01C05> at ata0-master UDMA100
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad0s1a
>How-To-Repeat:
Just install two 3Com PCI NICs 3C996 on the 64-bit PCI-E slots on a router box with the similar hard and software environment. 
Motherboard: Asus P5WDG2 WS Professional 975X P4EE/Pentium D/P4/Celeron 1066FSB LGA775 DDR2 ATX 

CPU: INTEL Core 2 Duo BX80557E6850 E6850 3GHz FSB1333MHz 4M LGA775 
>Fix:
Not yet, and the same problem is being reported with Intel em driver on the 6-STABLE branch

>Release-Note:
>Audit-Trail:
>Unformatted: