em 6.6.6 - watchdog timeout
Goran Lowkrantz
goran.lowkrantz at ismobile.com
Fri Oct 19 22:19:01 PDT 2007
<goran.lowkrantz at ismobile.com> wrote:
> Hi,
>
> After the update of em to 6.6.6 last, I experience watchdog timeouts on a
> server running 6-STABLE.
>
> I have two identical servers with Intel D915GAV boards. Both have Intel
> PRO/1000 PCI-Express network cards.
>
> Server balder:
> em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port
> 0xac00-0xac1f mem 0xff600000-0xff61ffff,0xff620000-0xff63ffff irq 16 at
> device 0.0 on pci5
> em0: Ethernet address: 00:1b:21:00:48:c4
> em0: [FAST]
>
># vmstat -i
> interrupt total rate
> irq1: atkbd0 3 0
> irq4: sio0 2 0
> irq6: fdc0 12 0
> irq14: ata0 68 0
> irq16: em0 uhci3 219828879 450
> irq19: uhci1++ 4287947 8
> irq22: ahc0 232717293 476
> irq23: uhci0 ehci0 1 0
> cpu0: timer 976552804 2000
> Total 1433387009 2935
>
># netstat -i
> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs
> Coll
> em0 1500 <Link#1> 00:1b:21:00:48:c4 209880531 773 206555522
> 84 0
> em0 1500 10.255.253/24 balder 215210996 - 212337968
> - -
> plip0 1500 <Link#2> 0 0 0 0
> 0
> lo0 16384 <Link#3> 12040055 0 12055326 0
> 0
> lo0 16384 fe80:3::1 fe80:3::1 0 - 0 -
> -
> lo0 16384 localhost ::1 6 - 6 -
> -
> lo0 16384 your-net localhost 6249979 - 6249980 -
> -
>
> 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory
> Controller Hub (rev 04)
> 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express
> Root Port (rev 04)
> 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL
> Integrated Graphics Controller (rev 04)
> 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 1 (rev 03)
> 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 2 (rev 03)
> 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 3 (rev 03)
> 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 4 (rev 03)
> 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #1 (rev 03)
> 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #2 (rev 03)
> 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #3 (rev 03)
> 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #4 (rev 03)
> 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB2 EHCI Controller (rev 03)
> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3)
> 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC
> Interface Bridge (rev 03)
> 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) IDE Controller (rev 03)
> 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA
> Controller (rev 03)
> 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> SMBus Controller (rev 03)
> 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
> Controller (Copper) (rev 06)
> 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev
> 01)
>
>
> Server midgard:
> em0: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port
> 0xac00-0xac1f mem 0xff500000-0xff51ffff,0xff520000-0xff53ffff irq 16 at
> device 0.0 on pci5
> em0: Ethernet address: 00:15:17:0e:05:f7
> admglz at midgard> vmstat -i
> interrupt total rate
> irq1: atkbd0 11 0
> irq4: sio0 2142746 0
> irq6: fdc0 14 0
> irq14: ata0 252 0
> irq16: em0+ 666640101 164
> irq19: atapci1+ 7932757 1
> irq22: ahc0 87074425 21
> cpu0: timer 3807810138 937
> Total 4571600444 1125
>
> admglz at midgard> netstat -i
> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs
> Coll
> em0 1500 <Link#1> 00:15:17:0e:05:f7 343771280 0 474609731
> 0 0
> em0 1500 10.255.253/24 midgard 347467842 - 478700485
> - -
> plip0 1500 <Link#2> 0 0 0 0
> 0
> lo0 16384 <Link#3> 16821054 0 16947668 0
> 0
> lo0 16384 fe80:3::1 fe80:3::1 0 - 0 -
> -
> lo0 16384 localhost ::1 2610 - 2610 -
> -
> lo0 16384 your-net localhost 12616879 - 12616879 -
> -
> lo0 16384 10.255.253.12 appsrv1 0 - 0 -
> -
> lo0 16384 10.255.253.10 ca.glz.hidden-pow 0 - 0 -
> -
> lo0 16384 10.255.253.11 test 0 - 0 -
> -
> lo0 16384 10.255.253.13 secure 0 - 0 -
> -
> lo0 16384 10.255.253.18 rscds.hidden-powe 7 - 0 -
> -
>
> midgard# lspci
> 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory
> Controller Hub (rev 04)
> 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express
> Root Port (rev 04)
> 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL
> Integrated Graphics Controller (rev 04)
> 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 1 (rev 03)
> 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 2 (rev 03)
> 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 3 (rev 03)
> 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> PCI Express Port 4 (rev 03)
> 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #1 (rev 03)
> 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #2 (rev 03)
> 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #3 (rev 03)
> 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB UHCI #4 (rev 03)
> 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) USB2 EHCI Controller (rev 03)
> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3)
> 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC
> Interface Bridge (rev 03)
> 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6
> Family) IDE Controller (rev 03)
> 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA
> Controller (rev 03)
> 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family)
> SMBus Controller (rev 03)
> 01:00.0 SCSI storage controller: Triones Technologies, Inc. Unknown
> device 2310 (rev 02)
> 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
> Controller (Copper) (rev 06)
> 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev
> 01)
> 06:02.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host
> Controller (rev 46)
>
>
> When running netstat between servers balder and midgard, server balder
> get watchdog timeouts and resets the connection for a few seconds.
> Oct 19 13:12:47 balder kernel: em0: watchdog timeout -- resetting
> Oct 19 13:12:47 balder kernel: em0: link state changed to DOWN
> Oct 19 13:12:51 balder kernel: em0: link state changed to UP
>
> I have switched the cable between the two servers but get exactly the
> same problem. The switch is a Netgear GS108T with the latest firmware.
>
> The resp. dmesg.boot are attached.
>
> Please let me know if there is any other information I can supply to
> clear this.
>
> Best regards,
> Göran L
>
I have managed to get my performance back in two ways:
- Switching to polling.
- Build a kernel without USB.
So it's the interrupt sharing between the network card and a USB hub that's
the problem.
/glz
More information about the freebsd-stable
mailing list