FreeBSD 9.1-RELEASE + bge0 == watchdog timeout

[Eugene M. Zheganin]

Hi.
Hi.

On 06.03.2013 12:26, YongHyeon PYUN wrote:
> If you were using latest stable/8, the result would be same on
> CURRENT.
> How frequently do you see the watchdog timeouts? Is there way to
> reproduce it?
> Would you show me the output of dmesg (bge(4) and brgphy(4) only)
> and "pciconf -lcbv"?
I upgraded one om my routers 2 days ago to 8.3-STABLE, and got today a
freeze. Uptime was less than a day.
I have like dozens of these IBM system x3250, all of them run various
8.2-STABLE's, that's why I worry that much. I don't know if this is
triggered by some of my actions. These routers run gre/ipsec, dirrerent
routing stuff (quagga, bird), proxies and pf. In 2011/early 2012 I saw
similar watchdog issues on these machines, and I disabled the tso on
them. I don't know whether this is a coincidence or it really helps, but
after that I didn't see these watchdog issues until today.

I've also discovered that this particular server is running some old
bioses/firmwares including the fact that it misses some NetXtreme
updates available from IBM. Would applying such updates resolve the
situation ?

I am ok with that fact that I cannot run ipmi/sol on these machines, but
it would be nice if this watchdog issue could be somehow resolved.
Furthermore, I have some spare machines that I can provide full access
to, including ipkvm stuff. Since the machine is only partially freezing,
I cannot even rely on the ichwd and watchdogd to reboot it.

pciconf (there's two controllers in this server, I use the first, but
anyway):

bge0 at pci0:2:0:0:        class=0x020000 card=0x03781014 chip=0x165a14e4
rev=0x00 hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'Broadcom NetXtreme BCM5722 Gigabit (94309)'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 64, base 0xe8200000, size 65536, enabled
    cap 01[48] = powerspec 3  supports D0 D3  current D0
    cap 03[50] = VPD
    cap 09[58] = vendor (length 120)
    cap 05[e8] = MSI supports 1 message, 64 bit enabled with 1 message
    cap 10[d0] = PCI-Express 1 endpoint max data 128(128) link x1(x1)
                 speed 2.5(2.5)
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 2 corrected
ecap 0002[13c] = VC 1 max VC0
ecap 0003[160] = Serial 1 001a64fffe21962d
ecap 0004[16c] = Power Budgeting 1
bge1 at pci0:3:1:0:        class=0x020000 card=0x026f1014 chip=0x16c714e4
rev=0x10 hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'BCM5703A3 NetXtreme Gigabit Ethernet'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 64, base 0xe8400000, size 65536, enabled
    cap 07[40] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split
transaction
    cap 01[48] = powerspec 2  supports D0 D3  current D0
    cap 03[50] = VPD
    cap 05[58] = MSI supports 8 messages, 64 bit

dmesg:

bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x00a200> mem 0xe8200000-0xe820ffff irq 16 at device 0.0 on pci2
bge0: CHIP ID 0x0000a200; ASIC REV 0x0a; CHIP REV 0xa2; PCI-E
miibus0: <MII bus> on bge0
bge0: Ethernet address: 00:1a:64:21:96:2d
bge0: [FILTER]
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x001100> mem 0xe8400000-0xe840ffff irq 21 at device 1.0 on pci3
bge1: CHIP ID 0x00001100; ASIC REV 0x01; CHIP REV 0x11; PCI on PCI-X 33
MHz; 32bit
miibus1: <MII bus> on bge1
bge1: Ethernet address: 00:1a:64:21:96:2e
bge1: [ITHREAD]
[emz at omega:~]# cat /var/run/dmesg.boot | egrep 'bge|brg'
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x00a200> mem 0xe8200000-0xe820ffff irq 16 at device 0.0 on pci2
bge0: CHIP ID 0x0000a200; ASIC REV 0x0a; CHIP REV 0xa2; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5722 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Ethernet address: 00:1a:64:21:96:2d
bge0: [FILTER]
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x001100> mem 0xe8400000-0xe840ffff irq 21 at device 1.0 on pci3
bge1: CHIP ID 0x00001100; ASIC REV 0x01; CHIP REV 0x11; PCI on PCI-X 33
MHz; 32bit
miibus1: <MII bus> on bge1
brgphy1: <BCM5703 10/100/1000baseTX PHY> PHY 1 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Ethernet address: 00:1a:64:21:96:2e
bge1: [ITHREAD]


Thanks.
Eugene.