ghosthunting: machine freeze 6.2R

Tue May 22 17:23:57 UTC 2007

On Tuesday 22 May 2007 11:41, Volker wrote:
> There's nothing of interest in the logs (I suspect it's too late to
> have any daemon write something to syslogd as the system is then
> dead). As I also think, this is somehow related to network traffic,
> I'll try to monitor it's traffic starting tomorrow. The tech guys at
> the office there have to install a hub first (yes, it's remote to me
> which makes investigation harder).
>
> As I think (well, guesswork) this might be related to GRE traffic
> (the poptop server), I may stop this traffic at the firewall (and
> take the service to another machine).
>
> What's the version of the system you're having trouble with? Is it
> pre-6.2?
>
> I'm wondering if your machine also passes GRE traffic? Does it
> provide pptp services?
>
> I've attached the latest dmesg, can you see any similarities (as it
> still might be a hardware issue)?

Volker,

Yes.  We get nothing in the logs, No kernel panics, and we can not break to 
the kernel debugger.

We are not running an GRE traffic or PPTP Servers.

Our current version is: FreeBSD  6.2-RELEASE FreeBSD 6.2-RELEASE #0

Looks like you are also using the EM network adapter driver.  We are 
suspecting the driver or Network adapters them self.  Our other boxes are 
using xl0 (3com) and are working fine.

I have also attached my dmesg output.  

Roger
-------------- next part --------------
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-RELEASE #0: Wed May  9 17:48:30 UTC 2007
    root@:/usr/obj/usr/src/sys/TINYBSD
WARNING: MPSAFE network stack disabled, expect reduced performance.
ACPI APIC Table: <IntelR AWRDACPI>
Timecounter "i8254" frequency 1193520 Hz quality 0
CPU: Intel(R) Celeron(R) CPU 2.80GHz (2792.85-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf34  Stepping = 4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x441d<SSE3,RSVD2,MON,DS_CPL,CNTX-ID,<b14>>
real memory  = 535691264 (510 MB)
avail memory = 519000064 (494 MB)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
acpi0: <IntelR AWRDACPI> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci1: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection Version - 6.3.9> port 0xc000-0xc01f mem 0xf2000000-0xf201ffff irq 18 at device 1.0 on pci1
em0: Ethernet address: 00:30:48:86:97:62
em0: [GIANT-LOCKED]
pcib2: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pci0: <serial bus, USB> at device 29.0 (no driver attached)
pci0: <serial bus, USB> at device 29.1 (no driver attached)
pci0: <base peripheral> at device 29.4 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 29.5 (no driver attached)
pci0: <serial bus, USB> at device 29.7 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <display, VGA> at device 9.0 (no driver attached)
em1: <Intel(R) PRO/1000 Network Connection Version - 6.3.9> port 0xd100-0xd13f mem 0xf1000000-0xf101ffff irq 19 at device 10.0 on pci3
em1: Ethernet address: 00:30:48:86:97:63
em1: [GIANT-LOCKED]
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 6300ESB SATA150 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f irq 18 at device 31.2 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_tz0: <Thermal Zone> on acpi0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounter "TSC" frequency 2792849472 Hz quality 800
Timecounters tick every 1.000 msec
IP Filter: v4.1.13 initialized.  Default = pass all, Logging = enabled
ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to accept, logging limited to 100 packets/entry by default
ad0: 977MB <SanDisk SDCFH-1024 HDX 3.19> at ata0-master PIO4
ad2: 76319MB <Seagate ST380811AS 3.AAB> at ata1-master SATA150
Trying to mount root from ufs:/dev/ad0s1a
em1: link state changed to DOWN
em0: link state changed to DOWN
bridge0: Ethernet address: 46:e0:af:c9:e6:b7
em0: promiscuous mode enabled
em1: promiscuous mode enabled
em0: link state changed to UP
em1: link state changed to UP
em1: link state changed to DOWN
em1: link state changed to UP
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...1 0 0 done
All buffers synced.
Uptime: 47m5s
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-RELEASE #0: Wed May  9 17:48:30 UTC 2007
    root@:/usr/obj/usr/src/sys/TINYBSD
WARNING: MPSAFE network stack disabled, expect reduced performance.
ACPI APIC Table: <IntelR AWRDACPI>
Timecounter "i8254" frequency 1193516 Hz quality 0
CPU: Intel(R) Celeron(R) CPU 2.80GHz (2792.84-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf34  Stepping = 4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x441d<SSE3,RSVD2,MON,DS_CPL,CNTX-ID,<b14>>
real memory  = 535691264 (510 MB)
avail memory = 519000064 (494 MB)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
acpi0: <IntelR AWRDACPI> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci1: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection Version - 6.3.9> port 0xc000-0xc01f mem 0xf2000000-0xf201ffff irq 18 at device 1.0 on pci1
em0: Ethernet address: 00:30:48:86:97:62
em0: [GIANT-LOCKED]
pcib2: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pci0: <serial bus, USB> at device 29.0 (no driver attached)
pci0: <serial bus, USB> at device 29.1 (no driver attached)
pci0: <base peripheral> at device 29.4 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 29.5 (no driver attached)
pci0: <serial bus, USB> at device 29.7 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <display, VGA> at device 9.0 (no driver attached)
em1: <Intel(R) PRO/1000 Network Connection Version - 6.3.9> port 0xd100-0xd13f mem 0xf1000000-0xf101ffff irq 19 at device 10.0 on pci3
em1: Ethernet address: 00:30:48:86:97:63
em1: [GIANT-LOCKED]
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 6300ESB SATA150 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f irq 18 at device 31.2 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_tz0: <Thermal Zone> on acpi0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounter "TSC" frequency 2792842836 Hz quality 800
Timecounters tick every 1.000 msec
IP Filter: v4.1.13 initialized.  Default = pass all, Logging = enabled
ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to accept, logging limited to 100 packets/entry by default
ad0: 977MB <SanDisk SDCFH-1024 HDX 3.19> at ata0-master PIO4
ad2: 76319MB <Seagate ST380811AS 3.AAB> at ata1-master SATA150
Trying to mount root from ufs:/dev/ad0s1a
em1: link state changed to DOWN
em0: link state changed to DOWN
bridge0: Ethernet address: 7a:c0:e1:a9:84:a1
em0: promiscuous mode enabled
em1: promiscuous mode enabled
em0: link state changed to UP
em1: link state changed to UP
em1: link state changed to DOWN
em1: link state changed to UP
em0: link state changed to DOWN
em1: link state changed to DOWN
em1: link state changed to UP
em0: link state changed to UP
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled
pflog0: promiscuous mode enabled
pflog0: promiscuous mode disabled