7.1 hangs, shutdown terminated

Fri Oct 10 09:09:28 UTC 2008

> This could be a periodic job (since you said this happens daily) which
> runs early in the morning (2-3am?) and for some reason isn't finishing
> in a timely manner.  You haven't provided any actual ps -auxwwwwwww
> data, so we can't easily discern if it's a periodic job or something
> amiss on your system (for all we know the system could be compromised).
>   
I wanted to, but since I could not log in...
> I'm also curious what controller your SCSI disks are attached to.  Can
> you provide that information?  
ARECA 1680 ix 12
> dmesg would be useful.  
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.1-PRERELEASE #3: Mon Oct  6 07:50:31 EDT 2008
    gandalf at shopzeus.chello.hu:/usr/obj/usr/src/sys/SHOPZEUS
module_register: module g_journal already exists!
Module g_journal failed to register: 17
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz (2508.72-MHz 
K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x10676  Stepping = 6

Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>

Features2=0xce3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,<b19>>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  Cores per package: 4
usable memory = 8571047936 (8173 MB)
avail memory  = 8260050944 (7877 MB)
ACPI APIC Table: <INTEL  S5000PSL>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
lapic0: Forcing LINT1 to edge trigger
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
acpi0: <INTEL S5000PSL> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on 
acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xca2,0xca3,0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2
pci4: <ACPI PCI bus> on pcib4
em0: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0x2020-0x203f mem 
0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 on pci4
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:15:17:81:99:d0
em1: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0x2000-0x201f mem 
0xb8800000-0xb881ffff,0xb8000000-0xb83fffff irq 19 at device 0.1 on pci4
em1: Using MSI interrupt
em1: [FILTER]
em1: Ethernet address: 00:15:17:81:99:d1
pcib5: <ACPI PCI-PCI bridge> at device 0.3 on pci1
pci5: <ACPI PCI bus> on pcib5
pcib6: <PCI-PCI bridge> at device 3.0 on pci0
pci6: <PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci9: <ACPI PCI bus> on pcib9
pcib10: <PCI-PCI bridge> at device 7.0 on pci0
pci10: <PCI bus> on pcib10
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib11: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci11: <ACPI PCI bus> on pcib11
arcmsr0: <Areca SAS Host Adapter RAID Controller (RAID6 capable)
 > mem 0xb8b00000-0xb8b01fff irq 16 at device 0.0 on pci11
ARECA RAID ADAPTER0: Driver Version 1.20.00.15 2007-10-07
ARECA RAID ADAPTER0: FIRMWARE VERSION V1.45 2008-04-29
arcmsr0: [ITHREAD]
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 
0x3080-0x309f irq 23 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <Intel 631XESB/632XESB/3100 USB controller USB-1> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 
0x3060-0x307f irq 22 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <Intel 631XESB/632XESB/3100 USB controller USB-2> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 
0x3040-0x305f irq 23 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <Intel 631XESB/632XESB/3100 USB controller USB-3> on uhci2
usb2: USB revision 1.0
uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: <Intel 631XESB/632XESB/3100 USB controller USB-4> port 
0x3020-0x303f irq 22 at device 29.3 on pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: <Intel 631XESB/632XESB/3100 USB controller USB-4> on uhci3
usb3: USB revision 1.0
uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xb8c00400-0xb8c007ff irq 
23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <Intel 63XXESB USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
uhub4: 8 ports with 8 removable, self powered
pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci12: <ACPI PCI bus> on pcib12
vgapci0: <VGA-compatible display> port 0x1000-0x10ff mem 
0xb0000000-0xb7ffffff,0xb8a00000-0xb8a0ffff irq 17 at device 12.0 on pci12
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 UDMA100 controller> port 
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30b0-0x30bf irq 20 at device 31.1 
on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
atapci1: <Intel 63XXESB2 SATA300 controller> port 
0x30c8-0x30cf,0x30e4-0x30e7,0x30c0-0x30c7,0x30e0-0x30e3,0x30a0-0x30af 
mem 0xb8c00000-0xb8c003ff irq 20 at device 31.2 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model IntelliMouse Explorer, device ID 4
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on 
acpi0
sio0: type 16550A
sio0: [FILTER]
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
sio1: [FILTER]
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
cpu4: <ACPI CPU> on acpi0
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
cpu5: <ACPI CPU> on acpi0
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
cpu6: <ACPI CPU> on acpi0
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
cpu7: <ACPI CPU> on acpi0
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
orm0: <ISA Option ROMs> at iomem 
0xc0000-0xc8fff,0xca800-0xcb7ff,0xcb800-0xcc7ff on isa0
ppc0: cannot reserve I/O port range
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
ipfw2 initialized, divert loadable, nat loadable, rule-based forwarding 
disabled, default to deny, logging limited to 100 packets/entry by default
The GEOM class JOURNAL is already loaded.
acd0: DVDR <Optiarc DVD RW AD-5200A/1.04> at ata0-master UDMA66
Waiting 5 seconds for SCSI devices to settle
(probe16:arcmsr0:0:16:0): inquiry data fails comparison at DV1 step
da0 at arcmsr0 bus 0 target 0 lun 0
da0: <Areca ARC-1680-VOL#000 R001> Fixed Direct Access SCSI-5 device
da0: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit)
da0: 1220702MB (2499997696 512 byte sectors: 255H 63S/T 155617C)
da1 at arcmsr0 bus 0 target 0 lun 1
da1: <Areca ARC-1680-VOL#001 R001> Fixed Direct Access SCSI-5 device
da1: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit)
da1: 476836MB (976561664 512 byte sectors: 255H 63S/T 60788C)
lapic1: Forcing LINT1 to edge trigger
SMP: AP CPU #1 Launched!
lapic3: Forcing LINT1 to edge trigger
SMP: AP CPU #3 Launched!
lapic2: Forcing LINT1 to edge trigger
SMP: AP CPU #2 Launched!
lapic4: Forcing LINT1 to edge trigger
SMP: AP CPU #4 Launched!
lapic7: Forcing LINT1 to edge trigger
SMP: AP CPU #7 Launched!
lapic5: Forcing LINT1 to edge trigger
SMP: AP CPU #5 Launched!
lapic6: Forcing LINT1 to edge trigger
SMP: AP CPU #6 Launched!
GEOM_JOURNAL: Journal 2122510179: da0s1e contains data.
GEOM_JOURNAL: Journal 2828859188: da0s1f contains data.
GEOM_JOURNAL: Journal 2122510179: da0s1g contains journal.
GEOM_JOURNAL: Journal da0s1e clean.
GEOM_JOURNAL: Journal 2828859188: da0s1h contains journal.
GEOM_JOURNAL: Journal da0s1f clean.
GEOM_JOURNAL: Journal 239656790: da0s2b contains journal.
GEOM_JOURNAL: Journal 239656790: da0s2d contains data.
GEOM_JOURNAL: Journal da0s2d clean.
GEOM_JOURNAL: Journal 623269810: da0s2e contains data.
GEOM_JOURNAL: Journal 623269810: da0s2f contains journal.
GEOM_JOURNAL: Journal da0s2e clean.
WARNING: Expected rawoffset 0, found 830994255
GEOM_JOURNAL: Journal 3587436884: da1s1b contains journal.
GEOM_JOURNAL: Journal 3587436884: da1s1d contains data.
GEOM_JOURNAL: Journal da1s1d clean.
WARNING: Expected rawoffset 0, found 63
Trying to mount root from ufs:/dev/da0s1a
em1: link state changed to UP
em0: link state changed to UP

>> My vote is on a hardware problem.  The watchdog timeout you see
>> indicates a portion of the system is locking up hard.  The sig 11 would
>> indicate a sudden segfault, which if unexpected, often indicates bad
>> memory or motherboard.
>>
>> I would recommend you start down the hardware path.  Replace the RAM and
>> the mainboard, and see what happens.
>>     
:-(

Thanks.

   Laszlo