Smartd, twe0 and HELP!! ;)

Wed Sep 24 08:42:33 UTC 2008

Hi,

	We lost primary and secondary power at our datacenter, and now we 
are having issues on a server... FreeBSD 5.5 (I'm sorry, its just one of
those that can't be upgraded)

	Its running smartd 5.36 (Again, I can't upgrade software on here.
Its scheduled for replacement, but the system replacing it took a hit and
has locked up twice in 2 hours, so I'm pretty stuffed)

	The dmesg saw :

Sep 24 01:34:05 erda kernel: twe0: <3ware Storage Controller. Driver version 1.50.01.002> port 0xb800-0xb80f mem 0xfe000000-0xfe7fffff irq 24 at device 3.0 on pci3
Sep 24 01:34:05 erda kernel: twe0: AEN: <twe0: port 2: drive not supported>
Sep 24 01:34:05 erda kernel: twe0: 8 ports, Firmware FE7S 1.05.00.050, BIOS BE 7X 1.08.00.046
Sep 24 01:34:05 erda kernel: twed0: <Unit 0, RAID5, Normal> on twe0
Sep 24 01:34:05 erda kernel: twed0: 1430844MB (2930370048 sectors)
Sep 24 01:34:05 erda kernel: Mounting root from ufs:/dev/twed0s1a
Sep 24 01:34:05 erda kernel: kernel dumps on /dev/twed0s1b
Sep 24 01:34:05 erda kernel: swapon: adding /dev/twed0s1b as swap device
Sep 24 01:34:05 erda kernel: /dev/twed0s1a: 1915 files, 33896 used, 472591 free (423 frags, 59021 blocks, 0.1% fragmentation)
Sep 24 01:34:05 erda kernel: /dev/twed0s1g: DEFER FOR BACKGROUND CHECKING
Sep 24 01:34:05 erda kernel: /dev/twed0s1e: DEFER FOR BACKGROUND CHECKING
Sep 24 01:34:05 erda kernel: /dev/twed0s1f: DEFER FOR BACKGROUND CHECKING
Sep 24 01:34:05 erda kernel: /dev/twed0s1d: DEFER FOR BACKGROUND CHECKING

	Smartmon started as :

Sep 24 01:35:27 erda kernel: Starting smartd.
Sep 24 01:35:27 erda smartd[700]: smartd version 5.36 [i386-portbld-freebsd5.3]
Copyright (C) 2002-6 Bruce Allen
Sep 24 01:35:27 erda smartd[700]: Home page is http://smartmontools.sourceforge.
net/
Sep 24 01:35:27 erda smartd[700]: Opened configuration file /usr/local/etc/smart
d.conf
Sep 24 01:35:27 erda smartd[700]: Drive: /dev/twe0 [3ware_disk_00], implied '-a'
 Directive on line 24 of file /usr/local/etc/smartd.conf
Sep 24 01:35:27 erda smartd[700]: Configuration file /usr/local/etc/smartd.conf
parsed.
Sep 24 01:35:27 erda smartd[700]: Device: /dev/twe0 [3ware_disk_00], opened
Sep 24 01:35:27 erda smartd[700]: Device: /dev/twe0 [3ware_disk_00], found in sm
artd database.
Sep 24 01:35:27 erda smartd[700]: Device: /dev/twe0 [3ware_disk_00], is SMART ca
pable. Adding to "monitor" list.
Sep 24 01:35:27 erda smartd[700]: Monitoring 1 ATA and 0 SCSI devices
Sep 24 01:35:28 erda smartd[703]: smartd has fork()ed into background mode. New
PID=703.
Sep 24 01:35:28 erda smartd[703]: file /var/run/smartd.pid written containing PI
D 703

	but later on did :

Sep 24 02:20:16 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:20:57 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:25:09 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:26:12 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:26:41 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:26:48 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:28:49 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:29:35 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:30:11 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:30:43 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:31:18 erda kernel: twe0: AEN: <twe0: port 4: sector repair occurred>
Sep 24 02:31:28 erda kernel: twe0: AEN: <twe0: drive error for unknown unit 4>

Sep 24 02:35:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 02:35:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], not capable
 of SMART self-check
Sep 24 02:35:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 02:35:28 erda smartd[703]: Error SMART Values Read failed: Input/output e
rror
Sep 24 02:35:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], failed to r
ead SMART Attribute Data
Sep 24 02:35:28 erda kernel: Sep 24 02:35:28 erda smartd[703]: Device: /dev/twe0
 [3ware_disk_00], failed to read SMART Attribute Data
Sep 24 02:35:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 02:35:28 erda smartd[703]: Error SMART Error Self-Test Log Read failed: I
nput/output error
Sep 24 02:35:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], Read SMART 
Self Test Log Failed
Sep 24 02:35:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 02:35:28 erda smartd[703]: Error SMART Error Log Read failed: Input/outpu
t error
Sep 24 02:35:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], Read SMARTError Log Failed

	and

Sep 24 03:05:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 03:05:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], not capable
 of SMART self-check
Sep 24 03:05:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 03:05:28 erda smartd[703]: Error SMART Values Read failed: Input/output e
rror
Sep 24 03:05:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], failed to r
ead SMART Attribute Data
Sep 24 03:05:28 erda kernel: Sep 24 03:05:28 erda smartd[703]: Device: /dev/twe0
 [3ware_disk_00], failed to read SMART Attribute Data
Sep 24 03:05:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 03:05:28 erda smartd[703]: Error SMART Error Self-Test Log Read failed: I
nput/output error
Sep 24 03:05:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], Read SMART 
Self Test Log Failed
Sep 24 03:05:28 erda smartd[703]: Command failed, ata.status=(0xc8), ata.command
=(0x50), ata.flags=(0x2c)
Sep 24 03:05:28 erda smartd[703]: Error SMART Error Log Read failed: Input/outpu
t error
Sep 24 03:05:28 erda smartd[703]: Device: /dev/twe0 [3ware_disk_00], Read SMART Error Log Failed

	I contacted the smartd people, the author replied he's not sure whats
wrong. SIGH.... I contacted the twe author and then realized maybe posting here
is a good idea.

	Any ideas/thoughts/suggestions/etc for whats happening? Pointers?

		Thanks, Tuc