Adaptec 3210S, 4.9-STABLE, corruption when disk fails

Uwe Doering gemini at
Mon Feb 28 23:27:12 GMT 2005

Don Bowman wrote:
> I have a machine running:
> $ uname -a
> FreeBSD 4.9-STABLE FreeBSD 4.9-STABLE #0:
> Fri Mar 19 10:39:07 EST 2004
> user at  i386
> It has an adaptec 3210S raid controller running a single raid-5, and
> runs postgresql 7.4.6 as its primary application.
> 3 times now I have had a drive fail, and have had corrupted files in the
> postgresql cluster @ the same time.
> The time is too closely correlated to be a coincidence. It passes fsck @
> the time that I got to it a couple of hours later, and the filesystem
> seems to be ok (with a failed drive, the raid in 'degrade' mode).
> It appears that the drive failure and the postgresql failure occur @
> exactly the same time (monitoring with nagios, within 1hr accuracy). It
> would appear that for some file(s) bad data was returned.
> Does anyone have any suggestions?

In my experience, in a situation like this RAID controllers can block 
the system for up to a couple of minutes, trying to revive a failed disk 
drive by sending it bus reset commands and the like, until they 
eventually give up and drop into degraded mode.  With sufficiently 
patient applications this is no problem, but if a program runs into 
internal timeouts during this period of time bad things can happen.

My point is that while the disk controller may trigger the problem the 
instance that actually corrupts the database might be PostgreSQL itself. 
  Of course, I'm aware that it's going to be quite hard to tell for sure 
who the culprit is.

Uwe Doering         |  EscapeBox - Managed On-Demand UNIX Servers
gemini at  |

More information about the freebsd-stable mailing list