Disk failure in danish part of ftp.freebsd.org

[Jan Chrillesen]

(I'm sending this mail since erwin at freebsd.org is on vacation).

It seems the European side of ftp.freebsd.org suffored a double disk
failure during the weekend and this means we lost all data on the
mirror.

At first only disk was showing up as missing but the controller reported
the RAID failed. We replaced the disk, and then a second disk showed up
as missing. We're currently replacing the second disk.

However this leaves us with no other options than building the RAID from
scratch, doing a newfs and resync all data from ftp-master. 

I'll keep ftpd disabled until we're back in sync.

We are continously monitoring the state of the RAID (using nagios) but
got no warning of a failed disk, so I fear the first disk might have
failed silently since two disk failures in 5 minutes (time between
nagios polls) is highly unlikely.

Regards
Jan