Strange delays in ZFS scrub or resilver

[Paul Kraus]

On Feb 23, 2013, at 11:23 PM, John Levine <johnl at iecc.com> wrote:

> I have a raidz of three 1 TB SATA drives, in USB enclosures.  One of
> the disks went bad, so I replaced it last night and it's been
> resilvering ever since.  I can watch the activity lights on the disks
> and it cranks away for a minute or so, then stops for a minute, then
> cranks for a minute, and so forth.  If I do a zpool status while it's
> stopped, the zpool waits until the I/O resumes, and a ^T shows it
> waiting for zio->io_cv.
> 
> I'm running FreeBSD 9.1, amd64 version, totally vanilla install on a
> mini-itx box with 4GB of RAM.  The root/swap disk is an SSD separate
> from the zfs disks.  When the disks are active, top shows about 10%
> system time and 4% interrupt.  When it isn't, top shows about 99.8%
> idle.  The server isn't doing much else, and nothing else currently
> touches the disks.  (They're for remote backup of a system somewhere
> else, and I have the backup job turned off until resilvering
> completes.)

	Under 9.0 I had some external drives attached via USB and saw truly terrible I/O performance. I moved them to ESATA and it got much better. Unfortunately, my external exclosure has a SATA port expander as I need to talk to 4 external drives. That gives me about a factor of 2 worse performance than the internal SATA drives (even if I am only talking to one drive via the external connection).

--
Paul Kraus
Deputy Technical Director, LoneStarCon 3
Sound Coordinator, Schenectady Light Opera Company