zfs mirror reads only on one disk

[Jeremy Chadwick]

On Tue, Aug 09, 2011 at 05:19:02PM +0200, Jeremie Le Hen wrote:
> On Tue, Aug 09, 2011 at 06:52:12AM -0700, Jeremy Chadwick wrote:
> > 
> > Performance was a little less than I estimated (I really don't care to
> > be honest), but this double-confirms that yes, reads do get split across
> > mirror members.
> 
> By the way, I tried with the same disk (WD Caviar Black) (not the same
> size though) on Solaris 11:
> 
>                     extended device statistics
>     r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
>  1104.5    0.0 141371.6    0.0  0.0  0.9    0.0    0.9   1  95 c5t5d0
>                     extended device statistics
>     r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
>  1101.7    0.0 141014.0    0.0  0.0  0.9    0.0    0.9   1  95 c5t5d0
> 
> It can reach 140 MB/s!

I'm aware of this, but the drive won't perform like that under random
seeks, nor as the actuator arm starts to work its way towards the centre
of the disk.  This is normal for mechanical drives.  Given that my disks
are about ~30% used, the performance I saw is about right.

I should also point out, even though it's unrelated to this thread, that
the WD1002FAEX drives use 1 less platter than the WD1001FALS (both are
1TB drives), and the WD1002FAEX has 64MBytes cache.  As such, the
overall speed improvement is about 20MBytes/sec..  Also, less platters =
better airflow; the idle temperature difference between the two models
is about 6C on my desktop system, and about 4-5C on my (low-noise)
servers.  I have a small side project to replace all our WD1001FALS
drives with WD1002FAEX due to all the positives.  It's worth it, IMO.

> > Therefore I cannot explain what you're seeing.  Maybe consider upgrading
> > to a newer RELENG_8 and ZFSv28 and see if things improve?  I wish I had
> > a way to confirm this would fix your problem but I do not.
> 
> Ok, I didn't know ZFS v28 was MFC'd to RELENG_8.  By the way, I see you
> switched to the new ata(4) driver according to your device names.  Is it
> painless to upgrade?  I'll give it a try this evening hopefully.

That depends on how complex your system is as well as what you define
"painless" as.  For me, it's as simple as adding ahci_load="yes" into
/boot/loader.conf and changing /etc/fstab.  That's literally all I have
to do on my systems to get them to work.  I'm excluding things like
adjusting configuration files for monitoring suites (e.g. changing
rrdbot's configuration to monitor ada0 instead of ad4, etc...).

Overall the ATA-to-CAM emulation stuff works beautifully, it's one of
the things that makes me a very happy camper with FreeBSD.  I wish I
could send mav@ a whole ton of beer.  :-)

FreeBSD 9.0-BETA1 defaults to using ada(4), even on IDE/PATA disks, by
the way.  Just thought I'd throw that out there.  (I verified it in
VMware Workstation using IDE disk emulation (not LSI Logic SAS) last
night).

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |