Poor I/O performance; maybe ZFS?

[Matthew Seaman]

On 08/10/15 16:12, Julio Merino wrote:
> Hello,
> 
> I have a PowerMac G5 running 11.0-CURRENT (upgraded yesterday).  The machine is a dual-CPU 2GHz G5 with 6GB of RAM and two drives:
> 
> ada0 at ata2 bus 0 scbus2 target 0 lun 0cd0 at ata0 bus 0 scbus0 target 0 lun 0
> ada0: <ST3160023AS 3.05> ATA-6 SATA 1.x device
> ada0: Serial Number 3JS478VL
> ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada0: 152627MB (312581808 512 byte sectors: 16H 63S/T 16383C)
> ada1 at ata3 bus 0 scbus3 target 0 lun 0
> ada1: <WDC WD6400AAKS-75A7B0 01.03B01> ATA8-ACS SATA 2.x device
> ada1: Serial Number WD-WMASY2696897
> ada1: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada1: 610480MB (1250263728 512 byte sectors: 16H 63S/T 16383C)
> 
> I have a UFS file system on ada0s4 for / and a ZFS pool on ada1s2 with file systems for /home and poudriere.
> 
> For a while, I have felt that the system is incredibly sluggish. For
> example, a simple "make buildworld" with up-to-date obj, which should be
> I/O bound, takes forever (something like 20-30 minutes). This is running
> with src and obj on ZFS. "svn update" is similarly infuriatingly slow.

Beware: 'svn update' is frequently limited by network bandwidth, so not
really a good test of your filesystems.

> Today, I decided to install bonnie and ran a couple of tests using "-s 4096".
> 
>               -------Sequential Output-------- ---Sequential Input-- --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
> ada0     4196 43973 62.4 46207 37.7 43259 41.9 89210 96.8 280589 95.9 17038.2 103.1
> tank     4196 54093 72.2 27596 93.0 21478 90.7 64214 83.5 65358 50.8 126.7  3.7
> 
> Note the vast difference on the numbers, especially random seeks.
> 
> In case this matters, this is how ada1s2 looks like:
> 
> 1. Name: ada1s2
>    Mediasize: 640135019520 (596G)
>    Sectorsize: 512
>    Stripesize: 0
>    Stripeoffset: 9216
>    Mode: r1w1e1
>    rawtype: FreeBSD-ZFS
>    label: 1
>    length: 640135019520
>    offset: 9216
>    type: freebsd-zfs
>    index: 1
>    end: 1250263727
>    start: 18
> 
> Any obvious things I should look at?

6GB is not a great deal when we're talking ZFS.  For a mixed read/write
workload like compiling, ZFS is going to be working its read cache 'ARC'
pretty hard.  It's also going to be competing with the traditional
buffer cache for the available memory.

Try installing 'zfs-stats' from ports and look at the output of
'zfs-stats -A' and 'zfs-stats -E' for clues.  You should also *limit*
the amount of memory ZFS can dedicate to the ARC -- counter intuitive,
but memory dedicated to ARC is memory that the compiler (for instance)
cannot be using.  I'd be looking at setting vfs.zfs.arc_max="3G" in
/boot/loader.conf were I in your shoes -- or possibly even less.
Whatever you do, you aren't going to get stellar ZFS performance out of
a machine with that small an amount of memory, and pushing all the iops
through a single SATA-1 drive, but these sort of measures should at
least improve things.

Beyond that there's a bunch of settings you can fiddle with documented here:

https://www.freebsd.org/doc/handbook/zfs-advanced.html
https://wiki.freebsd.org/ZFSTuningGuide

	Cheers,

	Matthew

PS. poudriere doesn't actually need ZFS nowadays.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20150810/77123e94/attachment.bin>