zfs performance degradation

Paul Kraus paul at kraus-haus.org
Thu Sep 24 18:46:14 UTC 2015

On Sep 24, 2015, at 12:17, Dmitrijs <war at dim.lv> wrote:

> I also get about 60-70MB/sec via CIFS or ftp, but my aim is to be limited by network, so 100MB is wanted. 

It’s nice to want things. But be prepared to pay for the things you want...

> Or, to understand why it is not possible on my config :)
> But simple dd of=/dev/null in the console shows me 110MB/sec…

dd of anything, but especially of /dev/null is a very poor way of measuring anything. Turn on compression and and do that test again.

> iozone gives me the same 100+Mb/sec both on read and write.

What size blocks ? Files ? Random or sequential I/O ? All that matters.

> 2x HGST HDN724040ALE640, 4Tb, 64Mb, 7200.

Consumer NAS drives… I have not purchased anything but an Enterprise drive for close to 10 years now. The small additional cost is well worth the longer (5 year) warranty and better build quality… there _is_ a difference. Even looking at the specs, the uncorrectable error spec is a very good indicator of build quality and these drives are typical 1 in 10^14 consumer drives. Enterprise drives are typically an order of magnitude better, 1 in 10^15.

In your original post you mentioned WD Green drives, also consumer grade. In my experience I have seen better performance from WD than HGST, with Seagate at the bottom of the ladder. I was comparing all Enterprise drives, and even among those offerings there are differences… the WD RE are noticeable faster than the SE. I look at svc_t as the primary metric for _comparing_ drives. I create a simple striped zpool, reboot the system to clear counters, then do _lots_ of (typically) random I/O, then look at iostat -x and compare svc_t, lower numbers are better.

I generally don’t buy matched drives for mirrors, but different makes and models if I can. That way of there is a bad production run I don’t lose all my drives at once. When a drive fails I RMA it under warranty and buy 2 more of the same type and capacity, one goes into the server and the other sits on the shelf. Eventually I have enough drives to grow the zpool and move on.

> For example, yesterday I explored QNAP TS-451
> official site: https://www.qnap.com/i/en/product/model.php?II=143&event=2 (Intel® Celeron® 2.41GHz dual-core processor, 1GB DDR3L, etc)
> and review: http://www.storagereview.com/qnap_ts451_nas_review
> 473euro

They _might_ be fine products, But I don’t trust my data to appliances. _I_ want to control the redundancy.

> Promised performance of the models is about 100Mb/sec, even up to 200Mb/sec but ok, it's marketing and pretty diagrams ;)

And all the tests were probably done on empty (to start) volumes. You can achieve similar numbers with ZFS with similar hardware and LOTS of parallel clients.

Paul Kraus
paul at kraus-haus.org

More information about the freebsd-questions mailing list