Areca vs. ZFS performance testing.

[Danny Carroll]

Jeremy Chadwick wrote:
> I think these sets of tests are good.  There are some others I'd like to
> see, but they'd only be applicable if the 1231-ML has hardware cache.  I
> can mention what those are if the card does have hardware caching.

The card comes standard with 256Mb of cache.

>> I do have some concern about the size of the eventual array and ZFS' use
>> of system memory.  Are there guidelines available that give advice on
>> how much memory a box should have with large ZFS arrays?
> 
> The general concept is: "the more RAM the better".  However, if you're
> using RELENG_7, then there's not much point (speaking solely about ZFS)
> to getting more than maybe 3 or 4GB; you're still limited to a 2GB kmap
> maximum.
> 
> Regarding size of the array vs. memory usage: as long as you tune kmem
> and ZFS ARC, you shouldn't have much trouble.  There have been some
> key people reporting lately that they run very large ZFS arrays without
> issue, with proper tuning.

I followed the recommendations here:
http://wiki.freebsd.org/ZFSTuningGuide

vm.kmem_size="1024M"
vm.kmem_size_max="1024M"
vfs.zfs.debug=1

And : kern.maxvnodes=400000

I have not added the following because they were listed in the i386
section.  (These values were quoted for a machine with 768Mb of ram)
vfs.zfs.arc_max="40M"
vfs.zfs.vdev.cache.size="5M"

Am I right in assuming these do not apply to amd64?  The article was not
specific.

> 
> Also, just a reminder: do not pick a value of 2048M for kmem_size or
> kmem_size_max; the machine won't boot/work.  You shouldn't go above
> something like 1536M, although some have tuned slightly above that
> with success.  (You need to remember that there is more to kernel
> memory allocation than just this, so you don't want to exhaust it all
> assigning it to kmap.  Hope that makes sense...)

It makes sense.   I'm using 1024 at the moment, but I've never really
looked into what memory is actually being used.

Tuning advice here would be well received :-)

>> Can an AMD64 kernel make use of memory above 2g?
> 
> Only on CURRENT; 7.x cannot, and AFAIK, will never be able to, as the
> engineering efforts required to fix it are too great.
> 
> I look forward to seeing your numbers.  Someone here might be able to
> compile them into some graphs and other whatnots to make things easier
> for future readers.

Ahhh, well, that will eventually decide my upgrade path when RELENG_8 is
released and stable.

> Thanks for doing all of this!

No worries, hopefully it will be useful information to future google
searches :-P

-D