Areca vs. ZFS performance testing.

[Jeremy Chadwick]

On Fri, Oct 31, 2008 at 01:44:25AM -0700, Simun Mikecin wrote:
> Jeremy Chadwick wrote:
> > The tuning variables I advocate for a system with 2GB of RAM or more,
> > on RELENG_7, are:
> > vm.kmem_size="1536M"
> > vm.kmem_size_max="1536M"
> 
> There is no point in setting vm.kmem_size_max. Setting vm.kmem_size is
> enough. vm.kmem_size_max is used for auto-tuning of kmem size which is
> in this case actually overriden by manually setting vm.kmem_size.

This information hasn't been mentioned by anyone before, so it's news
to me.

> > vfs.zfs.arc_min="16M"
> > vfs.zfs.arc_max="64M"
> > vfs.zfs.prefetch_disable="1"
> > You can gradually increase arc_min and arc_max by ~16MB increments as
> > you see fit; you should see general performance improvements as they
> > get larger (more data being kept in the ARC), but don't get too crazy.
> > I've tuned arc_max up to 128MB before with success, but I don't want
> > to try anything larger without decreasing kmem_size_*.
>
> Can you explain why would you have to decrease kmem_size to use larger
> ARC?  AFAIK it should be contrary to what you are saying: when you use
> larger kmem_size you can also use larger arc_max.

Well, my understanding (which is probably wrong) is that the memory
used for the ARC is somehow separate from that of the kmap.  I was
under the impression the kmap was used by ZFS for other things, and
did not include ARC.

If I'm incorrect, please state so -- it's cool, I just need to know.
:-)

> My suggestion if you are using kmem_size of 1536M would be to not tune
> arc_min and arc_max if your system isn't panicing. If it does you
> should try decreasing arc_max (from it's default value) until it
> doesn't.

People have advocated increasing arc_min and arc_max in the past, citing
large performance gains as arc_max gets larger; you might see people
mentioning that they see great performance increases when increasing
arc_max from 64M to 128M.  My understanding is that increasing the ARC
provides more actual cached data that ZFS can reference (vs. pulling it
off disk).  Again, if I'm incorrect, please state so.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |