zfs very poor performance compared to ufs due to lack of cache?

[Andriy Gapon]

on 14/09/2010 20:30 Steven Hartland said the following:
> 
> ----- Original Message ----- From: "Andriy Gapon" <avg at freebsd.org>
>> I'd really prefer to see description of your sources as svn revision rXXXXX plus
>> http link to a diff of your actual sources to that revision.
>> That would greatly help to see what you actually have, and what you don't have.
> 
> The zfs files don't seem to have any svn revision information in them. Is there
> something
> else that would id the revision of
> sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
> or the svn revision of the version of stable in use?


I don't know.  I haven't used cvsup since the switch to svn, but generally in
CVS each file has its own independent revision, so it's impossible to tell that
the whole tree is at revision R.
Maybe you could get yourself another source tree by doing svn checkout and
applying the patches there.  Then you could update it with svn update.
For you branch URI is svn://svn.freebsd.org/base/stable/8
You would be able to get a diff of your local changes with svn diff command.

>> Well, I would love to see the mentioned above graphs for this real test load.
>> Going below c_min likely means that you don't have all the latest stable/8 ZFS
>> code, but i am not sure.
> 
> It defintely is :(
> If its relavent the source was downloaded via cvsup from the uk mirror.
> 
>>> If someone could suggest a set of tests that would help I'll be happy to run
>>> them but
>>> from what's been said thus far is seems that the use of sendfile is forcing
>>> memory
>>> use
>>> other than that coming from arc which is what's expected?
>>>
>>> Would running the same test with sendfile disabled in nginx help?
>>
>> The more test data the better, we could have some base for comparison and
>> separation of general ARC issues from sendfile-specific issues.
> 
> Going to run the following tests:-
> 1. run a live test with "sendfile off" in the nginx config
> 2. run a live test with "sendfile on" in the nginx config.
> 
> During these tests I'm going to monitor the following every minute:-
> time, kstat.zfs.misc.arcstats.size, vm.stats.vm.v_pdwakeups,
> vm.stats.vm.v_cache_count, vm.stats.vm.v_inactive_count,
> vm.stats.vm.v_active_count, vm.stats.vm.v_wire_count,
> vm.stats.vm.v_free_count
> 
> Anything else that should be monitored?
> 
> Before each test the machine will be rebooted to try to ensure as direct a
> comparison
> as possible.
> 
> Anything else that I should add / change before running said tests or should
> monitor?

This sounds sufficiently good.
If you could arrange to draw the graphs of the data it would be terrific :)

-- 
Andriy Gapon