zfs very poor performance compared to ufs due to lack of cache?

[Steven Hartland]

----- Original Message ----- 
From: "Andriy Gapon" <avg at freebsd.org>
> I'd really prefer to see description of your sources as svn revision rXXXXX plus
> http link to a diff of your actual sources to that revision.
> That would greatly help to see what you actually have, and what you don't have.

The zfs files don't seem to have any svn revision information in them. Is there something
else that would id the revision of sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
or the svn revision of the version of stable in use?

> Well, I would love to see the mentioned above graphs for this real test load.
> Going below c_min likely means that you don't have all the latest stable/8 ZFS
> code, but i am not sure.

It defintely is :( 

If its relavent the source was downloaded via cvsup from the uk mirror.

>> If someone could suggest a set of tests that would help I'll be happy to run them but
>> from what's been said thus far is seems that the use of sendfile is forcing memory
>> use
>> other than that coming from arc which is what's expected?
>> 
>> Would running the same test with sendfile disabled in nginx help?
> 
> The more test data the better, we could have some base for comparison and
> separation of general ARC issues from sendfile-specific issues.

Going to run the following tests:-
1. run a live test with "sendfile off" in the nginx config
2. run a live test with "sendfile on" in the nginx config.

During these tests I'm going to monitor the following every minute:-
time, kstat.zfs.misc.arcstats.size, vm.stats.vm.v_pdwakeups,
vm.stats.vm.v_cache_count, vm.stats.vm.v_inactive_count,
vm.stats.vm.v_active_count, vm.stats.vm.v_wire_count,
vm.stats.vm.v_free_count

Anything else that should be monitored?

Before each test the machine will be rebooted to try to ensure as direct a comparison
as possible.

Anything else that I should add / change before running said tests or should monitor?

    Regards
    Steve

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster at multiplay.co.uk.