NFS on ZFS pure SSD pool

[aurfalien]

On Aug 27, 2013, at 4:08 PM, Rick Macklem wrote:

> I wrote:
>> Outback Dingo wrote:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Tue, Aug 27, 2013 at 3:29 PM, Rick Macklem <
>>> rmacklem at uoguelph.ca
>>>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> Eric Browning wrote:
>>>> Hello, first time posting to this list. I have a new server that
>>>> is
>>>> not
>>>> living up to the promise of SSD speeds and NFS is maxing out the
>>>> CPU.
>>>> I'm
>>>> new to FreeBSD but I've been reading up on it as much as I can. I
>>>> have
>>>> obscured my IP addresses and hostname with x's so just ignore
>>>> that.
>>>> Server has about 200 users on it each draing under 50Mb/s peak
>>>> sustained
>>>> around 1-2Mb/s
>>>> 
>>>> I've followed some network tuning guides for our I350t4 nic and
>>>> that
>>>> has
>>>> helped with network performance somewhat but the server is still
>>>> experiencing heavy load with pegging the CPU at 1250% on average
>>>> with
>>>> only
>>>> 50Mb/s of traffin in/out of the machine. All of the network
>>>> tuning
>>>> came
>>>> from https://calomel.org/freebsd_network_tuning.html since it was
>>>> relevant
>>>> to the same nic that I have.
>>>> 
>>>> Server Specs:
>>>> FreeBSD 9.1
>>>> 16 cores AMDx64
>>>> 64GB of ram
>>>> ZFS v28 with four Intel DC S3700 drives (800GB) as a zfs stripe
>>>> Intel DC S3500 for ZIL and enabling/disabling has made no
>>>> difference
>>>> Used a spare DC S3700 for the ZIL and that made no difference
>>>> either.
>>>> NFS v3 & v4 for Mac home folders whose Cache fodler is
>>>> redirected.
>>>> 
>>>> I've tried:
>>>> Compression on/of <-- no appreciable difference
>>>> Deduplication on/off <-- no appreciable difference
>>>> sync=disabled and sync=standard <-- no appreciable difference
>>>> setting arc cache to 56GB and also to 32GB <-- no difference in
>>>> performance
>>>> in terms of kern.
>>>> 
>>>> I've tried to follow the freebsd tuning guide:
>>>> https://wiki.freebsd.org/ZFSTuningGuide to no avail either. I've
>>>> read
>>>> everything I can find on NFS on ZFS and nothing has helped. WHere
>>>> am
>>>> I
>>>> going wrong?
>>>> 
>>> You could try this patch:
>>> http://people.freebsd.org/~rmacklem/drc4-stable9.patch
>>> - After applying the patch and booting a kernel built from the
>>> patched
>>> sources, you need to increase the value of vfs.nfsd.tcphighwater.
>>> (Try something like 5000 for it as a starting point.)
>>> 
>>> 
>>> 
>>> 
>>> can we get a brief on what this is supposed to improve upon ?
>>> 
>> It was developed for and tested by wollman@ to reduce mutex lock
>> contention and CPU overheads for the duplicate request cache, mainly
>> for NFS over TCP. (For the CPU overheads case, it allows the cache
>> to grow larger, reducing the frequency and, therefore, overhead of
>> trimming out stale entries.)
> Oh, and I should also mention that ivoras@ developed a similar patch which
> had better code structure than mine. I did use some of his code in the patch
> that went into head, but not as much as I would have liked, because I wanted
> to get it into head before code slush for 10.0. (I had already missed the 9.2
> release.)

I take it that this patch can be applied to 9.2 when its officially released?

You mentioned 

- aurf