ZFS performance gains real or imaginary?

Matt Simerson matt at corp.spry.com
Wed Dec 24 22:35:01 PST 2008

On Dec 24, 2008, at 9:29 PM, Adam McDougall wrote:

>> On Wed, Dec 24, 2008 at 01:00:14PM -0800, Matt Simerson wrote:
>>  On Dec 23, 2008, at 7:48 PM, Adam McDougall wrote:
>>>> On Tue, Dec 23, 2008 at 12:43:47PM -0800, Matt Simerson wrote:
>>>>> On Dec 18, 2008, at 4:12 PM, Andrew Snow wrote:
>>>>>> If so, then I really should be upgrading my production ZFS  
>>>>>> servers
>>>>>> to the latest -HEAD.
>>>>> Thats correct, that is the only way to get the best working  
>>>>> version
>>>>> of ZFS.  Of course, then everything is unstable and broken - eg.
>>>>> SMBFS became unusable for me and would crash the server.
>>>> Unfortunately, the newer kernel hangs much more frequently.
>>>> I have these settings in /boot/loader.conf
>>>> 	vm.kmem_size="1536M"
>>>> 	vm.kmem_size_max="1536M"
>>>> 	vfs.zfs.arc_max="100M"
>>>> I have also experimented with vfs.zfs.prefetch_disable,
>>>> vfs.zfs.arc_min in the past, and I'm open to suggestions on what
>>>> might help under this workload (multiple concurrent rsync
>>>> processes from remote systems to this one).
>>> Can you try:
>>> vm.kmem_size=2G
>>> vm.kmem_size_max=2G
>>> vfs.zfs.arc_max=512M
>>> This has been working for me on one amd64 system that only
>>> has 2G of  ram but had similar problem frequency to yours.  I
>>> don't know if its  coincidence with the data that I am rsyncing
>>> lately, but: 10:47PM  up 22 days,  7:12
>>  I made it 23 minutes. I've reduced my rsync concurrency to 1, so I'm
>>  not hitting the system nearly as hard but it seems not to matter.
>>  Other workloads, like a 'make buildworld' will complete with no
>>  problems. For whatever reason, rsync sessions of entire unix systems
>>  to my backup servers are very troublesome.
>>  Matt
> Ok.  Since you have 16G of ram, I suppose you could try setting both  
> kmem
> sizes to something like 8G to see if it makes a difference?  I'm  
> getting
> a feeling that even if we don't see an outright failure, it might be
> deadlocking due to a kmem shortage.

back01# w
10:17PM  up 40 mins, 2 users, load averages: 4.20, 3.07, 1.74

This is with:


I'll let it trundle along with that setting and see how long it lasts.


PS: These settings earlier today resulted in 12+ hours of uptime,  
until I rebooted to test raising kmem_size to 4G.


PPS: If/when it hangs with 4G, I'll raise it again to 6 or 8 GB and  
see how long it lasts. Whatever pattern emerges might be useful for  

More information about the freebsd-fs mailing list