ZFS performance gains real or imaginary?

[Matt Simerson]

On Dec 18, 2008, at 4:12 PM, Andrew Snow wrote:

>> If so, then I really should be upgrading my production ZFS servers  
>> to the latest -HEAD.
>
> Thats correct, that is the only way to get the best working version  
> of ZFS.  Of course, then everything is unstable and broken - eg.  
> SMBFS became unusable for me and would crash the server.

I got bit by the ARP bug (kern/129730) which really annoyed our  
network manager. After applying a patch for that, I've got a working  
kernel and upgraded ZFS.

Unfortunately, the newer kernel hangs much more frequently. Previously  
I was getting nearly a month between reboots. Now I don't even get 1  
day between hangs. Worse yet, I upgraded the ZFS version of the pools,  
to see if that would make any difference. It did not, and now I can't  
revert. :-(

I have these settings in /boot/loader.conf

	vm.kmem_size="1536M"
	vm.kmem_size_max="1536M"
	vfs.zfs.arc_max="100M"

I have also experimented with vfs.zfs.prefetch_disable,  
vfs.zfs.arc_min in the past, and I'm open to suggestions on what might  
help under this workload (multiple concurrent rsync processes from  
remote systems to this one).

Matt

Specs: SuperMicro 24 disk, 8-core Xeon, 16GB RAM, 2 x 12 disk HW RAID  
striped (RAID 0).

back01# uname -a
FreeBSD back01.int.spry.com 8.0-CURRENT FreeBSD 8.0-CURRENT #2: Fri  
Dec 19 15:37:12 PST 2008     root at back01.int.spry.com:/usr/obj/usr/src/ 
sys/BACK01  amd64


back01# zpool list
NAME     SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
back01  18.1T  16.5T  1.60T    91%  ONLINE  -


back01# zpool status
   pool: back01
  state: ONLINE
  scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	back01      ONLINE       0     0     0
	  da0       ONLINE       0     0     0
	  da1       ONLINE       0     0     0

errors: No known data errors


back01# zpool upgrade
This system is currently running ZFS pool version 13.

All pools are formatted using this version.