FreeBSD 9.0 and NFS async -off topic about ZFS ZIL devices

[Peter Maloney]

Am 14.12.2011 01:38, schrieb Rick Macklem:
> Johan Hendriks wrote:
>> Rick Macklem schreef:
>>> Johan Hendriks wrote:
>>>> Hello all.
>>>>
>>>> I used to use async on my 8.x nfs servers!
>>>> On the FreeBSD 9.0 server i can not do it through the old 8.x
>>>> sysctl.
>>>>
>>>> Is there an other way to set async on FreeBSD 9.x
>>>>
>>> You have two choices:
>>> 1 - Apply this patch to your NFS server's kernel sources and then
>>> set
>>>      vfs.nfsd.async=1
>>>      http://people.freebsd.org/~rmacklem/async.patch
>>>
>>> 2 - switch to using the old server by setting
>>>      oldnfs_server_enable="YES"
>>>      in your /etc/rc.conf and then setting the sysctl.
>>>
>>> I'll assume that you realize that doing this violates the NFS RFCs
>>> because
>>> it runs your server in a way where there is a risk of data loss
>>> (that the
>>> client won't know to re-write) when the server crashes.
>>>
>>> rick
>>>> regards,
>>>> Johan Hendriks
>>>> _______________________________________________
>>>> freebsd-fs at freebsd.org mailing list
>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>>> To unsubscribe, send any mail to
>>>> "freebsd-fs-unsubscribe at freebsd.org"
>> Yes i do know the risk.
>>
>> The thing is we want a dataset shared to a ESXi client using NFS.
>> I use NFS for my normal usage, (sharing ports tree and so on.) but now
>> we want to use it to share a ZFS dataset for a ESXi client.
>> We use iscsi now, but this way we miss some zfs goodies. like
>> snapshots(not a zvol) and most important, we can reach the files
>> directly.
>>
>> But with a virtual machine shared over NFS i get horrible performance.
>> If i copy a file to whatever virtual machine from a windows client
>> shared with iscsi , i get arround 80Mb per second (in the windows copy
>> window) almost at a steady pace. we are really pleased with that. !!
>> If i copy a virtual machine to the NFS share, fire it up, and do a
>> file
>> copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb
>> then
>> goes to 20 back to 10 and so on.
>> Also the machines feels sluggish in performance.
>>
>> Are there other less dangerous things i can try to boost performance?
>>
> I don't use ZFS, but others have reported using a dedicated SSD that has
> good write performance for the ZIL log in order to get better write
> performance for ZFS.
Indeed, I can get 65 MB/s using a consumer SSD with normal sync nfs
clients. But once you use ESXi's NFS client, it will drop to somewhere
between 5 and 9 MB/s. Mine currently goes around 7 MB/s. I also tried
adding a ramdisk as my ZIL, and then it still only goes 80 MB/s, even
though obviously the ramdisk should be able to handle random writes
better than anything, and should go GB/s if there was no other
bottleneck, not only 80 MB/s (over 10Gbps network, which is tested at
about 6 Gbps with 1500 MTU). Anything else has to go through RAM at some
point anyway. Unfortunatlely, I didn't test a normal sync nfs client
with the ramdisk... I'll put that on my mental todo list. I also tried a
UFS zvol, whidh was not using the log, but only went somewhere between
40 and 60 MB/s.

Here is a very nice thread with some scores with various SSDs using a
normal NFS client.
http://forums.freebsd.org/showthread.php?t=23566

Here is my post in the same thread comparing the linux NFS client with
sync option, to the ESXi client (writing to a virtual disk filesystem
with dd):
http://forums.freebsd.org/showpost.php?p=157154&postcount=55


>> regards,
>> Johan Hendriks
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"