Linux NFS client and FreeBSD server strangeness
Russell L. Carter
rcarter at pinyon.org
Wed Apr 4 20:29:03 UTC 2018
On 04/04/18 11:27, Mike Tancsa wrote:
> Not sure where the tweaking needs to happen, but I am getting strange
> behaviour between a Linux nfs client and FreeBSD RELENG_11 NFS server.
I initially had terrible performance with a Linux client writing to a
FreeBSD server several years back, and after a lot of false starts this
snippet in /etc/sysctl.conf solved it:
# NFS client sux0rs w/o:
net.inet.tcp.tso=0
A caveat is I use NFS V4.1, though.
Good luck,
Russell
> The FreeBSD server starts with
>
>
> nfs_client_enable="YES"
> nfs_server_enable="YES"
>
>
> rpcbind_enable="YES"
> rpc_lockd_enable="YES"
> rpc_statd_enable="YES"
> nfs_server_flags="-u -t -n 16"
>
> and on the Linux client I have been trying various options to no avail.
> The mount works, but on a straight up write to the FreeBSD server,
> everything is very bursty. I noticed this (I think) a few months ago
> where Linux dumps across an nfs mount seemed to take a lot longer and
> were getting very bursty.
>
> It seems if there are a mixture of reads and writes, everything is
> pretty fast. But if a client is just writing to the server, something,
> somewhere is blocking. Doing something simple like
> ls -l /nfsmount
> from the client "wakes" up the server/client so that write stream can
> keep going. Otherwise, it will do a big blast of writes and then several
> seconds of pausing on the dump.
>
> Linux Dump is a simple
>
> /sbin/dump u -0 -f - / | /bin/bzip2 >/backup/dump-root-0.bz2
>
> Mount is
>
> mount.nfs -o tcp,intr,noatime,vers=3 192.168.yy.xx:/path
>
> If I run ifstat on the FreeBSD nfs server, the traffic pattern looks like
>
> # ifstat -b -i cxl0
> cxl0
> Kbps in Kbps out
> 0.00 0.00
> 0.00 0.00
> 0.00 0.00
> 0.00 0.00
> 0.00 0.00
> 8.12e+06 45127.03
> 0.00 0.00
> 0.00 0.00
> 0.00 0.00
> 0.00 0.00
> 6.04e+06 33525.76
> 901122.1 4983.72
> 0.00 0.00
>
> if I do a bunch of ls -l /nfsmount on the client
>
> eg
>
> while true
> do
> ls -l /backup/ > /dev/null
> done
>
> traffic pattern is
>
>
> cxl0
> Kbps in Kbps out
> 0.00 0.00
> 3.31e+06 18520.03
> 5.89e+06 32571.52
> 4.84e+06 28325.71
> 2.12e+06 19466.56
> 614727.0 12246.10
> 874927.6 13557.18
> 1.06e+06 14386.78
> 917865.4 13696.87
> 1.09e+06 14608.64
> 1.06e+06 14376.12
> 164077.3 5286.64
>
>
> Leading up to the stall, pcap snippet attached.
>
> Note, doing something like
>
> dd if=/dev/zero of=/backup/test.bin bs=4096 count=5000000
>
> I can saturate the 10G link and max out the disk on the server
>
> # dd if=/dev/zero of=/backup/test.bin bs=4096 count=5000000
> 5000000+0 records in
> 5000000+0 records out
> 20480000000 bytes (20 GB, 19 GiB) copied, 36.6238 s, 559 MB/s
>
> and its a pretty steady stream unlike the dump. Any ideas whats going
> on and how I might be able to work around this ?
>
>
> 192.168.xx.yy:/zbackup1/virtbox4b/backup on /backup type nfs
> (rw,noatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.242.254,mountvers=3,mountport=774,mountproto=tcp,local_lock=none,addr=192.168.yy.xx)
>
>
>
> ---Mike
>
>
> -------------------
> Mike Tancsa, tel +1 519 651 3400 x203
> Sentex Communications, mike at sentex.net
> Providing Internet services since 1994 www.sentex.net
> Cambridge, Ontario Canada
>
>
>
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
More information about the freebsd-fs
mailing list