Strange NFS behaviour
Willem Jan Withagen
wjw at withagen.nl
Wed Jan 14 16:38:15 PST 2004
Hoi,
I'm not shure what to think of this...
Looking at building a NFS/Samba server to stick somewhere in a
corner in our house, I ran a few "performance" tests. If you would
like to call bonnie(1) such a beast. I used bonnie to get a first
feel for things, not as something I'd like to publish a paper with.
I just varied the size of the of the read and write block with UDP
and TCP. Slightly against what mount_nfs advises. But the results are
not very pleasing:
-------Sequential Output-------- ---Sequential
Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per
Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
local 256 759 10.1 886 1.1 750 2.4 4280 51.6 5383 7.1 96.3
2.8
udp-1k 256 629 9.3 638 1.0 400 4.2 1533 19.1 1522 2.9 79.6
4.7
udp-2k 256 718 10.2 714 1.1 488 3.5 2144 27.0 2561 4.8 68.3
3.4
udp-4k 256 774 10.7 783 1.2 551 3.1 3299 41.6 4379 8.0 79.9
3.1
udp-8k 256 757 10.1 807 1.0 765 2.4 3703 44.9 5147 7.3 84.0
2.7
udp-16k 256 1199 15.7 1137 1.3 1062 3.1 4525 54.5 5609 6.2 83.9
2.8
udp-32k 256 1215 15.8 1155 1.3 1067 3.1 4215 50.8 5629 6.4 86.1
3.0
udp-64k 256 1180 15.4 1137 1.3 1060 3.1 3815 46.0 5629 6.3 86.3
3.2
tcp-64k 256 1211 16.5 1002 1.2 245 0.8 4523 55.2 5689 5.8 61.5
3.9
tcp-64k 256 1160 15.8 1013 1.3 243 0.8 3971 48.1 5418 5.4 58.1
3.8
tcp-32k 256 1187 16.1 959 1.2 243 0.8 4088 49.6 4707 4.6 52.9
3.6
tcp-32k 256 1190 16.1 992 1.2 242 0.8 4204 51.2 5310 5.2 52.9
3.6
tcp-16k 256 885 11.9 798 0.9 139 0.5 3955 48.0 4576 5.8 76.3
3.6
tcp-16k 256 921 12.4 817 1.0 138 0.5 4241 51.5 4765 6.1 56.6
2.7
tcp-8k 256 750 10.0 844 1.0 75 0.3 3317 41.0 4140 6.4 53.1
2.1
tcp-8k 256 684 9.2 774 0.9 76 0.3 2895 35.7 4108 6.4 76.3
3.0
tcp-4k 256 758 10.2 734 1.1 74 0.6 2658 33.5 3170 6.1 69.0
3.5
tcp-4k 256 715 9.6 698 1.0 74 0.6 2618 33.1 3194 6.2 67.6
3.3
local is when I run bonnie on the local file system/server, which runs
5.1-RELEASE-p11/GENERIC on a P3-233mhz with 96Mb RAM and a 4 disk 170Gb
Vinum-Raid5.
(Which gives a feel on why the local number as so low while writing)
Other lines give the transfers using either TCP or UDP with 1K^2n block
size.
Now what strikes me as very odd, is the huge difference between TCP and UDP
in
the rewrite column. And, yes I'm aware of the fact that for every test
config
there is a setup that is going to perform poorly. But the "only" difference
is
the type if "virtual-wire" between the machines.
But such a huge difference????
BTW the client box runs: 5.2-CURRENT Mon Dec 22, and is a P3-533, with 256
Mb,
and has the diagnostics stuff off.
Both systems have an Intel based 259 fxp0 card set to 100Mbit, full-duplex,
connected by a simple switch.
I've found very little of errors, or unusual things.
On the client in netstat -s:
tcp:
5537543 segments updated rtt (of 2029581 attempts)
on the server:
tcp:
4787548 segments updated rtt (of 1864984 attempts)
Which is something that I can not really place.
Very little is dropped, retransmitted or out of sequence.
So the 1$ question is:
Why is this difference there, and should I(we) worry about it.
Note that I'm going to run this as well with a "real" workload:
compiling a kernel
But this was just a way of getting a flavour of things.
--WjW
More information about the freebsd-current
mailing list