Terrible NFS4 performance: FreeBSD 9.1 + ZFS + AWS EC2

[Berend de Boer]

>>>>> "Rick" == Rick Macklem <rmacklem at uoguelph.ca> writes:

    Rick> After you
    Rick> apply the patch and boot the rebuilt kernel, the cpu
    Rick> overheads should be reduced after you increase the value of
    Rick> vfs.nfsd.tcphighwater.

OK, completely disregard my previous email. I actually was testing
against a server in a different data centre, didn't think it would
matter too much, but clearly it does (ping times 2-3 times higher).

So moved server + disks into the same data centre as the nfs client.

1. Does not effect nfs3.

2. When I do not set vfs.nfsd.tcphighwater, I get a "Remote I/O error"
   on the client. On server I see:

     nfsd server cache flooded, try to increase nfsrc_floodlevel

   (this just FYI).

3. With vfs.nfsd.tcphighwater set to 150,000. I get very high cpu, 50%.

   Performance is now about 8m15s. Which is better, but still twice
   above a lower spec Linux NFS4 server, and four times slower than
   nfs3 on the same box.

4. With Garrett's settings, I looked at when the cpu starts to
   increase. It starts slow, but raises quickly to 50% in about 1
   minute.

   Time was similar 7m54s.

5. I lowered vfs.nfsd.tcphighwater to 10,000 but then it actually
   became worse, cpu quickly went to 70%, i.e. not much difference
   with FreeBSD without patch. Didn't keep this test running to see if
   it became slower over time.

   Making it 300,000 seems that the cpu increases are slower (but it
   keeps rising).

   So from what I observe from the patch is that it makes the rise in
   cpu increase slower, but doesn't stop it. I.e. after a few minutes,
   even with setting 300,000 the cpu is getting to 50%, but dropped a
   bit after a while to hover around 40%. Then it crept back to over 50%.

6. So the conclusion is: this patch helps somewhat, but nfs4 behaviour
   is still majorly impaired compared to nfs3.


--
All the best,

Berend de Boer


          ------------------------------------------------------
          Awesome Drupal hosting: https://www.xplainhosting.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP Digital Signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20130709/e2f43589/attachment.sig>