Need Advice for Tuning NFS to place nice with a Netapp
freminlins at gmail.com
Wed Aug 2 20:24:54 UTC 2006
On 02/08/06, N. Harrington <drumslayer2 at yahoo.com> wrote:
Cool! Can you share with me what sort of settings you
> use on your boxes? sysctl/kerneltunes/mount options?
This may be a disappointment to you but... I didn't have to do anything :-(
All I have is rw on the client.
It has taken me a over a month to even get to speak
> to someone high enough up he food chain at Netapp to
> not say "FreeBSD - that's a version of Linux right?"
It depends who you speak to. There are people at NetApp who know about
The web server replies (using either Apache and
> Lighthttpd) seem to max out at about 17mb/s. Response
> time for the web server will rise gradually, then
> suddenly become 10-20seconds for a reply. Much like a
> backup on a highway. They claim that the netapp unit
> is spending too much time dealing with file
> information IOPS than actual transfer of files.
> However even on a non in-use server, if I make a
> request for a file, that "heavy file access" seems
> GtAttr Lookup Rdlink Read Write Rename Access
> 0 0 0 0 0 0 0 0
> 248 160 0 4 0 0 236 0
> 0 0 0 0 0 0 0 0
I would dispute NetApp's claim. NetApp filers are very capable at doing NFS
operations. Static files tend to sit usefully in the buffer cache on web
servers. So unless you are doing something really odd with your web servers
I would tend to disagree with NetApp.
I've just looked on one of our POP3 servers (mounting NetApps). POP3 causes
far more random access than our web servers. As such it doesn't sit in the
buffer cache very long.
We have much higher figures than yours and absolutely no performance
Yes, 4X GigE from the filer via a Vif and trunking on
> the switch. A nice 10Gb ready HP unit. I have asked if
> using the Vif and trunking could have any effects but
> been assured it should not. It does mean I cannot use
> jumbo frames. But since web pages and images are
> small, I don't think there would be any benefit.
There is nothing wrong in theory with that setup. But is may not be what you
want. Try it with just one GigE interface.
You're right - you probably don't need jumbo frames.
Which interface does the HP unit have? Also, have a look at netstat -in. Are
there any IErrs or OErrs or Coll? Paste the results here!
9907187 bytes/sec for a 16M file.
> It will transfer in nanotime. So, I believe that
> eliminates network performance as an issue.
Well, not really. The figure above is showing < 10MB a second. That's not
quite Fast Ethernet speed. If you are pushing 17mb (I guess that's megabits)
that's not really a problem though.
I've just tested this on the same POP3 server above, using dd to write a
file onto a NetApp and I get 10889359 a second. And this machine is busy.
Also, it is mounting the NetApp over Fast Ethernet.
Hmm. Drat. We have some web servers that do nothing
> but send out data, but some that are used for
> uploading and file manipulation. I will have to make
> sure that global of an option will not effect what
> they do.
It is a per volume option. And frankly I've never seen much use for atime.
It's useful sometimes, but not a lot.
Can you also put in the output of nfsstat -W -c 2. Maybe it's best to put
this up on the web somewhere as it's wide, and it's not easy to read in
email. Let it run for a minute or so, and if possible do two runs. One
during the OK time, the other during the problem time.
I would go back to basics. One GigE interface. Just rw mount options, and
start testing. By testing I mean measuring. NFS tuning is fiddly. I've been
using NetApps with FreeBSD for 5 years. It is a good combination.
Can you also post the output of sysctl -a|grep nfs. But don't start fiddling
with them yet!
> One last thing - are you female?! In a UNIX
> > newsgroup?!
> Yup :)
> Oh, and yes, I do play the drums :)
Oh gawd. Whatever next? :-)
Thanks for your assistance!!
More information about the freebsd-questions