NFS Performance issue against NetApp

[Marc G. Fournier]

On 2013-05-12, at 17:58 , Jeremy Chadwick <jdc at koitsu.org> wrote:

> 
>> Why does it take 34x as many reads on FreeBSD, where rsize on both
>> Linux/FreeBSD are the same ... ?   The amount of data to be read  is
>> the same ... shouldn't the # of reads be within the same ballpark,
>> at least ... ?
> 
> Can you provide actual proof of this, re: that at the syscall level,
> JBoss is issuing the same nbytes count to read(2) on BSD as it is on
> Linux and Solaris?  You can accomplish this with strace on Linux and
> ktrace (not truss) on FreeBSD.
> 
> For all we know, JBoss could have some "OS optimisation" setting within
> it that says for BSDs use 512 bytes, while for Linux and/or Solaris use
> 16384 bytes.  

That would explain a 4x difference .. not a 34x difference … 

As for strace, if anyone feels this has merit and can tell me what I need to run, I'm happy to do so … 

> Probably off-topic but worth pointing out: I do not know about Solaris,
> but Linux has multiple layers of caching, and is well-known for doing
> things like caching (and aggregating!) reads/writes to **block** devices
> (this is why on Linux you have to make sure to avoid caching your
> application use O_DIRECT with open(2) or other mechanisms -- the BSDs do
> not do this, block devices are always non-cached).

Caching *should* only come into play after the first run of the application … the first run after a reboot of the server shouldn't have anything in cache  yet for caching to come into play … unless Linux/Solaris have some way of anticipating what application I'm going to run before me … ?