How to best send files over network?

[Bob Friesenhahn]

On Wed, 11 Jan 2012, Mikhail T. wrote:

> On 10.01.2012 18:49, Bob Friesenhahn wrote:
>> Don't use mmap on zfs since doing so wastes memory (zfs ARC is not coherent 
>> with mmap page cache).  Instead do normal file I/O (e.g. write, fwrite) 
>> using the filesystem blocksize (e.g. 128K) or a small multiple thereof.
> Well, that was the reason cited for not using sendfile over ZFS. But 
> mmap/write, it was claimed, was efficient.

You don't usually want to write files using mmap because it is hard to 
do so properly and because some operating systems are not very good 
about writing the dirty pages when they should.  You should avoid 
using mmap on zfs in general because it will double the memory 
consumption and double the buffer copies.

> No, thanks. I'm certainly not doing a read/write loop -- that's just too 
> disgusting in the age of better interfaces (even if those aren't well 
> implemented yet) :-)

Yes, it seems antique but if you are looking for optimum performance 
that you can tune and have control over, this is the way to go.

You could also try using async I/O.

>> Use it to obtain the filesystem block size. 
> Ok, but still -- how does one determine the filesystem, where a particular 
> file resides? Thanks!

I would tell you to use statvfs and the f_basetype member of the 
statvfs struct but it seems that FreeBSD only supports a very limited 
version of this structure.  Regardless, if the filesystem is mounted 
via NFS or SMB then the system is likely to lie about the true 
underlying filesystem.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/