How to best send files over network?

[Bob Friesenhahn]

On Tue, 10 Jan 2012, Mikhail T. wrote:

> I'm trying to implement a BSD-tuned sending of files. Most of the files are 
> fairly large. I thought, sendfile(2) is the way to go, but, it seems, there 
> are severe performance problems at the moment for the files located on a ZFS.
>
> mmap-ing the files and then write-ing them is claimed to be a lot faster. 
> Could somebody comment on this? Should one always mmap/write, or are there 
> situations, when sendfile is advantageous?

Don't use mmap on zfs since doing so wastes memory (zfs ARC is not 
coherent with mmap page cache).  Instead do normal file I/O (e.g. 
write, fwrite) using the filesystem blocksize (e.g. 128K) or a small 
multiple thereof.

Of course if you are doing this over the network then you will need a 
program on the receiving end to write the data, and a program on the 
sending end to send it.

It is useful to cache several blocks on the receiving end and use a 
thread to receive data from the network in case zfs temporarily stalls 
during write (which it periodically does).

> If, indeed, sendfile is best for UFS, but mmap/write is better over ZFS, what 
> is the best way to determine the underlying FS for each file? statfs(2) is 
> supposed to answer that question -- what should I look for in the struct 
> statfs, that it will return? Do I check, if f_fsid contains a magic number 
> for ZFS, or look for a magic string in f_fstypename? Could someone provide an 
> example?

Use it to obtain the filesystem block size.

> Thank you very much. Yours,

Good to hear from you!

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/