How to best send files over network?

[Rick Macklem]

Mikhail T. wrote:
> On 10.01.2012 18:49, Bob Friesenhahn wrote:
> > Don't use mmap on zfs since doing so wastes memory (zfs ARC is not
> > coherent with mmap page cache). Instead do normal file I/O (e.g.
> > write, fwrite) using the filesystem blocksize (e.g. 128K) or a small
> > multiple thereof.
> Well, that was the reason cited for not using sendfile over ZFS. But
> mmap/write, it was claimed, was efficient.
> 
> What's the general opinion of using mmap/write, when the file is on
> UFS?
> Is it just as good as sendfile, or can sendfile be better under some
> circumstances?
> 
> > It is useful to cache several blocks on the receiving end and use a
> > thread to receive data from the network in case zfs temporarily
> > stalls
> > during write (which it periodically does).
> No, thanks. I'm certainly not doing a read/write loop -- that's just
> too
> disgusting in the age of better interfaces (even if those aren't well
> implemented yet) :-)

I think Bob was referring to the receive end and not the send end, which
might be why the answer didn't make sense to you? (For the receive end,
it sounds like a good suggestion to me.)

> >
> >> If, indeed, sendfile is best for UFS, but mmap/write is better over
> >> ZFS, what is the best way to determine the underlying FS for each
> >> file? statfs(2) is supposed to answer that question -- what should
> >> I
> >> look for in the struct statfs, that it will return? Do I check, if
> >> f_fsid contains a magic number for ZFS, or look for a magic string
> >> in
> >> f_fstypename? Could someone provide an example?
> >
> > Use it to obtain the filesystem block size.
> Ok, but still -- how does one determine the filesystem, where a
> particular file resides? Thanks!
> 
Well, I think you can strcmp("zfs", sb.fs_fstypename) == 0 to check for
a zfs file system. However, if the file system type name gets changed
to something like zfsv32 (just a hypothetical example), the above breaks.
(There is a case of this in the NFS server code, so hopefully it won't
 break for a while.;-)

rick