How to best send files over network?

[Rick Macklem]

Matthew Fleming wrote:
> On Wed, Jan 11, 2012 at 8:07 AM, Rick Macklem <rmacklem at uoguelph.ca>
> wrote:
> > Mikhail T. wrote:
> >> On 10.01.2012 18:49, Bob Friesenhahn wrote:
> >> > Don't use mmap on zfs since doing so wastes memory (zfs ARC is
> >> > not
> >> > coherent with mmap page cache). Instead do normal file I/O (e.g.
> >> > write, fwrite) using the filesystem blocksize (e.g. 128K) or a
> >> > small
> >> > multiple thereof.
> >> Well, that was the reason cited for not using sendfile over ZFS.
> >> But
> >> mmap/write, it was claimed, was efficient.
> >>
> >> What's the general opinion of using mmap/write, when the file is on
> >> UFS?
> >> Is it just as good as sendfile, or can sendfile be better under
> >> some
> >> circumstances?
> >>
> >> > It is useful to cache several blocks on the receiving end and use
> >> > a
> >> > thread to receive data from the network in case zfs temporarily
> >> > stalls
> >> > during write (which it periodically does).
> >> No, thanks. I'm certainly not doing a read/write loop -- that's
> >> just
> >> too
> >> disgusting in the age of better interfaces (even if those aren't
> >> well
> >> implemented yet) :-)
> >
> > I think Bob was referring to the receive end and not the send end,
> > which
> > might be why the answer didn't make sense to you? (For the receive
> > end,
> > it sounds like a good suggestion to me.)
> 
> Some day, when there's time, I'd like to roll up Isilon's recvfile(2)
> into a proper patch. It's the obvious analogue to sendfile(2). I
> think it also requires two new VOPs, for efficiency, VOP_READ_MBUF
> (for sendfile) and VOP_WRITE_MBUF, which use an mbuf chain rather than
> uio to do their work.
> 
The NFS code has worked around this by allocating a bunch of mbufs and
creating an iovec that refers to their data areas, pretty well forever.
(Just guessing this would avoid adding the VOP_xxx_MBUF()s?)

However, don't mistakenly interpret this as me volunteering to do this:-) rick

> Perhaps at BSDCan, if this is my priority at the time. Or, if someone
> else wants to work on it I can provide patches that will require work.
> 
> Thanks,
> matthew