sendfile(2) SF_NOPUSH flag proposal

[Igor Sysoev]

On Thu, 29 May 2003, Terry Lambert wrote:

> Igor Sysoev wrote:
> > On Wed, 28 May 2003, Terry Lambert wrote:
> > > Igor Sysoev wrote:
> > > > The portability argument is bogus because sendfile() portability is nonsense.
> > >
> > > Darwin has sendfile.  See the released source code: it matches
> > > the FreeBSD semantics, from what I can tell.
> > 
> > So now FreeBSD/Darwin is second pair after AIX/MVS that has the same sendfile()
> > prototype.  It surely improves the sendfile() portability.  Undoubtedly.
> 
> FreeBSD, NetBSD, OpenBSD, Darwin.

There's no sendfile() implementation in NetBSD and OpenBSD.  If you
apply some experimental patch you can easy fix some non-portable issues.

By the way what's about kqueue(2) ?  Are you not confused that NetBSD
does not support EVFILT_AIO and OpenBSD does not support EVFILT_AIO and
EVFILT_TIMER ?  Does this mean that FreeBSD should not introduce any
new kqueue filters or flags ?

> > > > The drawback that really annoyed me is that sendfile() blocks on a reading
> > > > from a disk while a sending to non-blocking socket.  Although I see three
> > > > workarounds it's much better to fix this inside sendfile().
> > >
> > > There's no workaround for the latency issue, which comes from
> > > the fact that a trap handles the request for more pages, and
> > > that blocks all callers.  Threads has the same problem in libc_r.
> > 
> > The workaround idea is simple - a preloading.  But implementation on user
> > level is complex.  In FreeBSD 4.x I see three ways:
> > 
> > *) the use of aio_read() to read the single bytes;
> 
> This does not fix the problem.  See the extensive discussion,
> last month, about the differences between libthr and libc_r
> and libpthreads.  Even when doing async I/O, you stall all
> threads in any model that isn't 1:1 when you fault for a
> user page in a system call.
> 
> This is because a fault on a user page results in entering
> the trap handler, which suspends the calling program until
> such time as the fault is satisfied, or a SIGSEGV is raised
> to the caller because the fault cannot be satisfied.

I agree but I told not about the blocking on a page fault but the blocking
on the reading the file page from a disk by sendfile(). These pages
can be preloaded.

> > *) the use rfork()ed helper processes to read the single bytes;
> > *) and the use the pool of rfork()ed processes to handle connections.
> 
> We call this "libthr".

I believe that rfork() and libthr are different things.  I think that
the rfork()ed process in FreeBSD 5.x is still the process but not the thread.

> > > > Five people ?
> > >
> > > Bill Fenner, Matt Dillon, Peter Jeremy, Marc Slemko, Terry Lambert,
> > > Garance Droshin.
> > 
> > At time of your mail there were only 4 people, in order of appearance:
> > Peter Jeremy, you, Matt Dillon, and Marc Slemko. Bill Fenner's email
> > was sent one and a half hour after yours and just before my response.
> > Garance Droshin's mail was sent several hours later.
> 
> Apparently, I received some mail that did not go to the list.
> 
> However, if you are willing to include the contents of the
> "sendfile() broken" thread, I could add three more before
> your time deadline, including Bruce Evans.

Well, but then these "five people have told you [ but not me ] the correct
approach is to fix sendfile(2)".

> But more to the point, so far you are the only one who is not
> saying that sendfile() needs to be fixed, instead of kludged.

By the way I do not see that Peter Jeremy said that sendfile() needs
to be fixed, he said only that my flags are needless and I should
prove their usefulness.


Igor Sysoev
http://sysoev.ru/en/