sendfile(2) SF_NOPUSH flag proposal
rsharpe at richardsharpe.com
Tue Jun 17 08:09:39 PDT 2003
On Tue, 17 Jun 2003, Igor Sysoev wrote:
> On Mon, 16 Jun 2003, Richard Sharpe wrote:
> > While I was chasing down a performance problem with Samba using sendfile
> > on FreeBSD 4.6.2, I changed sendfile to:
> > 1. Use sosend for the header, and
> > 2. Not to push the header out if there was data following (by passing
> > MSG_MORE to sosend, and maybe frobbing sosend to do the right things)..
> > I was also using TCP_NODELAY, and sendfile was being used to to handle SMB
> > Read&X calls only.
> > The performance impact of doing this as measured by tests like NetBench
> > was negligible.
> > I did not test raw throughput (as NetBench is not really
> > about raw throughput), but I suspect that it would not make much
> > difference either.
> The sending the header and the first file part in the separate packets
> is the one part of the problem. The second part is that file pages can
> be sent in incomplete packets, e.g. three pages can be sent as
> 1460, 1460, 1176, 1460, 1460, 1176, 1460, 1460, ...
> or as 1460, 1460, 1460, 1460, 1460, 892, 1460, 1460, ...
> And this can be fixed by PRUS_MORETOCOME flag while sending the pages.
Yes, I noticed that.
What I didn't emphasize enough was that in my environment, ie an SMB
server (CIFS, or Samba) and NetBench, the majority of the requests I care
about are reads of less than 5100 bytes or so, and a very large proportion
of them are less than 1448 bytes (we use SACK). So the less than full
segment every third segment is not such an issue for me.
This is especially so when I have bigger problems to worry about, like
moving much of the SMB server into the kernel, making NFSv4 and SMB use
much of the same infrastructure, and creating a unified credentials system
where I can carry UIDs/GIDS and/or SIDs and/or KRB tickets around in the
> > I also modified sendfile so that it uses VOP_GETPAGES rather than
> > VOP_READ, and this had more impact, I believe.
> What is the difference of these operation ?
It was a throw away comment. Please ignore it.
Richard Sharpe, rsharpe[at]ns.aus.com, rsharpe[at]samba.org,
More information about the freebsd-arch