8.1-STABLE: zfs and sendfile: problem still exists

[Kostik Belousov]

On Fri, Oct 29, 2010 at 06:31:21PM +0400, Alexander Zagrebin wrote:
> > > I've tried the nginx with
> > > disabled sendfile (the nginx.conf contains "sendfile off;"):
> > > 
> > > $ dd if=/dev/random of=test bs=1m count=100
> > > 100+0 records in
> > > 100+0 records out
> > > 104857600 bytes transferred in 5.892504 secs (17795083 bytes/sec)
> > > $ fetch -o /dev/null http://localhost/test
> > > /dev/null                                     100% of  100 
> > MB   41 MBps
> > > $ fetch -o /dev/null http://localhost/test
> > > /dev/null                                     100% of  100 
> > MB   44 MBps
> > > $ fetch -o /dev/null http://localhost/test
> > > /dev/null                                     100% of  100 
> > MB   44 MBps
> > > 
> > 
> > I am really surprised with such a bad performance of sendfile.
> > Will you be able to profile the issue further?
> 
> Yes.
> 
> > I will also try to think of some measurements.
> 
> A transfer rate is too low for the _first_ attempt only.
> Further attempts demonstrates a reasonable transfer rate.
> For example, nginx with "sendfile on;":
> 
> $ dd if=/dev/random of=test bs=1m count=100
> 100+0 records in
> 100+0 records out
> 104857600 bytes transferred in 5.855305 secs (17908136 bytes/sec)
> $ fetch -o /dev/null http://localhost/test
> /dev/null                                       3% of  100 MB  118 kBps
> 13m50s^C
> fetch: transfer interrupted
> $ fetch -o /dev/null http://localhost/test
> /dev/null                                     100% of  100 MB   39 MBps
> 
> If there was no access to the file during some time, then everything
> repeats:
> The first attempt - transfer rate is too low
> A further attempts - no problems
> 
> Can you reproduce the problem on your system?

Could it be the priming of the vm object pages content ?
Due to double-buffering, and (possibly false) optimization to only
perform double-buffering when vm object already has some data cached,
reads can prime vm object page list before file is mmapped or
sendfile-ed.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101029/4b1bc517/attachment.pgp