8.1-STABLE: zfs and sendfile: problem still exists

[Artemiev Igor]

On Thu, Oct 28, 2010 at 09:57:22AM +0400, Alexander Zagrebin wrote:
> Hi!
> 
> I've noticed that ZFS on 8.1-STABLE still has problems with sendfile.
> When accessing a file at first time the transfer speed is too low, but
> on following attempts the transfer speed is normal.
...
> I've tried ftpd and nginx with "sendfile on". The behavior is the same.
> After disabling using sendfile in nginx ("sendfile off") the problem has
> gone.

Yep, this problem exists. You may workaround it via bumping up
net.inet.tcp.sendspace up to 128k.  zfs sendfile is very ineffective. I have
made a small investigation via DTrace, it reads MAXBSIZE chunks, but map in vm
only one page (4K).  I.e. if you have a file with size 512K, sendfile make
calls freebsd_zfs_read 128 times.