8.1-STABLE: zfs and sendfile: problem still exists

[Andriy Gapon]

on 29/10/2010 16:14 Alexander Zagrebin said the following:
>>> I've noticed that ZFS on 8.1-STABLE still has problems with 
>> sendfile.
>>
>> Which svn revision, just in case?
> 
> 8.1-STABLE
> The source tree was updated 2010-10-27

OK, good.

>>> When accessing a file at first time the transfer speed is 
>> too low, but
>>> on following attempts the transfer speed is normal.
>>>
>>> How to repeat:
>>>
>>> $ dd if=/dev/random of=/tmp/test bs=1m count=100
>>> 100+0 records in
>>> 100+0 records out
>>> 104857600 bytes transferred in 5.933945 secs (17670807 bytes/sec)
>>> $ sudo env LC_ALL=C /usr/libexec/ftpd -D
>>>
>>> The first attempt to fetch file:
>>>
>>> $ fetch -o /dev/null ftp://localhost/tmp/test
>>> /dev/null                                       1% of  100 
>> MB  118 kBps
>>> 14m07s^C
>>> fetch: transfer interrupted
>>>
>>> The transfer rate is too low (approx. 120 kBps), but any 
>> subsequent attempts
>>> are success:
>>>
>>> $ fetch -o /dev/null ftp://localhost/tmp/test
>>> /dev/null                                     100% of  100 
>> MB   42 MBps
>>> $ fetch -o /dev/null ftp://localhost/tmp/test
>>> /dev/null                                     100% of  100 
>> MB   47 MBps
>>
>> Can you do an experiment with the same structure but sendfile 
>> excluded?
> 
> IMHO, ftpd hasn't an option to disable sendfile.

Seems so.
The source could be hacked (unconditional goto oldway in libexec/ftpd/ftpd.c, but
anyway.

> I've tried the nginx with
> disabled sendfile (the nginx.conf contains "sendfile off;"):
> 
> $ dd if=/dev/random of=test bs=1m count=100
> 100+0 records in
> 100+0 records out
> 104857600 bytes transferred in 5.892504 secs (17795083 bytes/sec)
> $ fetch -o /dev/null http://localhost/test
> /dev/null                                     100% of  100 MB   41 MBps
> $ fetch -o /dev/null http://localhost/test
> /dev/null                                     100% of  100 MB   44 MBps
> $ fetch -o /dev/null http://localhost/test
> /dev/null                                     100% of  100 MB   44 MBps
> 

I am really surprised with such a bad performance of sendfile.
Will you be able to profile the issue further?
I will also try to think of some measurements.

-- 
Andriy Gapon