8.1-STABLE: zfs and sendfile: problem still exists

[Andriy Gapon]

on 29/10/2010 18:26 Artemiev Igor said the following:
> On Fri, Oct 29, 2010 at 05:41:59PM +0300, Andriy Gapon wrote:
> 
>> What svn revision of FreeBSD source tree did you test?
> 
> r213936. Revision seems a little old.
> 
>> Ah, I think I see what's going on.
>> Either sendfile should (have an option to) use VOP_GETPAGES to request data or ZFS
>> mappedread should use vm_grab_page instead of vm_lookup_page for UIO_NOCOPY case.
>> Currently ZFS would read a whole FS block into ARC, but populate only one page
>> with data and for the rest it would just wastefully do uiomove(UIO_NOCOPY) from
>> ARC data.
>> So, e.g. zpool iostat would show that there are only few actual reads from a pool.
>>  The rest of the time must be spent churning over the data already in ARC and
>> doing page-per-VOP_READ copies from it.
> I can test it, but what allocflags? VM_ALLOC_RETRY|VM_ALLOC_NORMAL?

Probably yes, but have to be careful there.
First, do vm_page_grab only for UIO_NOCOPY case.
Second, the first page is already "shared busy" after vm_page_io_start() call in
kern_sendfile; so you might need VM_ALLOC_IGN_SBUSY for that page to avoid a deadlock.

I think that it may be good to separate UIO_NOCOPY/sendfile case from mappedread
into a function of its own.


P.S. doing VOP_GETPAGES instead of vn_rdwr() in kern_sendfile() might be a better
idea still.  But there are some additional details to that, e.g. a mount/fs flag
to tell which mechanism is preferred.  Because, as I've been told, vn_rdwr() has
better performance than VOP_GETPAGES.  Although, I don't understand why it
could/should be that way.

-- 
Andriy Gapon