sendfile and page usage statistics

[Igor Sysoev]

On Wed, Oct 10, 2007 at 03:27:56AM -0700, David G Lawrence wrote:

> > > But I think that the actual patch is wrong. The PG_REFERENCED bit shall
> > > be set when hardware access bit is set on pte. You patch would set it in
> > > advance.
> > 
> > The problem is that ethernet card that does TX chsum, DMA and TSO never
> > sets PG_A, so sendfile() should set it in advance.
> > 
> > I'm going to test the patch on machine where all files takes more memory
> > than physical memory and there are some popular files.
> > 
> > I espect to see
> > 1) increased active memory.
> > 2) decreased number i/o.
> 
>    I would say that it is far from clear that this change would result in
> better performance. It is generally the case in FreeBSD that we bias in
> favor of executable pages (usually the image itself as its pages are
> executed, but also the image data pages). Pages involved in regular file
> I/O take a back seat. This is because a regular file is accessed typically
> in one burst, whereas an executable page is accessed in small chunks over
> a much larger period of time (and thus is affected more by a cache miss).
> Thus, PG_REFERENCED gets set for pages that are mmaped, but not for
> pages involved in read, write, sendfile, etc. Different workloads of
> course will have different behavior. 
>    One side effect of this will likely be much more system paging activity
> as sendfile causes process pages to be pushed out of memory in favor of
> cached file data. This is generally a bad thing and would result in reduced
> interactive performance.

The read() from file certainly updates vnode pages' PG_A/PG_REFERENCED,
because coping to userland is made by CPU. I think write() does the same.

I consider sendfile() as send() of mmap()ed file with the difference that 
user process does not see this mmap().


-- 
Igor Sysoev
http://sysoev.ru/en/