vfs.ffs.rawreadahead

[Igor Sysoev]

On Wed, Sep 03, 2008 at 10:44:46AM -0600, Scott Long wrote:

> On Wed, 3 Sep 2008, Igor Sysoev wrote:
> >On Wed, Sep 03, 2008 at 03:39:55PM +0300, Kostik Belousov wrote:
> >
> >>On Wed, Sep 03, 2008 at 01:53:52PM +0400, Igor Sysoev wrote:
> >>>Hi,
> >>>
> >>>could anyone tell what does vfs.ffs.rawreadahead enable ?
> >>>As I understand it's used in DIRECTIO code that allows read data
> >>>directly to an userland buffer bypassing the buffer cache.
> >>>What I can not understand where the read ahead data can be placed in ?
> >>
> >>The operation of the ffs_rawread is more accurately described as
> >>bypassing the page cache. It creates the physical buffer that maps
> >>the user pages.
> >>
> >>The readahead is performed only when the supplied user memory region
> >>is bigger then blocksize. In this case, two reads are performed
> >>simultaneously, with both buffers mapping consequent blocks from
> >>user-supplied buffers. The read operation looks like footsteps.
> >
> >Nice!
> >
> >As I understand the size limit of one read operation is MAXPHYS, which is
> >equal to 128K due to LBA28 ATA limit. On SCSI, SATA, and LBA48 ATA this 
> >limit
> >can be increased. Is it safe ?
> 
> The value of MAXPHYS is unrelated to capabilities or limitations of ATA.
> It was chosen based on the needs to prevent an excessive amount of
> parallel I/O from exhausting the kernel address space and system memory. 
> In fact, the concern was with SCSI, not with ATA.
> 
> MAXPHYS can be raised, especially on 64bit platforms, but doing so also
> bloats the sizes of a few key data structures.  I've been looking at a
> solution for this, and I'd rather that people keep their MAXPHYS changes
> confined to their local trees rather than changing FreeBSD unless they
> also solve the associated side effects.

As I understand MAXPHYS affects at least on pager_map size: on modern
machines it's usually 256 * MAXPHYS = 32M, therefore increasing MAXPHYS
will increase the map too.

The 128K is probably good value and I do not suggest to increase it by
default, I just want to increase MAXPHYS to improve disk throughput
on some hosts where nginx serves large files (1G+) using DIRECTIO.

BTW, is it possible to change MAXPHYS to a loader tunnable ?


-- 
Igor Sysoev
http://sysoev.ru/en/