physio and vmapbuf

[son goku]

Thx John for the information.
Seems weird to me that FreeBSD still requires kernel mapping. There must be
some performance hit due to the additional mapping which are not really
needed (devices after all mostly deal with bus relative address space which
is usually plain physical addresses unless there is some IOMMU). I know for
sure that most modern UNIX OS, and even Windows don't generate those mapping
unless the driver asked for it explicitly. I guess this is noticeable only
for raw I/O from user-mode, all other IO clients (such as file-system) are
kernel based so this shouldn't be an issue.
I now wonder whether FreeBSD is often used for running high-performance
database which use user-mode raw I/O...

Many thanks,
Goku.


On Wed, Mar 10, 2010 at 2:53 PM, John Baldwin <jhb at freebsd.org> wrote:

> On Wednesday 10 March 2010 6:40:18 am son goku wrote:
> > Hi hackers,
> > I have some experience with other UNIX kernels but none with FreeBSD.
> > FreeBSD interests me for a potential project I might be involved in , and
> > therefore I started researching it.
> > Browsing through the I/O layer code, I stumbled upon something that
> looked
> > strange, and perhaps you guys can help me with it.
> > Mainly, when reading the physio path, when the I/O buffer is from
> user-mode,
> > physio calls vmapbuf. The comments says that vmapbuf actually maps the
> > user-buffer into kernel address space. From my experience, such mapping
> is
> > needed only for very old devices that use PIO. Newer device just use DMA
> for
> > data copy, and therefore don't need the mapping, instead they just need
> to
> > pin the memory. Checking out vmapbuf only confused me further, I couldn't
> > find where such kernel mapping was taken place, nevertheless I did saw
> that
> > the buf->b_data was updated according to buf->b_saveaddr in some weird
> > offset calculation.
> > Adding to this confusion, there is a buf field called b_kvabase which
> > supposedly stores the kernel virtual address which wasn't touched at
> all!!
> >
> > Any help with understanding the flow and the usage of those
> fields/variables
> > will be greatly appreciated!!
>
> bus_dma (which drivers use to manage DMA), still uses virtual addressed
> buffers to build scatter/gather lists of physical addresses for DMA
> transfers.
> There has been much discussion and some prototype work to change this, but
> it's not a trivial project.  (I have converted physio to using bio's backed
> by
> sglist's with no kernel addresses in a prototype branch in p4 for example.)
>
> --
> John Baldwin
>