current meaning of BUS_DMA_COHERENT

[Ian Lepore]

On Fri, 2015-03-20 at 21:31 -0400, John Wehle wrote:
> On Mar 20, 2015, at 8:27 AM, Ian Lepore <ian at freebsd.org> wrote:
> > The main problem is shared concurrent-access memory, such as buffer
> > descriptor rings which are accessed simultaneously by the cpu and
> > network hardware.  (The same thing happens with other hardware, but NICs
> > are the prime example of it.)
> > ...
> > What we really need is a new type of busdma memory (BUS_DMA_DESCRIPTOR)
> > and a special sync call to use in conjunction with it that takes an
> > offset and length, and the sync is a single operation, no pre/post
> > stuff.
> 
> If you have descriptor A being modified by the device at the same time
> descriptor B is modified by the CPU and they are in the same cache line
> you'll still have problems even with a sync call which takes an offset
> and length.
> 

Not if the memory is uncached, which shared-descriptor memory must
necessarily be on the arm systems we currently support.  Just because
it's uncached doesn't mean you need no sync ops, though.  In particular,
you likely still need some sort of drain-write-buffer action and/or
memory barrier operations.

> Doesn't being able to sync individual descriptors also require padding the
> descriptor size so it's a multiple of the cache line size?

Typically the descriptor size and packing into an array or ring
structure is dictated by the hardware and you're not free to change it.

-- Ian