Partial cacheline flush problems on ARM and MIPS

[Warner Losh]

On Aug 27, 2012, at 9:32 AM, Ian Lepore wrote:

> On Sun, 2012-08-26 at 17:13 -0600, Warner Losh wrote:
>> On Aug 26, 2012, at 12:25 PM, Ian Lepore wrote:
>>> In this regard, it's the busdma implementation that's broken, because it
>>> should bounce those IOs through a DMA-safe buffer.  There's absolutely
>>> no rule that I've ever heard of in FreeBSD that says IO can only take
>>> place using memory allocated from busdma.
>> 
>> That's partially true.  Since BUSDMA grew up in the storage area, you must allocate the memory from busdma, or it must be page aligned has been the de-facto rule here.  
> 
> Where does anything say that you must allocate the memory from busdma if
> it's not mbuf/page-aligned?  I've never seen it in any docs.  I
> certainly find contrary evidence in existing driver code (and I'm not
> talking about USB here).

Where does it say that you are allowed to not use the routines?

>> The mbuf and uio variants of load were invented to cope with common cases of mbufs and user I/O to properly flag things.
>> 
> 
> What does that mean, "to properly flag things"?
> 
> I think with uio we come to the crux of the issue.  Userland buffers are
> not allocated with knowledge of the busdma constraints.  That leads to
> some choices:
> 
>      * We don't support IO with a userland buffer at all.

You may have to bounce here.

>      * We support userland IO if the userland buffers are accidentally
>        aligned properly for the platform, otherwise the call fails.

You may have to bounce here.

>      * We support userland buffers by making any driver which wants to
>        do so responsible for always copying the data to aligned buffers
>        (each individual driver handles bounces).

This is what most drivers that want to do zero copying do.

>      * We support userland buffers by having the busdma layer handle
>        the bouncing when required.

I thought that's what the uio variants of the map loading routines were supposed to do.

> The first two seem untenable to me.  The third one comes down to "every
> driver must always bounce all userland data" because the driver doesn't
> actually have access to the info to decide whether a userland buffer is
> aligned properly or not for a given platform.
> 
> That leaves the option where the busdma layer handles bouncing for
> unaligned userland buffers.  If that's possible, then the busdma layer
> could automatically bounce any unaligned request, whether it came from
> userland or a driver reading a status response into an unaligned local
> buffer in kernel memory.

Right, the uio load option is supposed to do this.

>> How does busdma know that it is using memory that's not from its allocator?
> 
> The busdma code allocates the map along with the buffer, and can record
> information in the map that it can use during mapping and sync
> operations to know whether it allocated the buffer.
> 
>>> The rule is only that the
>>> proper sequence of busdma operation must be called, and beyond that it's
>>> up to the busdma implementation to make it work.  
>> 
>> No.  Bouncing is needed due to poor alignment of the underlying device.  Not due to cache effects.
> 
> Says who?  This statement seems to me to be more about opinion and dogma
> than about a documented API or technical concerns about how to implement
> it.  IMO, bouncing is needed when that's the only way to make a
> particular busdma sequence work correctly given the parameters of the
> hardware and the transfer.

Says the people that wrote and use busdma?  There's no requirement for cache effects for the transfer, so there's no constraints for the transfer.  The requirement is about having a buffer that's suitable for DMA.  This means either the buffer (not the start of the transfer) be aligned to a cache line and be an integral number of cache lines in size, or it means that you have 100% perfect hygiene when it comes to ensuring the CPU touches the buffer or the device touches it and ever have cross threading.

I'd agree that better docs here would help.

> To put it another way, the busdma subsystem is responsible for helping
> make DMA transfers work on a platform without every driver needing to
> contain platform-specific code, and bouncing is one of the techniques
> available to it.

Buffers allocated through the busdma system do this.

>> There's a limited number of things that we support with busdma.  Arbitrary data from malloc that might be shared with the CPU isn't on that list.
>> 
> 
> Where are those limitations documented?  

Where is it documented that memory returned from malloc may be used for DMA?

I agree that docs could be better here.

> This isn't some small oversight in the documention, if the rule is "IO,
> any IO at all, can only be performed using buffers allocated from
> busdma" that's a pretty major thing that ought to appear in multiple
> manpages relating to driver development.
> 
> If you don't think "any IO at all" is the right characterization, read
> all the way to the end before responding.

Yes.  I'd state it another way.  "Buffers returned from the busdma allocation routines is always safe." Everything else may or may not be safe.  mbufs, for example, happen to be safe because there is a rigid protocol for sharing between the driver and the device.  uio buffers can be made safe.  busdma has routines to ensure that these types of data are handled properly.

>>> Our biggest problem, I think, is that we don't have a sufficient
>>> definition of "the proper sequence of busdma operations."
>> 
>> I disagree.  The sequence has been known for a long time.
>> 
>>> I don't think it will be very hard to make the arm and mips busdma
>>> implementations work correctly.  It won't even be too hard to make them
>>> fairly efficient at bouncing small IOs (my thinking is that we can make
>>> small bounces no more expensive than the current partial cacheline flush
>>> implementation which copies the data multiple times).  Bouncing large IO
>>> will never be efficient, but the inefficiency will be a powerful
>>> motivator to update drivers that do large IO to work better, such as
>>> using buffers allocated from busdma.
>> 
>> I don't think the cache line problem can be solved with bounce buffers.  Trying to accommodate broken drivers is what lead us to this spot.  We need to fix the broken drivers.  If that's impossible, then the best we can do is have the driver set a 'always bounce' flag in the tag it creates and use that to always bounce for operations through that tag.
> 
> So you'd be okay with a driver setting a flag that says "always bounce"
> but not okay with the busdma layer bouncing only when it's actually
> necessary?  I'm confused -- do you think the busdma layer will be unable
> to detect when it's necessary unless directed from the outside?

Actually, it is good you are confused.  I was wrong when I said I'd be happy with a flag that says always bounce. The reason is that the driver can do this all the time for those cases like the at91 mci driver does.

> Let me pose a question back to you... if it is up to a driver to
> allocate DMA buffers using the busdma allocator, how is any given driver
> to know whether DMA is going to be involved in the transfer?  

because it does the dma?

> Don't think USB or ATA here... think iicbus/foo_temperature_sensor, or
> the mmc/sd driver.  Those drivers know nothing about the hardware bridge
> layers beneath them and whether DMA is involved or not.  They just know
> "I need to read a 2-byte value from a register on this IIC chip" or "I
> need to retrieve the 32-bit CSD data from the SD card" and they
> accomplish that by calling a read method of some lower-level bridge
> driver.  In each of those cases we have both PIO and DMA implementations
> of the lower-level drivers that move bits over a wire.

The bridge here would know and have to cope.  However, the real issue in this case wouldn't be the transfer alignment, but what traffic might be happening to other parts of the cache line.

For small things like this, typically data copies are involved.  There's a few places that break these rules (I think mci might be breaking the rules for small transfers, for example).

> If the concensus is that such drivers should always allocate all IO
> buffers using busdma, even though they have no idea whether DMA is going
> to be involved, then we certainly have a lot of work ahead of us in
> terms of "fixing broken drivers."  That also implies that
> bus_dmamem_alloc() is misnamed and implemented in the wrong place,
> because it's not really about dma at all.

You can't get around the hardware requirement that any I/O going to the device cannot share cache lines with other data on the system.

> If you push the responsibility down to the layer where it is known
> whether DMA is going to be involved, then we're back to the need for
> many individual drivers to bounce every request, because they don't have
> access to the info needed to know whether bouncing is required or not
> for a given buffer they were handed from higher layers.  (Remember when
> I proposed exporting that info to drivers so they could decide whether a
> buffer is dma-aligned or not, the response was "No, keep it all hidden
> in the busdma layer," which I think is the right response.)

Right now we support only a few things doing DMA on the system.  We support pages, mbufs and to a limited extent uio buffers (but to be honest, we may have exposure on smaller transfers here).

> Or if you push the responsibility down to the busdma layer where all
> that info resides, all drivers which use DMA and adhere to the busdma
> rules just work, without needing to know anything about the buffers that
> were handed to them and without needing to know anything about the
> platform requirements.

I'm not sure I follow this last bit...

Warner