Partial cacheline flush problems on ARM and MIPS

[Warner Losh]

On Aug 23, 2012, at 5:59 PM, Adrian Chadd wrote:

> On 23 August 2012 16:55, Ian Lepore <freebsd at damnhippie.dyndns.org> wrote:
> 
>>> In what case though would the hardware say it can only DMA on a 64k
>>> alignment BUT move one byte at a time? Then what would the starting
>>> address be for each DMA?
>>> 
>> 
>> Maybe the device has a reduced number of address bits in its registers
>> and the low-order bits always start at zero and increment internally in
>> a wider register so that any length dma can happen, but it has to start
>> on a 64k boundary.
>> 
>> Maybe the address you pass it has to be a 64k boundary, but then the
>> bytes actually end up in one-of-N slots within that 64k region, based on
>> other parameters of the transfer.
>> 
>> I've worked with some strange hardware over the years.
> 
> Right. That's pretty odd though. But now I understand where you're coming from.
> 
> I still think the short term solution should be "fix the USB stack to
> not do that!"
> 
> The longer term problem is likely to figure out what makes sense. Eg,
> if you're going to allow the allocator to interleave 16 byte chunks
> (on a 32 byte cache line platform) with some being DMA buffers and
> others being non-DMA buffers; or whether you enforce that the whole
> chunk is a DMA buffer for your hardware and you look after it, or
> something else..

The bottom line is that you can't mix things like that when cache lines are involved.  The current code that tries is doomed to failure. Doomed. You just can't control all flushes, as Ian's missive demonstrates, and trying to accommodate code that does this I don't think can possibly work.  All the interrupt masking, copying in and out, etc I fear is doomed to utter and abject failure.  

Warner