About the "USB Cache and busdma usage in USB" thread
gjb at semihalf.com
Tue Aug 4 15:01:17 UTC 2009
Hans Petter Selasky wrote:
> CC'ed current: We have a case on ARM where bus_dmamap_sync() is not suffient
> to update the CPU cache. One reason for this is that USB needs to invalidate
> the same memory area multiple times. Busdma sync expects paired operation when
> using the PRE and POST flags, from what I understand. I do not consider this
> an USB issue, hence Semihalf has got the USB stack working by manually
> inserting CPU flush/invalidate calls into usb_pc_cpu_invalidate() and
> usb_pc_cpu_flush(). Their other solution however which modifies the
> bus_dmamap_sync() flags will break on platforms with more than 4 GByte of
> Maybe Rafal can give a quick summar to new people at the -current list, or see
> previous thread on the ARM mailing list.
> USB needs a solution where it can call a function given a busdma mapping,
> preferably with an offset and length, which handles the cache sync issue and
> works with bounce pages on +4GB systems.
New USB stack uses busdma in a little unconventional way. As you
mentioned in one of previous mails your assumptions are:
XXX_PREXXX functions should be used prior to read/write device access.
In other words, PRE has to be a flush operation.
XXX_POSTXXX functions should be used after read/write device access.
In other words, POST has to be an invalidate operation.
Generally it is true, but if you look at ARM code you will find out that
it is not that simple. You assumed that after
bus_dmamap_sync(..,BUS_DMASYNC_POSTWRITE | BUS_DMASYNC_POSTREAD) there
will be no data in cache, but it that's not true.
Cache operation are performed on cache lines (32 bytes on our ARM
device). Let's say you want to invalidate buffer with size 10 bytes. In
this case first whole cache line is invalidated ( and now all
requirements related to busdma synchronization are fulfilled, old
contents of cache is gone). The second step is to restore back into
cache 22 bytes of data which were not a part of buffer. After this
second step data are loaded into cache line (it is because our device
uses write allocate feature).
So busdma on ARM "Perform any synchronization required after an update
of host memory by the device", but we still end up with not invalidated
It is hard to fix it. We cannot just invalidate whole cache line. We
cannot also use cpu_dcache_wbinv, because this function is called after
buffer was used by device so we dont want to overwrite those data with
old cache contents.
One possible solution is to call first
bus_dmamap_sync(..,BUS_DMASYNC_POSTREAD) and then
bus_dmamap_sync(..,BUS_DMASYNC_PREREAD) in usb_pc_cpu_invalidate(), but
this is ugly workaround which applies probably only to ARM case.
The second problem is that you cannot use cpu_dcache_wb(inv) function
directly because you need to handle bounce pages in USB code. I think
that duplication of busdma code makes no sense. Probably it takes less
work to add bus_dmamap_sync() before/after each transaction.
Could you give us a quick overview of buffer handling in USB stack? I
want to understand what is the relation between
usb_pc_cpu_invalidate/flush() functions and reading/writing to USB
device? From yours previous mail I understand that invalidate is called
*before* reading and flush *before* writing. Is that true? Can we add a
functions which will be called *after* reading/writing?
If you have any questions regarding cache operation on ARM. please let
me know, I will try to answer them.
More information about the freebsd-arm