CPU Cache and busdma usage in USB

[Hans Petter Selasky]

Hi Piotr,

On Thursday 09 July 2009 17:21:00 Piotr Zięcik wrote:
> Wednesday 08 July 2009 12:50:56 Hans Petter Selasky napisał(a):
> > By flush you mean write from CPU cache to RAM, right. And nothing else?
> > You don't mean discard CPU cache by "flush" ???
>
> Yes. By flush I mean write all valid and modified cache lines to RAM.
>
> > > However looking into logs which I have sent you yesterdat I see one
> > > difference which may be significant. My patch changes Invalidate into
> > > Writeback Invalidate. In original code if driver write something to
> > > memory and then invalidate cache
> >
> > If that is the case I'm very surprised. Could you make another printout
> > for me. Compile the kernel with KDB, and add a call to "kdb_backtrace()"
> > in bus_dmamap_sync() before printing out the writeback and/or invalidate,
> > so that I can see the backtrace.
> >
> > > , the write will be lost.
> >
> > I'm aware about that.
>
> Full log with backtraces is here:
> http://people.freebsd.org/~raj/logs/usb-cache.log

1) My analysis: Only the data areas are being flushed/invalidated. No transfer 
descriptors are flushed/invalidated. I see no cache operations happening on 
any DMA control structures, even though there are calls from EHCI to 
xxx_pc_flush() and xxx_pc_invalidate().

One patch you can try is to add an additional unload call to 
"usbd_transfer_setup_sub_malloc()":

==== //depot/projects/usb/src/sys/dev/usb/usb_transfer.c#159 - 
/home/hans_other/usb.p4/src/sys/dev/usb/usb_transfer.c ====
@@ -268,6 +268,10 @@
                    pg, z, align)) {
                        return (1);     /* failure */
                }
+
+               bus_dmamap_unload(parm->dma_page_cache_ptr->tag,
+                       parm->dma_page_cache_ptr->map);
+
                /* Set beginning of current buffer */
                buf = parm->dma_page_cache_ptr->buffer;
                /* Make room for one DMA page cache and one page */

This will avoid the same memory area being loaded twice. Not sure if there is 
still a bug in pmap about this!

2) You can also try to remove the "BUS_DMA_COHERENT" flag from "usb_busdma.c".

My time is limited today, because I have to work on libusb v1.0 support.

Could you provide a new trace, showing an enumeration failure. In your 
previous trace there was no error, because the printouts probably caused to 
CPU to flush out its cache.

Debug details: /boot/loader.conf

hw.usb.debug=15
hw.usb.ehci.debug=15
hw.usb.ohci.debug=15
hw.usb.uhci.debug=15

> I am using current from 2009.06.15. In later versions is bug in ARM pmap
> causing segmentation fault in ehci atatch code (for more details loot at
> http://www.nabble.com/pmap-problem-in-FreeBSD-current-td24352351.html).

Ok.

I have a feeling we will soon figure out the problem.

--HPS