sparc64/141918: [ehci] ehci_interrupt: unrecoverable error, controller halted (sparc64)

[Hans Petter Selasky]

On Tuesday 09 February 2010 00:45:43 Marius Strobl wrote:
> On Mon, Feb 08, 2010 at 07:24:54PM +0100, Hans Petter Selasky wrote:
> > On Monday 08 February 2010 19:05:49 Pyun YongHyeon wrote:
> > > On Mon, Feb 08, 2010 at 03:20:26PM +0100, Hans Petter Selasky wrote:
> > > > On Monday 08 February 2010 10:10:03 Marius Strobl wrote:
> > > > > The following reply was made to PR sparc64/141918; it has been
> > > > > noted by GNATS.
> > > > >
> > > > > From: Marius Strobl <marius at alchemy.franken.de>
> > > > > To: linimon at freebsd.org, bug-followup at freebsd.org, bel at orel.ru
> > > > > Cc:
> > > > > Subject: Re: sparc64/141918: [ehci] ehci_interrupt: unrecoverable
> > > > > error, controller halted (sparc64) Date: Mon, 8 Feb 2010 10:07:42
> > > > > +0100
> > > > >
> > > > >  On Mon, Feb 08, 2010 at 07:05:29AM +0000, linimon at freebsd.org 
wrote:
> > > > >  > hps claims that this may be sparc64-specific.
> > > > >
> > > > >  As outlined here it's unlikely that this is a problem of the
> > > > > sparc64 bus_dmamap_sync(9):
> > > >
> > > > Hi,
> > > >
> > > > > http://lists.freebsd.org/pipermail/freebsd-sparc64/2009-December/00
> > > > >6866 .ht ml There are however known problems with usb(4) in this
> > > > > regard, see for
> > > >
> > > > The issue mentioned above was patched in 9-current some months back.
> > > > Have you tried 9-current?
> > > >
> > > > >  http://svn.freebsd.org/viewvc/base?view=revision&revision=203080
> > > >
> > > > On point about busdma is that you should be able to pass any kernel
> > > > virtual address to be loaded into DMA. If the kernel virtual address
> > > > is not correctly aligned, a bounce page must be used, so that
> > > > surrounding memory is not disturbed. And that is not an USB problem.
> > >
> > > Would you elaborate on this? I don't think sparc64 needs bounce
> > > buffer as it uses DVMA. I have no idea how bounce buffer can
> > > address the alignment mismatches unless bounce buffer is created on
> > > certain alignment boundary.
> >
> > If you put the data in a bounce buffer, you can clear data before and
> > after the actual data without caring too much about it.
> 

Hi,

> Are you talking about the busdma implementation or the DMA engine/
> driver clearing data before/after the "actual" data? The latter two
> simply can't make such assumptions.

I'm talking about what busdma is doing. If there is a cache line requirement, 
then it should be handled by busdma, when it gets the busdma pointer. USB 
should not have to care about that.

> If the former amd64/i386 style bounce buffers won't happen on sparc64
> as this would require bypassing the IOMMU which is affected by hardware
> errata and thus is no viable option. 

If the kernel virtual buffer, start and end address, is placed so that the DMA 
engine cannot properly transfer data to/from it, without disrupting nearby 
data, then busdma should allocate a new buffer, and bounce forth and back.

> Is there way to avoid the alignment
> mismatches in usb(4)? 

On the one hand:

Not as long as we are allowed to be passed any kernel virtual data buffer 
pointer from the USB client driver. If you look back in the archives, that was 
one of the design decisions I was force to make, to support zero-copy.

On the other hand:

Yes, USB is sometimes using mbufs and other buffers passed to it which are 
correctly aligned.


Regarding alignment in USB:

All Host/Device controller structures are properly aligned. Alignment wise, 
the only problem is the data-part. 


Regarding busdma confusion:

It appears busdma might get confused if the same kernel virtual memory area is 
mapped multiple times into DMA. USB works like this, that it allocates a 
memory page of PAGE_SIZE bytes, which is loaded into DMA. Then the page is 
split into smaller parts (for USB DMA transfer descriptors and the alike), 
which are loaded into DMA a second time, because the bus_dmamap_sync() cannot 
operate on a region inside a busdma MAP.

We do this, to avoid excessive memory allocation. I.E. when allocating 64 
bytes you don't want to allocate 4K (PAGE_SIZE) bytes, because of problems 
regarding cache instructions.

You can try to #if 0 the following and see what happens:

#if 1
                /*
                 * XXX BUS-DMA workaround - FIXME later:
                 *
                 * We assume that that the aligment at this point of
                 * the code is greater than or equal to the size and
                 * less than two times the size, so that if we double
                 * the size, the size will be greater than the
                 * alignment.
                 *
                 * The bus-dma system has a check for "alignment"
                 * being less than "size". If that check fails we end
                 * up using contigmalloc which is page based even for
                 * small allocations. Try to avoid that to save
                 * memory, hence we sometimes to a large number of
                 * small allocations!
                 */
                if (size <= (USB_PAGE_SIZE / 2)) {
                        size *= 2;
                }
#endif


> Looking at usb_pc_alloc_mem() it seems to play
> with the alignment and size for no real good reason.

There are some tricks there, because busdma else will allocate much more 
memory than we ask for.

Again, I consider this to be a problem inside busdma.

BTW: I have no sparc64 hardware at hand to test on.

--HPS