busdma dflt_lock on amd64 > 4 GB

Jacques Caron jc at oxado.com
Thu Oct 27 05:37:11 PDT 2005


At 20:01 26/10/2005, Søren Schmidt wrote:
ATA does all the tag/map creates/allocs/loads for the SGlist and
>simple workspace stuff at channel attach time, there are NO further
>creates or allocs after that. So that is exactly what I would expect
>the ALLOCNOW flags to make busdma support, if that doesn't work
>busdma needs to be fixed IMNHO.

So I tested with ALLOCNOW set on all 4 tags, and the end result is 
exactly the same, ch->dma->load fails (due to EINPROGRESS, most 
probably), and a panic is trigerred by busdma dflt_lock . And I agree 
that this is a busdma bug, as the busdma manpage says, in the 
description of bus_dmamap_load:

               EINPROGRESS  The mapping has been deferred for lack of
                            resources.  The callback will be called as soon as
                            resources are available.  Callbacks are serviced in
                            FIFO order.  DMA maps created from DMA tags that
                            are allocated with the BUS_DMA_ALLOCNOW flag will
                            never return this status for a load operation.

So either the busdma API needs to be changed and ALLOCNOW no longer 
means that EINPROGRESS cannot happen (and that might break other 
things), or busdma needs to be fixed to conform to the documentation.

Interestingly, if I remove the ALLOCNOW in all tag creations in ata, then:
- ata attaches during boot are a bit longer (like fxp which is really long)
- 320 bounces pages are allocated
- I can do apparently do everything I want and not panic the machine.
- I suppose that since there's plenty of available bounce pages, I 
should not have problems with ENOMEM.

I'll go with that configuration for now, but that's most probably not 
the correct fix, just a workaround. The correct fix is to make busdma 
work as documented.

Here I see several things to look into:
- make sure enough ressources are actually allocated at tag create 
time if ALLOCNOW is set (this does not necessarily mean allocating 
maxsize additional pages, some logic about already required pages 
needs to be added I think)
- let bus_dmamap_create allocate additional bounce pages if needed 
even if ALLOCNOW was set initially (i.e. don't use 
BUS_DMA_MIN_ALLOC_COMP), though I'm not sure the page allocation 
logic there is entirely correct either, as it will allocate 
additional pages even if there are already enough
- in _bus_dmamap_load_buffer, consider ALLOCNOW like the undocumented 
NOWAIT, i.e. return ENOMEM if enough ressources aren't available 
(which I believe wouldn't be correct either: ALLOCNOW should prevent 
EINPROGRESS but also ENOMEM at map load time and should return ENOMEM 
immediately if there aren't enough ressources, not later).

Another thing to look into is why bounce page allocation is that long 
(at least I believe that's what is taking so long), but I haven't 
looked at that bit at all yet.

I also wonder if allocating bounce pages on <=4G configurations is 
really needed/useful? There are probably situations where this is 
needed, but I'm not sure I understand why.


More information about the freebsd-amd64 mailing list