busdma dflt_lock on amd64 > 4 GB
scottl at samsco.org
Thu Oct 27 07:03:40 PDT 2005
Jacques Caron wrote:
> At 20:01 26/10/2005, Søren Schmidt wrote:
> ATA does all the tag/map creates/allocs/loads for the SGlist and
>> simple workspace stuff at channel attach time, there are NO further
>> creates or allocs after that. So that is exactly what I would expect
>> the ALLOCNOW flags to make busdma support, if that doesn't work
>> busdma needs to be fixed IMNHO.
> So I tested with ALLOCNOW set on all 4 tags, and the end result is
> exactly the same, ch->dma->load fails (due to EINPROGRESS, most
> probably), and a panic is trigerred by busdma dflt_lock . And I agree
> that this is a busdma bug, as the busdma manpage says, in the
> description of bus_dmamap_load:
> EINPROGRESS The mapping has been deferred for lack of
> resources. The callback will be called as
> soon as
> resources are available. Callbacks are
> serviced in
> FIFO order. DMA maps created from DMA tags that
> are allocated with the BUS_DMA_ALLOCNOW flag
> never return this status for a load operation.
> So either the busdma API needs to be changed and ALLOCNOW no longer
> means that EINPROGRESS cannot happen (and that might break other
> things), or busdma needs to be fixed to conform to the documentation.
> Interestingly, if I remove the ALLOCNOW in all tag creations in ata, then:
> - ata attaches during boot are a bit longer (like fxp which is really long)
> - 320 bounces pages are allocated
> - I can do apparently do everything I want and not panic the machine.
> - I suppose that since there's plenty of available bounce pages, I
> should not have problems with ENOMEM.
> I'll go with that configuration for now, but that's most probably not
> the correct fix, just a workaround. The correct fix is to make busdma
> work as documented.
> Here I see several things to look into:
> - make sure enough ressources are actually allocated at tag create time
> if ALLOCNOW is set (this does not necessarily mean allocating maxsize
> additional pages, some logic about already required pages needs to be
> added I think)
> - let bus_dmamap_create allocate additional bounce pages if needed even
> if ALLOCNOW was set initially (i.e. don't use BUS_DMA_MIN_ALLOC_COMP),
> though I'm not sure the page allocation logic there is entirely correct
> either, as it will allocate additional pages even if there are already
> - in _bus_dmamap_load_buffer, consider ALLOCNOW like the undocumented
> NOWAIT, i.e. return ENOMEM if enough ressources aren't available (which
> I believe wouldn't be correct either: ALLOCNOW should prevent
> EINPROGRESS but also ENOMEM at map load time and should return ENOMEM
> immediately if there aren't enough ressources, not later).
> Another thing to look into is why bounce page allocation is that long
> (at least I believe that's what is taking so long), but I haven't looked
> at that bit at all yet.
> I also wonder if allocating bounce pages on <=4G configurations is
> really needed/useful? There are probably situations where this is
> needed, but I'm not sure I understand why.
It sounds like the only real bug in busdma is that it gets confused by
ALLOCNOW and doesn't allow enough pages to be allocated when
bus_dmamap_create() is called. I'll look into this.
More information about the freebsd-amd64