busdma dflt_lock on amd64 > 4 GB

[Scott Long]

Jacques Caron wrote:
> Hi,
> 
> At 20:01 26/10/2005, Søren Schmidt wrote:
> ATA does all the tag/map creates/allocs/loads for the SGlist and
> 
>> simple workspace stuff at channel attach time, there are NO further
>> creates or allocs after that. So that is exactly what I would expect
>> the ALLOCNOW flags to make busdma support, if that doesn't work
>> busdma needs to be fixed IMNHO.
> 
> 
> So I tested with ALLOCNOW set on all 4 tags, and the end result is 
> exactly the same, ch->dma->load fails (due to EINPROGRESS, most 
> probably), and a panic is trigerred by busdma dflt_lock . And I agree 
> that this is a busdma bug, as the busdma manpage says, in the 
> description of bus_dmamap_load:
> 
>               EINPROGRESS  The mapping has been deferred for lack of
>                            resources.  The callback will be called as 
> soon as
>                            resources are available.  Callbacks are 
> serviced in
>                            FIFO order.  DMA maps created from DMA tags that
>                            are allocated with the BUS_DMA_ALLOCNOW flag 
> will
>                            never return this status for a load operation.
> 
> So either the busdma API needs to be changed and ALLOCNOW no longer 
> means that EINPROGRESS cannot happen (and that might break other 
> things), or busdma needs to be fixed to conform to the documentation.
> 
> Interestingly, if I remove the ALLOCNOW in all tag creations in ata, then:
> - ata attaches during boot are a bit longer (like fxp which is really long)
> - 320 bounces pages are allocated
> - I can do apparently do everything I want and not panic the machine.
> - I suppose that since there's plenty of available bounce pages, I 
> should not have problems with ENOMEM.
> 
> I'll go with that configuration for now, but that's most probably not 
> the correct fix, just a workaround. The correct fix is to make busdma 
> work as documented.
> 
> Here I see several things to look into:
> - make sure enough ressources are actually allocated at tag create time 
> if ALLOCNOW is set (this does not necessarily mean allocating maxsize 
> additional pages, some logic about already required pages needs to be 
> added I think)
> - let bus_dmamap_create allocate additional bounce pages if needed even 
> if ALLOCNOW was set initially (i.e. don't use BUS_DMA_MIN_ALLOC_COMP), 
> though I'm not sure the page allocation logic there is entirely correct 
> either, as it will allocate additional pages even if there are already 
> enough
> - in _bus_dmamap_load_buffer, consider ALLOCNOW like the undocumented 
> NOWAIT, i.e. return ENOMEM if enough ressources aren't available (which 
> I believe wouldn't be correct either: ALLOCNOW should prevent 
> EINPROGRESS but also ENOMEM at map load time and should return ENOMEM 
> immediately if there aren't enough ressources, not later).
> 
> Another thing to look into is why bounce page allocation is that long 
> (at least I believe that's what is taking so long), but I haven't looked 
> at that bit at all yet.
> 
> I also wonder if allocating bounce pages on <=4G configurations is 
> really needed/useful? There are probably situations where this is 
> needed, but I'm not sure I understand why.
> 
> Jacques.
> 
> 

It sounds like the only real bug in busdma is that it gets confused by 
ALLOCNOW and doesn't allow enough pages to be allocated when 
bus_dmamap_create() is called.  I'll look into this.

Scott