Re: About PHYS_TO_DMAP

Mitchell Horne <mhorne@freebsd.org> writes:

> On 4/14/23 04:31, Dmitry Salychev wrote:
>> Hi,
>> I'm struggling to understand which KVA will be returned by
>> PHYS_TO_DMAP
>> on arm64. For example, if I'll create a DMA tag this way:
>> 	bus_dma_tag_create(
>> 	    bus_get_dma_tag(dev),
>> 	    sc->buf_align, 0,		/* alignment, boundary */
>> 	    DMAP_MAX_PHYSADDR,		/* low restricted addr */
>> 	    DMAP_MIN_PHYSADDR,		/* high restricted addr */
>
> I think you are confused about the purpose of lowaddr and
> highaddr. They specify the window of bus-space addresses that are
> *inaccessible* *to the device* for DMA. It does not prevent you from
> providing a buffer backed by physical memory within this range, but in
> this case bounce pages will be used as an intermediate to perform the
> DMA transaction.
>
> Most commonly, if the device can only address a 32-bit range, then the
> values lowaddr=BUS_SPACE_MAXADDR_32BIT and highaddr=BUS_SPACE_MAXADDR
> will be given. If the entire bus range is accessible to the device for
> DMA, a zero-sized window will be given: lowaddr=BUS_SPACE_MAXADDR,
> highaddr=BUS_SPACE_MAXADDR.
>
> This is all described in adequate detail in busdma(9), but it is still
> not easily understood without a lot of code study, in my experience.
>

According to busdma(9), the window contains all addresses *greater than*
lowaddr and *less than or equal to* highaddr, i.e. your example with the
32-bit range looks like:


              allowed                      prohibited
    ------------------------------(xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
                            lowaddr                           highaddr
            BUS_SPACE_MAXADDR_32BIT                  BUS_SPACE_MAXADDR

But my example looks a bit different:

       prohibited              allowed                 prohibited
    xxxxxxxxxxxxxxxx]----------------------------(xxxxxxxxxxxxxxxxxxx>
             highaddr                      lowaddr
    DMAP_MIN_PHYSADDR            DMAP_MAX_PHYSADDR

I've found the only example of such DMA tag configuration in
src/sys/dev/vnic/nicvf_queues.c at line 379 (rbdr_buff_dmat).


>> 	    NULL, NULL,			/* filter, filterarg */
>> 	    BUF_SIZE, 1,		/* maxsize, nsegments */
>> 	    BUF_SIZE, 0,		/* maxsegsize, flags */
>> 	    NULL, NULL,			/* lockfunc, lockarg */
>> 	    &dmat);
>> in order to restrict any physical addresses but a window defined by
>> DMAP_MIN_PHYSADDR and DMAP_MAX_PHYSADDR. Later on when I'll be
>> mapping my mbuf (BUF_SIZE) with
>> 	bus_dmamap_load_mbuf_sg(dmat, dmap,
>> 	    m, &segs, &nsegs, BUS_DMA_NOWAIT);
>> I expect that
>> 	m->m_data == PHYS_TO_DMAP(segs[0].ds_addr)
>
> Why do you expect or need this to be the case?
>
> busdma is not responsible for setting or modifying m_data, it comes
> from wherever you allocated the mbuf. Calling
> bus_dmamap_load_mbuf_sg() will prepare a DMA mapping where the m_data
> buffer is used as the source/destination for the DMA transaction, but
> busdma does not allocate the buffer itself.
>

I don't need this to be the case exactly, but I'd like to be able to
access a "frame annotation" (64 bytes long) which is located exactly at
the start of m_data buffer having that physical address is provided, i.e.

	fa = (struct dpaa2_fa *) mbuf->m_data;
        /* ... fa populated ... */
        /* ... DMA transaction of the Tx frame (together with fa) ... */

        /* ... Tx confirmation from HW (bus address of the frame only) ...*/
        fa = (struct dpaa2_fa *) PHYS_TO_DMAP(paddr);

>> but it isn't true. Could somebody explain what exactly is returned
>> by
>> PHYS_TO_DMAP in this case and whether it's possible to translate
>> physical address to KVA as fast as possible (O(1) ideally).
>>
>
> PHYS_TO_DMAP is always a linear calculation of: physaddr + DMAP_MIN_ADDRESS.
>
> I do not think PHYS_TO_DMAP is in use at all in this example, or
> anywhere within busdma really.
>

Is there any other way to obtain KVA of a buffer mapped for DMA
transaction by the physical address? I've been crawling source code for
sometime already, but DMAP is the only thing I managed to find.

>> Regards,
>> Dmitry
>> --
>> Open source software/hardware enthusiast
>> hackaday.io/dsl | github.com/mcusim | patreon.com/salychev
>>

Regards,
Dmitry
-- 
Open source software/hardware enthusiast
hackaday.io/dsl | github.com/mcusim | patreon.com/salychev