request for help: 'fixing' the 802.11 TX path

[PseudoCylon]

On Mon, Oct 29, 2012 at 3:47 AM, Andre Oppermann <oppermann at networx.ch> wrote:
> On 29.10.2012 04:53, Adrian Chadd wrote:
>>
>> On 28 October 2012 20:43, PseudoCylon <moonlightakkiy at yahoo.ca> wrote:
>>
>>> Cannot we just add custom hand off function to ieee80211_start()?
>>
>>
>> Yes. That's the general idea. But what I don't want to do is have it
>> just wake up the driver TX taskqueue - well, unless we have to.
>>
>> That means we'll have two context switches for each frame being
>> transmitted and that as a concept just sucks.

Plan B

vap->iv_ifp->if_transmit = ieee80211_transmit;

ieee80211_transmit() /* new */
{
        /* Alternatively, we can make a list of param and attach mbuf to it. */
        HANDOFF(parent->if_snd, m);

        ieee80211_new_tx(vap, m);
}

/* enque packet, but keep working on the same mbuf */
ieee80211_new_tx(vap, m)
{
        encap;
        if (fragment)
                insert_fragmented_packet_to_queue;

        /* don't forget about a fragmented packet */
        for (; m->m_nextpkt != NULL; m = m->m_nextpkt)
                parent->if_new_tx(vap, m);
}

/* keep working on the same mbuf */
driver_new_tx(vap, m)
{
        do_descriptor_stuff;

        m->m_flags |= ALL_SET;

        /*
         * If, for instance, processing of queue #5 packet finished
before queue #1,
         * #5 packet will stay in queue until all of preceding packets
get processed.
         */
        if (parent->if_sc->sc_tx == NOT_RUNNING &&
            ifq_head->m_flags & ALL_SET)
                driver_pass2hw(parent);
}

/* finally, process mbuf from the head of queue */
driver_pass2hw()
{
        /* only one thread to dequeue */
        if (atomic_compset(&sc->sc_tx, NOT_RUNNING, RUNNING) == 0)
                return;

        for (;;) {
                DEQUEUE(ifq, m);
                if (!(m->m_flags & ALL_SET)) {
                        PREPEND();
                        break;
                }

                /*
                 * want to do seq stuff somewhere in ieee80211_*(),
                 * but I guess this is the only place could do.
                 */
                do_seqnum_stuff;

                /* simply put a packet onto dma-able memory area */
                pass2hw;
        }
        sc->sc_tx = NOT_RUNNING;
}

No additional context switching, no long-held lock, but first queue first tx.


AK

>>
>> See my (very recent) email to -wireless - I broke TCP throughput quite
>> substantially by moving ath(4) TX into the taskqueue. I thought the
>> problem was _just_ going to be how overlapping, direct dispatch TX
>> could be preempted by the RX tasklet and TX completion, but there's
>> obviously more going on.
>
>
> I can't believe that TCP is getting broken by just introducing some
> additional delay in the TX path.  That can't add more than 300ms,
> can it?  There must be something else going on.  Most likely either
> severe packet loss (the m_nextpkt leak you mentioned earlier) or
> severe packet re-ordering.
>
> So don't rule out the TX taskqueue concept quite yet.
>
> --
> Andre
>