[mq_bpf] status report #4

[Takuya ASADA]

*Project summary
The project goal is to support multiqueue network interface on BPF,
and provide interfaces for multithreaded packet processing using BPF.
Modern high performance NICs have multiple receive/send queues and RSS
feature, this allows to process packet concurrently on multiple
processors.
Main purpose of the project is to support these hardware and get
benefit of parallelism.

*Subject changed
I changed status report subject from this report, from submitted date
to week number.
Here's old subjects and week number mapping:
Weekly status report (27th May) => week1
Weekly status report (6th June) => week2
Weekly status report (14th June) => week3

Here's status update from last week:
*multiqueue bpf emulation on SOFTRSS improvements
http://p4web.freebsd.org/@@195038?ac=10

In previous submit, IFCAP_MULTIQUEUE is assigned the interface when
multiqueue BPF enabled(BIOCENAQMASK).
I decided to not assign the flag, but assign IFCAP_SOFT_MULTIQUEUE.
So now we can identify it's real multiqueue NIC or software emulation.

Added rxqid/txqid setting code in ether_output_frame, to handle TX
queue handling.
(m->m_pkthdr.txqid is always 0, ifp->if_txq_num is 1 when SOFTRSS)

Added ifp->if_rxq_affinity/ifp->if_txq_affinity initialization code.

*Reduce lock contention on BPFIF_LOCK
http://p4web.freebsd.org/@@195044?ac=10
mtx replaced with rmlock on BPFIF_LOCK.
Looks working, but more tests and performance analysis are required.

*thoughts
Is there nwsid != cpuid situations? If so, which is used in netisr_select_cpuid?
If netisr_select_cpuid using real cpuid, when cpuid > nws_count
situation occurs, bpf_mtap refers outside of an array(on
d->qmask.qm_rxq_mask), kernel must die there.

Even if netisr_select_cpuid using nwsid, if there's nwsid != cpuid
situation, BIOCRXQAFFINITY should return the mapping info.

*thoughts2
Can we make situation rxqid != cpuid on multiqueue NIC by changing
MSI-x interrupt assignments?