[Bug 248652] iflib: netmap pkt-gen large TX performance difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

--- Comment #16 from Sylvain Galliano <sg at efficientip.com> ---
(In reply to Vincenzo Maffione from comment #14)

Here are the results:

X520 with 1 queue
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 1/2048, RX 1/2048

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=0  (default value)

pkt-gen:
683.502433 main_thread [2639] 4.215 Mpps (4.227 Mpkts 2.023 Gbps in 1002819
usec) 465.43 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=300

pkt-gen:
750.688608 main_thread [2639] 6.496 Mpps (6.646 Mpkts 3.118 Gbps in 1023000
usec) 465.45 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=200

pkt-gen:
771.736855 main_thread [2639] 8.907 Mpps (9.112 Mpkts 4.275 Gbps in 1022999
usec) 465.45 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=100

pkt-gen:
804.554603 main_thread [2639] 14.136 Mpps (14.147 Mpkts 6.785 Gbps in 1000748
usec) 465.45 avg_batch 99999 min_space
-> close to 10G line rate

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=90

pkt-gen:
872.156329 main_thread [2639] 14.880 Mpps (15.054 Mpkts 7.142 Gbps in 1011721
usec) 466.96 avg_batch 99999 min_space



Now using same X520 NIC using 4 queues.

ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048

*******************************************

sysctl dev.ix.1.iflib.nm_tx_tmr_us=0 (default)

pkt-gen:
047.988586 main_thread [2639] 13.596 Mpps (13.623 Mpkts 6.526 Gbps in 1002002
usec) 443.03 avg_batch 99999 min_space
-> close to max speed (thanks to 4 queue)

*******************************************

sysctl dev.ix.1.iflib.nm_tx_tmr_us=400

pkt-gen:
094.224581 main_thread [2639] 14.887 Mpps (14.904 Mpkts 7.146 Gbps in 1001173
usec) 440.75 avg_batch 99999 min_space


Looks really good for X520 NIC whatever the number of queue I used.



Now same tests using XL710 NIC (40G) using 1 queue:

ixl1: PCI Express Bus: Speed 8.0GT/s Width x8
ixl1: netmap queues/slots: TX 1/1024, RX 1/1024

*******************************************

sysctl dev.ixl.1.iflib.nm_tx_tmr_us=0 (default)

pkt-gen:
324.883066 main_thread [2639] 12.270 Mpps (13.044 Mpkts 5.890 Gbps in 1063000
usec) 16.53 avg_batch 99999 min_space

*******************************************

sysctl dev.ixl.1.iflib.nm_tx_tmr_us=100

pkt-gen:
350.497566 main_thread [2639] 12.246 Mpps (12.258 Mpkts 5.878 Gbps in 1001003
usec) 16.48 avg_batch 99999 min_space

no changes.


Now testing XL710 with 4 queues:

ixl0: PCI Express Bus: Speed 8.0GT/s Width x8
ixl0: netmap queues/slots: TX 4/1024, RX 4/1024

*******************************************

sysctl dev.ixl.0.iflib.nm_tx_tmr_us=0 (default)

pkt-gen:
614.766048 main_thread [2639] 13.671 Mpps (14.539 Mpkts 6.562 Gbps in 1063494
usec) 15.75 avg_batch 99999 min_space

*******************************************

sysctl dev.ixl.0.iflib.nm_tx_tmr_us=100

pkt-gen:
640.652549 main_thread [2639] 13.672 Mpps (13.795 Mpkts 6.562 Gbps in 1009001
usec) 15.79 avg_batch 99999 min_space


No changes using XL710 NIC (as a reminder, using FreeBSD 11 without iflib, I
can reach +40Mpps on XL710 using pkt-gen)

-- 
You are receiving this mail because:
You are on the CC list for the bug.