[Bug 248652] netmap: pkt-gen TX huge pps difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

--- Comment #8 from Sylvain Galliano <sg at efficientip.com> ---
After looking at iflib_netmap_timer_adjust() & iflib_netmap_txsync() in
sys/net/iflib.c,
I made some tuning on kern.hz:

Still using X520 with 1 queue
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 1/2048, RX 1/2048

*****************

/boot/loader.conf:
kern.hz=1000  (default)

pkt-gen result:
204.153802 main_thread [2639] 2.562 Mpps (2.567 Mpkts 1.230 Gbps in 1001994
usec) 465.32 avg_batch 99999 min_space
205.155321 main_thread [2639] 2.561 Mpps (2.565 Mpkts 1.229 Gbps in 1001519
usec) 465.45 avg_batch 99999 min_space

5500 irq/s:

*****************

/boot/loader.conf:
kern.hz=1999

pkt-gen result:
41.375049 main_thread [2639] 5.117 Mpps (5.222 Mpkts 2.456 Gbps in 1020510
usec) 465.45 avg_batch 99999 min_space
42.375546 main_thread [2639] 5.118 Mpps (5.121 Mpkts 2.457 Gbps in 1000497
usec) 465.42 avg_batch 99999 min_space

11000 irq/s

X2 performance & irq/s

*****************

/boot/loader.conf:
kern.hz=2000

pkt-gen result:
797.608080 main_thread [2639] 2.560 Mpps (2.563 Mpkts 1.229 Gbps in 1001001
usec) 465.50 avg_batch 99999 min_space
798.609079 main_thread [2639] 2.560 Mpps (2.563 Mpkts 1.229 Gbps in 1000999
usec) 465.41 avg_batch 99999 min_space

5500 irq/s

Same performance & irq/s as kern.hz=1000 (due to limit at 2000 in
iflib_netmap_timer_adjust & iflib_netmap_txsync)

*****************

Last test, this one I forced 'ticks' parameter to '1' in callout_reset_on on
iflib_netmap_timer_adjust & iflib_netmap_txsync
by increasing the 2000 limit to 20000 in both functions
and put an insame value for kern.hz

/boot/loader.conf:
kern.hz=10000

pkt-gen result:
345.415939 main_thread [2639] 14.880 Mpps (14.890 Mpkts 7.142 Gbps in 1000699
usec) 430.97 avg_batch 99999 min_space
346.429134 main_thread [2639] 14.880 Mpps (15.076 Mpkts 7.142 Gbps in 1013196
usec) 432.17 avg_batch 99999 min_space

29000 irq/s

Same performance as FreeBSD 11

Looks like callout_reset_on to iflib_timer have a look high delay.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.