high CPU usage in FreeBSD for a PCIe card driver

[Steevan Rodrigues]

Hi

I  come from Linux background and new to FreeBSD.

I am investigating a high CPU usage issue on FreeBSD for PCIe driver of a
card . The card can process data at a rate of 30 to 50 Gbps . However, I
see very high CPU usage even when I run a single threaded user side
application which uses this card .

I ran hotkernel script from Dtrace toolkit.   A portion of the output is
given in the attached file  dtraceDiff.txt. Here I see that lock_delay
seems to be the reason for high CPU usage. Looks like it comes from mutex
locks

This driver uses  mutex locks ( mtx_lock).

The CPU usage is very high  on a multicore (Xeon CPU) server node than a
dual core (Intel i3) Desktop .

It shows 30 to 40 % CPU usage on a Intel i3 ( 2 core 4 threads) and 60 to
70% CPU usage  on a Xeon  CPU with 12 core (24 threads) for same output
data rate .

I wonder whether usage of mtx_lock is the cause of high CPU usage ? Any
pointers or suggestions welcome.

Using  FreeBSD 11.1 Release

Thanks

Steev
-------------- next part --------------
---------------------------------------------------

CPU usage details on a  12 core 24 thread server PC. 

part of the perl hotkernel  output 

kernel`free                                              1314   0.2%
kernel`_sx_slock                                         1415   0.3%
kernel`dev_relthread                                     1435   0.3%
kernel`devvn_refthread                                   1842   0.3%
kernel`_mtx_lock_spin_cookie                             2059   0.4%
kernel`copyin                                            2164   0.4%
kernel`amd64_syscall                                     2235   0.4%
kernel`__mtx_lock_spin_flags                             3566   0.6%
kernel`bzero                                             4889   0.9%
kernel`__mtx_lock_sleep                                  9323   1.7%
kernel`acpi_cpu_idle                                    11166   2.0%
kernel`pmap_extract                                     17491   3.1%
kernel`spinlock_exit                                    64850  11.5%
kernel`cpu_idle                                        134617  23.9%
kernel`lock_delay                                      263330  46.8%

-------------------------------------------------

CPU usage details on a  2 core 4 thread Intel i3 based Desktop PC. 

part of the perl hotkernel  output 


kernel`0xffffffff80                                       558   0.6%
kernel`copyout                                            592   0.6%
0xffffffff822368a3                                        639   0.7%
kernel`amd64_syscall                                      683   0.7%
kernel`__mtx_unlock_flags                                 728   0.8%
kernel`copyin                                            1311   1.4%
kernel`bzero                                             1434   1.5%
kernel`pmap_extract                                      1615   1.7%
kernel`lock_delay                                        2329   2.4%
0xffffffff822b0b5e                                       2528   2.6%
kernel`spinlock_exit                                    20950  21.8%
kernel`cpu_idle                                         48124  50.2%

------------------------------------------------------