Re: High CPU usage / high number of interrupts

From: Paul Procacci <pprocacci_at_gmail.com>
Date: Sun, 27 Nov 2022 04:45:46 UTC
What's catching my eye here is the values for ugen0.6.  It's a usb
bluetooth device.

With that in mind I searched the bug database for bluetooth usb and xhci.
I came across the following:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238235

There does seem to be some sort of weird interaction with bluetooth and
xhci as documented in the above link.
I believe the patch listed in that bug report made it to 13.1....so can you
try:

sysctl -w net.bluetooth.usb_isoc_enable=0


Does that make any difference?
(This btw is a wild guess just based on the values from the output you
provided me)

Thanks,
Paul

On Sat, Nov 26, 2022 at 11:05 PM 0x1eef <0x1eef@protonmail.com> wrote:

> The results are sort of interesting. At first, UE_INTERRUPT_OK was at 3k+
> for the USB mouse. I unplugged the mouse, and then the keyboard jumped from
> 0 to 1k+ for UE_INTERRUPT_OK.
>
> I have since reattached the mouse, and now both the mouse and the keyboard
> have a rising interrupt count. I would guess they jump by 20, or 30
> interrupts every 2-3 seconds, with the keyboard jumping with a higher
> frequency.
>
> Paste
>
> ugen0.1: <Intel XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER
> (5.0Gbps) pwr=SAVE (0mA)
>
> {
>     UE_CONTROL_OK       : 0
>     UE_ISOCHRONOUS_OK   : 0
>     UE_BULK_OK          : 0
>     UE_INTERRUPT_OK     : 0
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 0
>     UE_INTERRUPT_FAIL   : 0
> }
>
> ugen0.2: <Realtek USB 10/100/1000 LAN> at usbus0, cfg=0 md=HOST spd=HIGH
> (480Mbps) pwr=ON (350mA)
>
> {
>     UE_CONTROL_OK       : 7969
>     UE_ISOCHRONOUS_OK   : 0
>     UE_BULK_OK          : 11224
>     UE_INTERRUPT_OK     : 0
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 94
>     UE_INTERRUPT_FAIL   : 0
> }
>
> ugen0.4: <Sonix Technology Co., Ltd. Integrated Camera> at usbus0, cfg=0
> md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)
>
> {
>     UE_CONTROL_OK       : 10
>     UE_ISOCHRONOUS_OK   : 0
>     UE_BULK_OK          : 0
>     UE_INTERRUPT_OK     : 0
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 0
>     UE_INTERRUPT_FAIL   : 0
> }
>
> ugen0.6: <vendor 0x0cf3 product 0xe500> at usbus0, cfg=0 md=HOST spd=FULL
> (12Mbps) pwr=ON (100mA)
>
> {
>     UE_CONTROL_OK       : 16
>     UE_ISOCHRONOUS_OK   : 53214
>     UE_BULK_OK          : 0
>     UE_INTERRUPT_OK     : 9
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 0
>     UE_INTERRUPT_FAIL   : 0
> }
>
> ugen0.5: <vendor 0x30fa USB OPTICAL MOUSE> at usbus0, cfg=0 md=HOST
> spd=LOW (1.5Mbps) pwr=ON (100mA)
>
> {
>     UE_CONTROL_OK       : 11
>     UE_ISOCHRONOUS_OK   : 0
>     UE_BULK_OK          : 0
>     UE_INTERRUPT_OK     : 1707
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 0
>     UE_INTERRUPT_FAIL   : 0
> }
>
> ugen0.3: <SINO WEALTH Gaming KB> at usbus0, cfg=0 md=HOST spd=FULL
> (12Mbps) pwr=ON (500mA)
>
> {
>     UE_CONTROL_OK       : 21
>     UE_ISOCHRONOUS_OK   : 0
>     UE_BULK_OK          : 0
>     UE_INTERRUPT_OK     : 1790
>     UE_CONTROL_FAIL     : 0
>     UE_ISOCHRONOUS_FAIL : 0
>     UE_BULK_FAIL        : 0
>     UE_INTERRUPT_FAIL   : 0
> }
>
> Best,
> 0x1eef
>
> ------- Original Message -------
> On Sunday, November 27th, 2022 at 12:31 AM, Paul Procacci <
> pprocacci@gmail.com> wrote:
>
> usbconfig dump_stats
>
> Can you provide the output of the above? Run that a couple of times if you
> don't mind and do your very best to provide any rates of increases for any
> of the fields. An estimation would be perfectly fine.
>
> Thanks,
> Paul
>
> On Sat, Nov 26, 2022 at 10:09 PM 0x1eef <0x1eef@protonmail.com> wrote:
>
>> > Can you determine if irq 128 is being shared by any devices?
>>
>> I couldn't determine that from dmesg.boot, but I think there could be
>> some useful information in that file. I attached the file to this e-mail.
>> Thank you!
>>
>> Best,
>> 0x1eef
>>
>> Sent with Proton Mail <https://proton.me/> secure email.
>>
>> ------- Original Message -------
>> On Saturday, November 26th, 2022 at 8:49 PM, Paul Procacci <
>> pprocacci@gmail.com> wrote:
>>
>> Can you determine if irq 128 is being shared by any devices?
>> Usually this information can be found in `dmesg' or '/var/run/dmesg.boot'.
>>
>> vmstat indeed shows a device but it sometimes doesn't show all the
>> devices sharing that IRQ. It's possible you're being misled by vmstat.
>> Just trying to get the complete picture here of devices. ;)
>>
>> Thanks,
>> Paul Procacci
>>
>> On Sat, Nov 26, 2022 at 6:21 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>
>>> Hi !
>>>
>>> > Out of curiosity, have you pulled a usb device one by one until the
>>> interrupts disappear?
>>>
>>> I have three USB devices connected: mouse, keyboard, and an ethernet
>>> adapter.
>>> I tried to remove each one by one, and I did not see the interrupt rate
>>> change.
>>> I have also tried a cold boot without any USB devices connected, and the
>>> interrupt rate was about the same too.
>>>
>>> I don't know if it could be related, but there's a trackpad connected to
>>> the laptop that does not work. Maybe it has no relation to the issue, but
>>> setting "hw.psm.synaptics_support" to "0" also did not help.
>>>
>>> When Chromium loses focus, CPU usage usually drops to 0% and does not go
>>> above 10% - for as long as I am not using Chromium. I am using the i915 /
>>> drm kernel modules.. I saw another report of high CPU usage related to
>>> using those two kernel modules, but I wasn't able to identify that as the
>>> problem in my case.
>>>
>>> Thanks for the help.
>>>
>>> Sent with Proton Mail <https://proton.me/> secure email.
>>>
>>> ------- Original Message -------
>>> On Saturday, November 26th, 2022 at 8:06 PM, Paul Procacci <
>>> pprocacci@gmail.com> wrote:
>>>
>>> Hey,
>>>
>>> Not sure of the problem, but I don't see the correlation between Chrome
>>> and any usb driver.
>>> Out of curiosity, have you pulled a usb device one by one until the
>>> interrupts disappear?
>>>
>>> I'd be curious to know which device is slamming the system.
>>>
>>> Thanks,
>>> Paul
>>>
>>> On Sat, Nov 26, 2022 at 6:02 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>>
>>>> Hi, everyone!
>>>>
>>>> When I use Chromium, I see a high rate of CPU usage across all four
>>>> cores. The rate can be anywhere from 20% to 50%, even above that. I am not
>>>> doing anything intensive, just browsing twitter, reddit, YouTube or GitHub.
>>>> It has been like this since I installed FreeBSD, but since it's not a
>>>> blocker I have been lazy about looking into it.
>>>>
>>>> I don't know why it happens. I can see that there are a high number of
>>>> interrupts on 'xhci0', and that seems to carry over to each CPU core as
>>>> well:
>>>>
>>>> # vmstat -i
>>>> interrupt total rate
>>>> irq1: atkbd0 50 0
>>>> irq9: acpi0 403 0
>>>> cpu0:timer 30716618 98
>>>> cpu1:timer 25457926 81
>>>> cpu2:timer 34344531 109
>>>> cpu3:timer 25542867 81
>>>> irq128: xhci0 328107434 1044
>>>> irq130: nvme0:admin 15 0
>>>> irq131: nvme0:io0 701041 2
>>>> irq132: nvme0:io1 692045 2
>>>> irq133: nvme0:io2 792760 3
>>>> irq134: nvme0:io3 693091 2
>>>> irq135: hdac0 1718425 5
>>>> irq136: vgapci0 6273295 20
>>>> Total 455040501 1448
>>>>
>>>>
>>>> # dmesg | grep xhci0
>>>> xhci0: <Intel Ice Lake-LP USB 3.1 controller> mem 0x95110000-0x9511ffff
>>>> at device 20.0 on pci0
>>>> xhci0: 32 bytes context size, 64-bit DMA
>>>> usbus0 on xhci0
>>>>
>>>> It might also be helpful to know that I tried OpenBSD on the same
>>>> computer but it was unusable for a similar reason: 95%+ interrupts on CPU.
>>>> The impact that had made all tasks extremely slow. On FreeBSD it is not as
>>>> bad, but I still think think it is not normal.
>>>>
>>>> Can anyone suggest what might be wrong, tips to debug, etc ? If more
>>>> information is needed, please let me know. Thanks for your time.
>>>>
>>>> Best,
>>>> 0x1eef
>>>>
>>>>
>>>>
>>>
>>> --
>>> __________________
>>>
>>> :(){ :|:& };:
>>>
>>>
>>>
>>
>> --
>> __________________
>>
>> :(){ :|:& };:
>>
>>
>>
>
> --
> __________________
>
> :(){ :|:& };:
>
>
>

-- 
__________________

:(){ :|:& };: