Re: High CPU usage / high number of interrupts

From: 0x1eef <0x1eef_at_protonmail.com>
Date: Mon, 28 Nov 2022 22:44:27 UTC
> man eventtimers

That's good to know!

> When I add up all my timer values, their sum is slightly over 1300 so I'm not entirely convinced it's a problem.

Also good to know.

> I'm running around quite a bit today so I'll look a bit later to see if anything stands out and let you know.

No worries. You have already helped a ton. God willing, I will see if the event timers man page leads anywhere. If not, it's not the end of the world. My system is not unusable because of this problem, but the CPU usage does seem a bit high for what I'd expect to be normal.

Thanks!

0x1eef

Sent with [Proton Mail](https://proton.me/) secure email.

------- Original Message -------
On Sunday, November 27th, 2022 at 6:29 PM, Paul Procacci <pprocacci@gmail.com> wrote:

> I've personally never had to fiddle with event timers so this may be out of my realm.
> With that said the following man page describes that various timers that one could employ:
>
> man eventtimers
> When I add up all my timer values, their sum is slightly over 1300 so I'm not entirely convinced it's a problem.
> I'm running around quite a bit today so I'll look a bit later to see if anything stands out and let you know.
> Thanks,
> Paul
>
> On Sun, Nov 27, 2022 at 1:12 PM 0x1eef <0x1eef@protonmail.com> wrote:
>
>> Yes ! That makes a huge difference. I added "net.bluetooth.usb_isoc_enable=0" to /boot/loader.conf, and rebooted. The interrupts have gone down to between 0, and 47.
>>
>> # vmstat -i
>> interrupt total rate
>> irq1: atkbd0 10 0
>> irq9: acpi0 9 0
>> cpu0:timer 70196 342
>> cpu1:timer 51387 250
>> cpu2:timer 50952 248
>> cpu3:timer 51181 249
>> irq128: xhci0 9659 47
>> irq130: nvme0:admin 15 0
>> irq131: nvme0:io0 3253 16
>> irq132: nvme0:io1 3243 16
>> irq133: nvme0:io2 3341 16
>> irq134: nvme0:io3 3883 19
>> irq135: hdac0 13 0
>> irq136: vgapci0 37917 185
>> Total 285059 1389
>>
>> However, the CPU interrupts seem quite high now. If you combine cpu*:timer i think it is over 1000, and maybe that relates to the original problem I reported ?
>>
>> Thanks!
>>
>> 0x1eef
>>
>> Sent with [Proton Mail](https://proton.me/) secure email.
>>
>> ------- Original Message -------
>> On Sunday, November 27th, 2022 at 1:45 AM, Paul Procacci <pprocacci@gmail.com> wrote:
>>
>>> What's catching my eye here is the values for ugen0.6. It's a usb bluetooth device.
>>>
>>> With that in mind I searched the bug database for bluetooth usb and xhci.
>>> I came across the following:
>>>
>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238235
>>>
>>> There does seem to be some sort of weird interaction with bluetooth and xhci as documented in the above link.
>>> I believe the patch listed in that bug report made it to 13.1....so can you try:
>>>
>>> sysctl -w net.bluetooth.usb_isoc_enable=0
>>>
>>> Does that make any difference?
>>> (This btw is a wild guess just based on the values from the output you provided me)
>>> Thanks,
>>> Paul
>>>
>>> On Sat, Nov 26, 2022 at 11:05 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>>
>>>> The results are sort of interesting. At first, UE_INTERRUPT_OK was at 3k+ for the USB mouse. I unplugged the mouse, and then the keyboard jumped from 0 to 1k+ for UE_INTERRUPT_OK.
>>>>
>>>> I have since reattached the mouse, and now both the mouse and the keyboard have a rising interrupt count. I would guess they jump by 20, or 30 interrupts every 2-3 seconds, with the keyboard jumping with a higher frequency.
>>>>
>>>> Paste
>>>>
>>>> ugen0.1: <Intel XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 0
>>>> UE_ISOCHRONOUS_OK : 0
>>>> UE_BULK_OK : 0
>>>> UE_INTERRUPT_OK : 0
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 0
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>>
>>>> ugen0.2: <Realtek USB 10/100/1000 LAN> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (350mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 7969
>>>> UE_ISOCHRONOUS_OK : 0
>>>> UE_BULK_OK : 11224
>>>> UE_INTERRUPT_OK : 0
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 94
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>>
>>>> ugen0.4: <Sonix Technology Co., Ltd. Integrated Camera> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 10
>>>> UE_ISOCHRONOUS_OK : 0
>>>> UE_BULK_OK : 0
>>>> UE_INTERRUPT_OK : 0
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 0
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>>
>>>> ugen0.6: <vendor 0x0cf3 product 0xe500> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 16
>>>> UE_ISOCHRONOUS_OK : 53214
>>>> UE_BULK_OK : 0
>>>> UE_INTERRUPT_OK : 9
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 0
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>>
>>>> ugen0.5: <vendor 0x30fa USB OPTICAL MOUSE> at usbus0, cfg=0 md=HOST spd=LOW (1.5Mbps) pwr=ON (100mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 11
>>>> UE_ISOCHRONOUS_OK : 0
>>>> UE_BULK_OK : 0
>>>> UE_INTERRUPT_OK : 1707
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 0
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>>
>>>> ugen0.3: <SINO WEALTH Gaming KB> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (500mA)
>>>>
>>>> {
>>>> UE_CONTROL_OK : 21
>>>> UE_ISOCHRONOUS_OK : 0
>>>> UE_BULK_OK : 0
>>>> UE_INTERRUPT_OK : 1790
>>>> UE_CONTROL_FAIL : 0
>>>> UE_ISOCHRONOUS_FAIL : 0
>>>> UE_BULK_FAIL : 0
>>>> UE_INTERRUPT_FAIL : 0
>>>> }
>>>> Best,
>>>> 0x1eef
>>>>
>>>> ------- Original Message -------
>>>> On Sunday, November 27th, 2022 at 12:31 AM, Paul Procacci <pprocacci@gmail.com> wrote:
>>>>
>>>>> usbconfig dump_stats
>>>>>
>>>>> Can you provide the output of the above? Run that a couple of times if you don't mind and do your very best to provide any rates of increases for any of the fields. An estimation would be perfectly fine.
>>>>> Thanks,
>>>>> Paul
>>>>>
>>>>> On Sat, Nov 26, 2022 at 10:09 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>>>>
>>>>>>> Can you determine if irq 128 is being shared by any devices?
>>>>>>
>>>>>> I couldn't determine that from dmesg.boot, but I think there could be some useful information in that file. I attached the file to this e-mail. Thank you!
>>>>>>
>>>>>> Best,
>>>>>> 0x1eef
>>>>>>
>>>>>> Sent with [Proton Mail](https://proton.me/) secure email.
>>>>>>
>>>>>> ------- Original Message -------
>>>>>> On Saturday, November 26th, 2022 at 8:49 PM, Paul Procacci <pprocacci@gmail.com> wrote:
>>>>>>
>>>>>>> Can you determine if irq 128 is being shared by any devices?
>>>>>>> Usually this information can be found in `dmesg' or '/var/run/dmesg.boot'.
>>>>>>> vmstat indeed shows a device but it sometimes doesn't show all the devices sharing that IRQ. It's possible you're being misled by vmstat.
>>>>>>> Just trying to get the complete picture here of devices. ;)
>>>>>>> Thanks,
>>>>>>> Paul Procacci
>>>>>>>
>>>>>>> On Sat, Nov 26, 2022 at 6:21 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>>>>>>
>>>>>>>> Hi !
>>>>>>>>
>>>>>>>>> Out of curiosity, have you pulled a usb device one by one until the interrupts disappear?
>>>>>>>>
>>>>>>>> I have three USB devices connected: mouse, keyboard, and an ethernet adapter.
>>>>>>>> I tried to remove each one by one, and I did not see the interrupt rate change.
>>>>>>>> I have also tried a cold boot without any USB devices connected, and the interrupt rate was about the same too.
>>>>>>>>
>>>>>>>> I don't know if it could be related, but there's a trackpad connected to the laptop that does not work. Maybe it has no relation to the issue, but setting "hw.psm.synaptics_support" to "0" also did not help.
>>>>>>>>
>>>>>>>> When Chromium loses focus, CPU usage usually drops to 0% and does not go above 10% - for as long as I am not using Chromium. I am using the i915 / drm kernel modules.. I saw another report of high CPU usage related to using those two kernel modules, but I wasn't able to identify that as the problem in my case.
>>>>>>>>
>>>>>>>> Thanks for the help.
>>>>>>>>
>>>>>>>> Sent with [Proton Mail](https://proton.me/) secure email.
>>>>>>>>
>>>>>>>> ------- Original Message -------
>>>>>>>> On Saturday, November 26th, 2022 at 8:06 PM, Paul Procacci <pprocacci@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hey,
>>>>>>>>>
>>>>>>>>> Not sure of the problem, but I don't see the correlation between Chrome and any usb driver.
>>>>>>>>> Out of curiosity, have you pulled a usb device one by one until the interrupts disappear?
>>>>>>>>> I'd be curious to know which device is slamming the system.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>> On Sat, Nov 26, 2022 at 6:02 PM 0x1eef <0x1eef@protonmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, everyone!
>>>>>>>>>>
>>>>>>>>>> When I use Chromium, I see a high rate of CPU usage across all four cores. The rate can be anywhere from 20% to 50%, even above that. I am not doing anything intensive, just browsing twitter, reddit, YouTube or GitHub. It has been like this since I installed FreeBSD, but since it's not a blocker I have been lazy about looking into it.
>>>>>>>>>>
>>>>>>>>>> I don't know why it happens. I can see that there are a high number of interrupts on 'xhci0', and that seems to carry over to each CPU core as well:
>>>>>>>>>>
>>>>>>>>>> # vmstat -i
>>>>>>>>>> interrupt total rate
>>>>>>>>>> irq1: atkbd0 50 0
>>>>>>>>>> irq9: acpi0 403 0
>>>>>>>>>> cpu0:timer 30716618 98
>>>>>>>>>> cpu1:timer 25457926 81
>>>>>>>>>> cpu2:timer 34344531 109
>>>>>>>>>> cpu3:timer 25542867 81
>>>>>>>>>> irq128: xhci0 328107434 1044
>>>>>>>>>> irq130: nvme0:admin 15 0
>>>>>>>>>> irq131: nvme0:io0 701041 2
>>>>>>>>>> irq132: nvme0:io1 692045 2
>>>>>>>>>> irq133: nvme0:io2 792760 3
>>>>>>>>>> irq134: nvme0:io3 693091 2
>>>>>>>>>> irq135: hdac0 1718425 5
>>>>>>>>>> irq136: vgapci0 6273295 20
>>>>>>>>>> Total 455040501 1448
>>>>>>>>>>
>>>>>>>>>> # dmesg | grep xhci0
>>>>>>>>>> xhci0: <Intel Ice Lake-LP USB 3.1 controller> mem 0x95110000-0x9511ffff at device 20.0 on pci0
>>>>>>>>>> xhci0: 32 bytes context size, 64-bit DMAusbus0 on xhci0
>>>>>>>>>>
>>>>>>>>>> It might also be helpful to know that I tried OpenBSD on the same computer but it was unusable for a similar reason: 95%+ interrupts on CPU. The impact that had made all tasks extremely slow. On FreeBSD it is not as bad, but I still think think it is not normal.
>>>>>>>>>>
>>>>>>>>>> Can anyone suggest what might be wrong, tips to debug, etc ? If more information is needed, please let me know. Thanks for your time.
>>>>>>>>>>
>>>>>>>>>> Best,0x1eef
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> __________________
>>>>>>>>>
>>>>>>>>> :(){ :|:& };:
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> __________________
>>>>>>>
>>>>>>> :(){ :|:& };:
>>>>>
>>>>> --
>>>>>
>>>>> __________________
>>>>>
>>>>> :(){ :|:& };:
>>>
>>> --
>>>
>>> __________________
>>>
>>> :(){ :|:& };:
>
> --
>
> __________________
>
> :(){ :|:& };: