TSC as timecounter makes system lag
Jia-Shiun Li
jiashiun at gmail.com
Mon Jan 16 14:34:59 UTC 2017
On Mon, Jan 16, 2017 at 8:00 PM, Konstantin Belousov <kostikbel at gmail.com>
wrote:
> On Mon, Jan 16, 2017 at 12:28:54PM +0800, Jia-Shiun Li wrote:
> > BTW please see my other mail of this thread. It seems to be related to
> > EARLY_AP_STARTUP option.
> Yes, I noted, I might have an idea, but the report that changing the
> timecounter makes the lags go away still does not fit into my understanding
> of the code.
>
> Most likely this is an interaction between the EARLY_AP_STARTUP and the
> fact that HPET interrupt is global, while most modern systems use LAPIC
> event timer, which is per-cpu, and the testing of the option was done
> on them. There are some differences in handling the configurations, see
> sys/kern/kern_clocksource.c, the option and ET_FLAG_PERCPU.
>
>
And, changing the _timecounter_ fixes the issue ? Can you double-check
> this ?
>
Yes. I noticed this because systat refreshes looked slower,
and keystroke did not repeat smoothly for 30/s.
I have system clock shown on tmux status line. On c2d it drifted away.
Setting timecounter brings it back to normal.
See also eventtimer & timecounter tests below.
>
> With the settings above, i.e. HPET for both eventtimer and timecounter,
> please show vmstat -ia output for two times with the interval of 2 secs.
>
Attached.
>
> What if you change _eventtimer_ to APIC and then immediately back to HPET,
> does the problem go away ?
>
>
It doesn't look so. But keeping LAPIC as eventtimer helps.
jsli at jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=LAPIC && sysctl
kern.eventtimer.timer=HPET && ntpdate tw.pool.ntp.org && sleep 30 &&
ntpdate tw.pool.ntp.org
kern.eventtimer.timer: HPET -> LAPIC
kern.eventtimer.timer: LAPIC -> HPET
16 Jan 22:00:21 ntpdate[8472]: step time server 203.71.244.7 offset
18.980716 sec
16 Jan 22:01:56 ntpdate[8601]: step time server 103.226.213.30 offset
58.813079 sec
jsli at jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=LAPIC && ntpdate
tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org
kern.eventtimer.timer: HPET -> LAPIC
16 Jan 22:02:36 ntpdate[8666]: step time server 103.226.213.30 offset
19.773086 sec
16 Jan 22:03:13 ntpdate[8776]: adjust time server 103.226.213.30 offset
0.000455 sec
jsli at jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=HPET && ntpdate
tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org
kern.eventtimer.timer: LAPIC -> HPET
16 Jan 22:03:47 ntpdate[8853]: step time server 103.226.213.30 offset
6.344004 sec
16 Jan 22:05:18 ntpdate[8975]: step time server 61.216.153.105 offset
54.908872 sec
jsli at jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=HPET && ntpdate
tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org
kern.timecounter.hardware: TSC-low -> HPET
16 Jan 22:06:29 ntpdate[9073]: step time server 59.124.29.241 offset
39.211691 sec
16 Jan 22:07:05 ntpdate[9185]: adjust time server 61.216.153.105 offset
0.001015 sec
jsli at jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=TSC-low &&
ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org
kern.timecounter.hardware: HPET -> TSC-low
16 Jan 22:07:28 ntpdate[9244]: step time server 61.216.153.105 offset
3.122954 sec
16 Jan 22:08:58 ntpdate[9357]: step time server 61.216.153.104 offset
53.758451 sec
jsli at jsli-bsd:/home/jsli #
> Also, if you set the loader tunable kern.eventtimer.timer to LAPIC,
> and do not enable C2+, does the system boot into the usable state ?
>
>
Not sure if you mean disabling C2 from BIOS.
Right now I don't have BIOS access to the c2d, and my notebook only
has minimal BIOS settings to play with. Anyway on notebook
set kern.eventtimer.timer=LAPIC
in loader prompt to boot , and the system still lags. But after setting
sysctl dev.cpu.0.cx_lowest=C1 && sysctl dev.cpu.1.cx_lowest=C1
system clock returns to normal speed.
-Jia-Shiun.
-------------- next part --------------
jsli at jsli-bsd:~ % vmstat -ia 2 2
interrupt total rate
??? 0 0
irq1: atkbd0 2 0
stray irq1 0 0
irq0: attimer0 0 0
stray irq0 0 0
irq3: 0 0
stray irq3 0 0
irq4: uart0 0 0
stray irq4 0 0
irq5: 0 0
stray irq5 0 0
irq6: 0 0
stray irq6 0 0
irq7: 0 0
stray irq7 0 0
irq8: atrtc0 0 0
stray irq8 0 0
irq9: acpi0 0 0
stray irq9 0 0
irq10: 0 0
stray irq10 0 0
irq11: 0 0
stray irq11 0 0
irq12: 0 0
stray irq12 0 0
irq13: 0 0
stray irq13 0 0
irq14: 0 0
stray irq14 0 0
irq15: 0 0
stray irq15 0 0
irq16: em0:irq0++ 19 0
stray irq16 0 0
irq17: 0 0
stray irq17 0 0
irq18: uhci2 ehci0+ 18 0
stray irq18 0 0
irq19: uhci4 0 0
stray irq19 0 0
irq20: hpet0 13592 192
stray irq20 0 0
irq21: uhci1 0 0
stray irq21 0 0
irq22: 0 0
stray irq22 0 0
irq23: uhci3 ehci1 0 0
stray irq23 0 0
cpu0:timer 0 0
cpu1:timer 0 0
irq256: hdac0 95 1
stray irq256 0 0
irq257: pcib1 0 0
stray irq257 0 0
irq258: pcib2 0 0
stray irq258 0 0
irq259: pcib3 0 0
stray irq259 0 0
irq260: re0 16191 229
stray irq260 0 0
irq261: ahci0:ch0 4210 60
stray irq261 0 0
irq262: ahci0:ch1 0 0
stray irq262 0 0
irq263: ahci0:ch2 0 0
stray irq263 0 0
irq264: ahci0:ch3 0 0
stray irq264 0 0
irq265: ahci0:ch4 0 0
stray irq265 0 0
irq266: ahci0:ch5 0 0
stray irq266 0 0
irq267: ahci0:6 0 0
stray irq267 0 0
irq268: ahci0:7 0 0
stray irq268 0 0
irq269: ahci0:8 0 0
stray irq269 0 0
irq270: ahci0:9 0 0
stray irq270 0 0
irq271: ahci0:10 0 0
stray irq271 0 0
irq272: ahci0:11 0 0
stray irq272 0 0
irq273: ahci0:12 0 0
stray irq273 0 0
irq274: ahci0:13 0 0
stray irq274 0 0
irq275: ahci0:14 0 0
stray irq275 0 0
irq276: ahci0:15 0 0
stray irq276 0 0
Total 34127 483
??? 0 0
irq1: atkbd0 0 0
stray irq1 0 0
irq0: attimer0 0 0
stray irq0 0 0
irq3: 0 0
stray irq3 0 0
irq4: uart0 0 0
stray irq4 0 0
irq5: 0 0
stray irq5 0 0
irq6: 0 0
stray irq6 0 0
irq7: 0 0
stray irq7 0 0
irq8: atrtc0 0 0
stray irq8 0 0
irq9: acpi0 0 0
stray irq9 0 0
irq10: 0 0
stray irq10 0 0
irq11: 0 0
stray irq11 0 0
irq12: 0 0
stray irq12 0 0
irq13: 0 0
stray irq13 0 0
irq14: 0 0
stray irq14 0 0
irq15: 0 0
stray irq15 0 0
irq16: em0:irq0++ 0 0
stray irq16 0 0
irq17: 0 0
stray irq17 0 0
irq18: uhci2 ehci0+ 0 0
stray irq18 0 0
irq19: uhci4 0 0
stray irq19 0 0
irq20: hpet0 81 40
stray irq20 0 0
irq21: uhci1 0 0
stray irq21 0 0
irq22: 0 0
stray irq22 0 0
irq23: uhci3 ehci1 0 0
stray irq23 0 0
cpu0:timer 0 0
cpu1:timer 0 0
irq256: hdac0 0 0
stray irq256 0 0
irq257: pcib1 0 0
stray irq257 0 0
irq258: pcib2 0 0
stray irq258 0 0
irq259: pcib3 0 0
stray irq259 0 0
irq260: re0 580 289
stray irq260 0 0
irq261: ahci0:ch0 0 0
stray irq261 0 0
irq262: ahci0:ch1 0 0
stray irq262 0 0
irq263: ahci0:ch2 0 0
stray irq263 0 0
irq264: ahci0:ch3 0 0
stray irq264 0 0
irq265: ahci0:ch4 0 0
stray irq265 0 0
irq266: ahci0:ch5 0 0
stray irq266 0 0
irq267: ahci0:6 0 0
stray irq267 0 0
irq268: ahci0:7 0 0
stray irq268 0 0
irq269: ahci0:8 0 0
stray irq269 0 0
irq270: ahci0:9 0 0
stray irq270 0 0
irq271: ahci0:10 0 0
stray irq271 0 0
irq272: ahci0:11 0 0
stray irq272 0 0
irq273: ahci0:12 0 0
stray irq273 0 0
irq274: ahci0:13 0 0
stray irq274 0 0
irq275: ahci0:14 0 0
stray irq275 0 0
irq276: ahci0:15 0 0
stray irq276 0 0
Total 661 330
jsli at jsli-bsd:~ % su
Password:
jsli at jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=HPET
kern.timecounter.hardware: TSC-low -> HPET
jsli at jsli-bsd:/home/jsli # vmstat -ia 2 2
interrupt total rate
??? 0 0
irq1: atkbd0 2 0
stray irq1 0 0
irq0: attimer0 0 0
stray irq0 0 0
irq3: 0 0
stray irq3 0 0
irq4: uart0 0 0
stray irq4 0 0
irq5: 0 0
stray irq5 0 0
irq6: 0 0
stray irq6 0 0
irq7: 0 0
stray irq7 0 0
irq8: atrtc0 0 0
stray irq8 0 0
irq9: acpi0 0 0
stray irq9 0 0
irq10: 0 0
stray irq10 0 0
irq11: 0 0
stray irq11 0 0
irq12: 0 0
stray irq12 0 0
irq13: 0 0
stray irq13 0 0
irq14: 0 0
stray irq14 0 0
irq15: 0 0
stray irq15 0 0
irq16: em0:irq0++ 19 0
stray irq16 0 0
irq17: 0 0
stray irq17 0 0
irq18: uhci2 ehci0+ 18 0
stray irq18 0 0
irq19: uhci4 0 0
stray irq19 0 0
irq20: hpet0 14679 147
stray irq20 0 0
irq21: uhci1 0 0
stray irq21 0 0
irq22: 0 0
stray irq22 0 0
irq23: uhci3 ehci1 0 0
stray irq23 0 0
cpu0:timer 0 0
cpu1:timer 0 0
irq256: hdac0 95 1
stray irq256 0 0
irq257: pcib1 0 0
stray irq257 0 0
irq258: pcib2 0 0
stray irq258 0 0
irq259: pcib3 0 0
stray irq259 0 0
irq260: re0 23596 237
stray irq260 0 0
irq261: ahci0:ch0 4331 43
stray irq261 0 0
irq262: ahci0:ch1 0 0
stray irq262 0 0
irq263: ahci0:ch2 0 0
stray irq263 0 0
irq264: ahci0:ch3 0 0
stray irq264 0 0
irq265: ahci0:ch4 0 0
stray irq265 0 0
irq266: ahci0:ch5 0 0
stray irq266 0 0
irq267: ahci0:6 0 0
stray irq267 0 0
irq268: ahci0:7 0 0
stray irq268 0 0
irq269: ahci0:8 0 0
stray irq269 0 0
irq270: ahci0:9 0 0
stray irq270 0 0
irq271: ahci0:10 0 0
stray irq271 0 0
irq272: ahci0:11 0 0
stray irq272 0 0
irq273: ahci0:12 0 0
stray irq273 0 0
irq274: ahci0:13 0 0
stray irq274 0 0
irq275: ahci0:14 0 0
stray irq275 0 0
irq276: ahci0:15 0 0
stray irq276 0 0
Total 42740 429
??? 0 0
irq1: atkbd0 0 0
stray irq1 0 0
irq0: attimer0 0 0
stray irq0 0 0
irq3: 0 0
stray irq3 0 0
irq4: uart0 0 0
stray irq4 0 0
irq5: 0 0
stray irq5 0 0
irq6: 0 0
stray irq6 0 0
irq7: 0 0
stray irq7 0 0
irq8: atrtc0 0 0
stray irq8 0 0
irq9: acpi0 0 0
stray irq9 0 0
irq10: 0 0
stray irq10 0 0
irq11: 0 0
stray irq11 0 0
irq12: 0 0
stray irq12 0 0
irq13: 0 0
stray irq13 0 0
irq14: 0 0
stray irq14 0 0
irq15: 0 0
stray irq15 0 0
irq16: em0:irq0++ 0 0
stray irq16 0 0
irq17: 0 0
stray irq17 0 0
irq18: uhci2 ehci0+ 0 0
stray irq18 0 0
irq19: uhci4 0 0
stray irq19 0 0
irq20: hpet0 42 21
stray irq20 0 0
irq21: uhci1 0 0
stray irq21 0 0
irq22: 0 0
stray irq22 0 0
irq23: uhci3 ehci1 0 0
stray irq23 0 0
cpu0:timer 0 0
cpu1:timer 0 0
irq256: hdac0 0 0
stray irq256 0 0
irq257: pcib1 0 0
stray irq257 0 0
irq258: pcib2 0 0
stray irq258 0 0
irq259: pcib3 0 0
stray irq259 0 0
irq260: re0 219 108
stray irq260 0 0
irq261: ahci0:ch0 0 0
stray irq261 0 0
irq262: ahci0:ch1 0 0
stray irq262 0 0
irq263: ahci0:ch2 0 0
stray irq263 0 0
irq264: ahci0:ch3 0 0
stray irq264 0 0
irq265: ahci0:ch4 0 0
stray irq265 0 0
irq266: ahci0:ch5 0 0
stray irq266 0 0
irq267: ahci0:6 0 0
stray irq267 0 0
irq268: ahci0:7 0 0
stray irq268 0 0
irq269: ahci0:8 0 0
stray irq269 0 0
irq270: ahci0:9 0 0
stray irq270 0 0
irq271: ahci0:10 0 0
stray irq271 0 0
irq272: ahci0:11 0 0
stray irq272 0 0
irq273: ahci0:12 0 0
stray irq273 0 0
irq274: ahci0:13 0 0
stray irq274 0 0
irq275: ahci0:14 0 0
stray irq275 0 0
irq276: ahci0:15 0 0
stray irq276 0 0
Total 261 129
jsli at jsli-bsd:/home/jsli #
More information about the freebsd-current
mailing list