cx_lowest and CPU usage

[Andriy Gapon]

on 14/02/2008 23:31 Andriy Gapon said the following:
> I ran a series of tests, repeating each twice to be sure that I didn't
> make any mistake.
> All tests were performed in single-user mode, so the system was as idle
> as possible, top reported idle as 99.N% - 100%.
> Then I set hw.acpi.cpu.cx_lowest=C2 and ran top again.
> Here's the results:
> GENERIC, SCHED_4BSD, default HZ=1000 ==> C2-interrupt 11-14% (!!)
> GENERIC, SCHED_4BSD, kern.hz="100" ==> C2-interrupt 99-100%
> customized kernel, SCHED_ULE, default HZ=1000 ==> C2-interrupt 99-100%
> customized kernel, SCHED_ULE, kern.hz="100" ==> C2-interrupt 99-100%

I did some testing and here are some additional data.
Running with sched_ule and hz=1000 I measured that mean time spent in C2
state (from starting to go into it, till completely returning from it)
is ~860us (end_time - start_time, no overhead adjustments).
Additionally, 98.1% of sleeps lasted 800-900us, 1.8% lasted 700-800us,
the rest is spread almost uniformly over the remaining intervals of
range 0-1000%.

Here's a typical backtrace for statclock execution:
#9  0xc04cce26 in statclock (usermode=0) at
/system/src/sys/kern/kern_clock.c:447
#10 0xc06b5f0c in rtcintr (frame=0xcfb30c78) at
/system/src/sys/i386/isa/clock.c:236
#11 0xc06a5275 in intr_execute_handlers (isrc=0xc0736900,
frame=0xcfb30c78) at /system/src/sys/i386/i386/intr_machdep.c:362
#12 0xc06b558c in atpic_handle_intr (vector=8, frame=0xcfb30c78) at
/system/src/sys/i386/isa/atpic.c:596
#13 0xc06a0d71 in Xatpic_intr8 () at atpic_vector.s:70
#14 0xc06a70c9 in spinlock_exit () at cpufunc.h:365
#15 0xc04e6993 in ithread_loop (arg=0xc22f5980) at
/system/src/sys/kern/kern_intr.c:1137
#16 0xc04e32c1 in fork_exit (callout=0xc04e6640 <ithread_loop>,
arg=0xc22f5980, frame=0xcfb30d38) at /system/src/sys/kern/kern_fork.c:754
#17 0xc06a0bc0 in fork_trampoline () at
/system/src/sys/i386/i386/exception.s:205

I.e. swi4 kernel thread is "caught" by IRQ8 just when it exits from
thread_unlock() in the following snippet:
1132                    if (!ithd->it_need && !(ithd->it_flags & IT_DEAD)) {
1133                            TD_SET_IWAIT(td);
1134                            ie->ie_count = 0;
1135                            mi_switch(SW_VOL, NULL);
1136                    }
1137                    thread_unlock(td);

This happens almost deterministically.

-- 
Andriy Gapon