Xentimer/Lapic differences when used as a eventtimer source

[Shrikanth Kamath]

Trying to understand the implementation differences for xentimer v/s lapic
when used as eventtimer sources. I have 2 setups where there is FreeBSD
11.0 based Junos running in a virtual environment one on a xen based
hypervisor and another kvm based. The xen version has xentimer as the
eventtimer source v/s lapic for the kvm version. For an application that is
doing nanosleep with same resolution of 100 ns and running on these two
systems the rate of timer interrupts is order of magnitude different and
hogs the cpu in case of lapic but for xentimer it is not as significant.
Below I have listed some readings from dtrace probes I ran to understand,

For xen setup using xentimer as eventtimer

# sysctl kern.eventtimer

kern.eventtimer.periodic: 0

kern.eventtimer.timer: XENTIMER

kern.eventtimer.idletick: 0

kern.eventtimer.singlemul: 2

kern.eventtimer.choice: XENTIMER(950) LAPIC(100)

i8254(100) RTC(0)

kern.eventtimer.et.XENTIMER.quality: 950

kern.eventtimer.et.XENTIMER.frequency: 1000000000

kern.eventtimer.et.XENTIMER.flags: 6

For kvm setup using lapic as eventtimer

# sysctl kern.eventtimer

kern.eventtimer.periodic: 0

kern.eventtimer.timer: LAPIC

kern.eventtimer.idletick: 0

kern.eventtimer.singlemul: 2

kern.eventtimer.choice: LAPIC(600) RTC(0)

kern.eventtimer.et.LAPIC.quality: 600

kern.eventtimer.et.LAPIC.frequency: 3000050812

kern.eventtimer.et.LAPIC.flags: 7


Running fbt::et_start:entry with conditional for execname to be the
application and using an aggregation to count() calls, shows ~6K calls per
second for xentimer v/s ~95K calls for lapic


Trying to count number of timer interrupts on each with


dtrace -n 'fbt:kernel-xen:xentimer_intr:entry /cpu != 0/ { @cnt[cpu] =
count();} tick-1s { printa(@cnt); clear(@cnt);}' gives --> 6083 interrupts
per second

v/s

dtrace -n 'fbt:kernel:lapic_handle_timer:entry /cpu != 0/ { @cnt[cpu] =
count();} tick-1s { printa(@cnt); clear(@cnt);}' gives --> 99063 interrupts
per second


For xentimer

#vmstat -i | grep :xen

interrupt    total                 rate

cpu1:xen  3642296440    6084


v/s lapic

# vmstat -i

interrupt    total                 rate

cpu1:timer 61458150612 102715


Also does lapic source have additional overhead of VM entry/VM exit?


-- 
Shrikanth R K