VirtualBox 4.2.4 on FreeBSD 9.1-PRERELEASE problem: VMs behave very different when pinned to different cores

[Alex Chistyakov]

Hello,

I am back with another problem. As I discovered previously setting a
CPU affinity explicitly helps to get decent performance on guests, but
the problem is that guest performance is very different on core #0 and
cores #5 or #7. Basically when I use 'cpuset -l 0 VBoxHeadless -s
"Name" -v on' to start the VM it is barely usable at all. The best
performance results are on cores #4 and #5 (I believe they are the
same physical core due to HT). #7 and #8 are twice as slow as #5, #0
and #1 are the slowest and other cores lay in the middle.
If I disable a tickless kernel on the guest running on #4 or #5 it
becomes as slow as a guest running on #7 so I suspect this is a
timer-related issue.
I also discovered that there are quite a lot of system interrupts on
slow guests (%si is about 10-15) but Munin does not render them on its
CPU graphs for some reason.
All my VMs are on cores #4 and #5 right now but I want to utilize
other cores too. I am not sure what to do next, this looks like a
VirtualBox bug. What can be done to solve this?

Thank you!

--
SY,
Alex