Bheve: Slow linux syscalls on AMD

[Peter Grehan]

Hi Willem,

> I've got KTR compiled in, but last time I switched it on.
> I got swamped in traffic, and I sort of got locked out of the server... :(
> Could also be because I was writing it to a file as well.
>
> So you'll have to help/tell me what to do.

  For looking at this, I'd use one single-vCPU guest to avoid KTR noise 
from other vCPU/guests. The problem shows up with this so no issue there.

  First, boot the VM and get it sitting at a shell prompt idling.

  On the host, first see that there are enough entries in the KTR buffer

# sysctl debug.ktr.entries
<32/64K is a good number>

  If it's too small, it can be bumped up

# sysctl debug.ktr.entries=65536
debug.ktr.entries: 8192 -> 65536

  The bhyve kernel module uses the ktr mask value 0x01. This should be 
turned off by default:

# sysctl debug.ktr.mask
debug.ktr.mask: 0

  To get some traces, turn this on for a few seconds

# sysctl debug.ktr.mask=1 ; sleep 3 ; sysctl debug.ktr.mask=0

  The ktr trace buffer can now be examined:

# ktrdump -ct

  To repeat/rerun the process, make sure the ktr buffer is cleared with

# sysctl debug.ktr.clear=1
debug.ktr.clear: 0 -> 0

later,

Peter.