FreeBSD 10-R 8 vCPU panics at boot under XenServer (on 8 'core' CPU)

[Roger Pau Monné]

On 17/02/14 15:44, Karl Pielorz wrote:
> 
> Hi,
> 
> I've got a FreeBSD 10-R amd64 DomU guest I'm using under XenServer 6.2
> (SP1) - this was working fine (i.e. had been restarted many times -
> while I look at things like HAST).
> 
> I noticed the other day it was only set to use 4 vCPU's - so I increased
> this to 8 (the machine has an 4 Core, 8 Thread Xeon 1230v3 in it - which
> Xen see's as 8 CPU cores).
> 
> However, it won't boot reliably now:
> 
> "
> ...
> SMP: AP CPU #5 Launched
> panic: can't schedule timer
> cpuid = 0
> KDB: stack backtrace:
> #0 0xffffffff808e7dd0 at kdb_backtrace+0x60
> #1 0xffffffff808af8b5 at panic+0x155
> #2 0xffffffff807a14dd at xentimer_et_start+0xed
> #3 0xffffffff80d66d6d at loadtimer+0xfd
> #4 0xffffffff80d657fd at handleevents+0x308
> #5 0xffffffff80d65fc8 at timercb+0x308
> #6 0xffffffff807a152d at xentimer_intr+0x4d
> #7 0xffffffff80883e5b at intr_event_handle+0x9b
> ...
> "
> 
> Less than 8 vCPU's seems to boot OK (e.g. 7) and 8 vCPU's has booted a
> couple of times (out of 30+ reboots).

I usually do most of my testing on a Xen W3550 (8-ways), with a 8 vCPU
guest, and I've never seen this crash before. I've even booted a 12 vCPU
guest on this 8-way system, and it was fine.

How many guests are you running on this host, and how many vCPUs has
each one assigned?

> The system is running GENERIC with:
> 
>  options   NO_ADAPTIVE_MUTEXES
>  options   NO_ADAPTIVE_RWLOCKS
>  options   NO_ADAPTIVE_SX
> 
> In addition XenServer is set to pass through the bare machine's LSI and
> two Intel NIC's (which it does - and are working, once FreeBSD is booted).

I don't think those modifications have any effect on the timer, but
could you try to recompile without the NO_ADAPTIE_* modifications and
without any device pass-through?

In order to provide more debug info, could you apply the following patch:

http://xenbits.xen.org/people/royger/0001-xen-debug-Xen-PV-timer.patch

It will expand the panic message a little bit. Also, after applying the
patch you can manually edit sys/dev/xen/timer/timer.c and increase
NUM_RETRIES to see if that solves the problem.

Roger.