From nobody Mon Nov 21 13:15:45 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NG7Dj4RLxz4j1Wm; Mon, 21 Nov 2022 13:15:49 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NG7Dj3v0xz44pf; Mon, 21 Nov 2022 13:15:49 +0000 (UTC) (envelope-from kp@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669036549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VvIx3/j1aCpRfweiWpxmNZFYSCOdR/9BQAJ1aTL4j74=; b=AIZqKoKOg361pejId/at2DL1uq1/lFD5xG4KrpIFcbVxYmoGR9kK0sWX6GF4OQFoXbRyhQ fyQm0MWvYtYWX0sDF80ijX/pBdxvcXE9MNwN9Y0TSjpq/n8ZeaixlkDvvASm1mDT0C3lCH RsJeYSPW2G26dt7y/sEuk76/Sil1Dgy6Wv5rYSA8LotMFDc7My0579wjgTC4CQ8M7EsZRW 8aiqYkB5UxzDvmBplcktnek1udQrT4nPaCedHkVcTL4xjRy1dip2isxQ/CuvLQLeR1zWdu FpWmTt8cqpLZJwT3ZCuGCtVuvCest4g2JOUYuNjCUhSWyfqD3FYYX4xJXq8hMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669036549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VvIx3/j1aCpRfweiWpxmNZFYSCOdR/9BQAJ1aTL4j74=; b=VfhQFcUiesRFJHrQ6UhGj3Q9oVoC4C48zM8OmtBnF0xzpBzg/168dMbYWVtqgCQLaK/PHK RjqAblgFD/vgPYUy5LRv2V2IBU5uGAdkEU9LN/I1499keeAcXa8GlxE6dO5slFh56+XP3v 91RirCc0dj80ezIFfJZFYL0ONh54Twe4JDIMaOqD11LIy7cuIXDTvFgAvDZNzpF1jWJ0Z+ B6e5XAMMvcrF5zd46a+Nxg7EjZPDYbJnIjnXC9IcFvxb+5shc4hWS5CFAbLbfVT+IUNYq1 YOUxlBClXINF6NX7W96iZapERy1ZAg38KeAgyDwugfK4PJe3epPl8mITtAdfGA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1669036549; a=rsa-sha256; cv=none; b=vEwxOZwElafnlNdbJGs2Ps823k4lzEUJcklPv8bVfxNcGge1zXRfWErOc3XF6/ERx2Vgwh /dJhq2pefELD9rKndA/s1K8tMIzIVPlXbgFIVxuHhJQ7rYxSKhIiFJ5iJDgBp2Q4BVtycC UsPTtaLb5KjWd39z/mYstVPnz4n+egBA98X2Rexr9AJf7X2Umi/YWt0Of91Wzehw6cDqZj RVyQ48tTeEpqRAqCzR4WGfkFJTriEwnd6W3H093NgBm63f80uwa93+rRfmGjZCdmxsBpyl daUmbmfjNigw3BtMqgMhHa+ZmJ1YSHy4hpBWBaZyMQkAgsSIjQ6xp6qjrjWClA== Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NG7Dj1Snlz1FVk; Mon, 21 Nov 2022 13:15:49 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 4A44344E72; Mon, 21 Nov 2022 14:15:46 +0100 (CET) From: Kristof Provost To: John Baldwin Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: c0f35dbf19c3 - main - vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. Date: Mon, 21 Nov 2022 14:15:45 +0100 X-Mailer: MailMate (1.14r5918) Message-ID: In-Reply-To: <202211181826.2AIIQssl030757@gitrepo.freebsd.org> References: <202211181826.2AIIQssl030757@gitrepo.freebsd.org> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_9195AF87-15B8-46D2-B75E-5791331F0D7F_=" X-ThisMailContainsUnwantedMimeParts: N --=_MailMate_9195AF87-15B8-46D2-B75E-5791331F0D7F_= Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable On 18 Nov 2022, at 19:26, John Baldwin wrote: > The branch main has been updated by jhb: > > URL: = > https://cgit.FreeBSD.org/src/commit/?id=3Dc0f35dbf19c3c8825bd2b321d8efd= 582807d1940 > > commit c0f35dbf19c3c8825bd2b321d8efd582807d1940 > Author: John Baldwin > AuthorDate: 2022-11-18 18:05:10 +0000 > Commit: John Baldwin > CommitDate: 2022-11-18 18:25:38 +0000 > > vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. > > Retire the boot_state member of struct vlapic and instead use a = > cpuset > in the VM to track vCPUs waiting for STARTUP IPIs. INIT IPIs add > vCPUs to this set, and STARTUP IPIs remove vCPUs from the set. > STARTUP IPIs are only reported to userland for vCPUs that were = > removed > from the set. > > In particular, this permits a subsequent change to allocate vCPUs = > on > demand when the vCPU may not be allocated until after a STARTUP = > IPI is > reported to userland. > > Reviewed by: corvink, markj > Differential Revision: https://reviews.freebsd.org/D37173 > --- > sys/amd64/include/vmm.h | 3 +++ > sys/amd64/vmm/io/vlapic.c | 46 = > ++++++++++-------------------------------- > sys/amd64/vmm/io/vlapic_priv.h | 7 ------- > sys/amd64/vmm/vmm.c | 27 +++++++++++++++++++++++++ > 4 files changed, 41 insertions(+), 42 deletions(-) > I=E2=80=99m seeing a panic starting a bhyve guest, and I think this commi= t = might be the responsible one: login: panic: acquiring blockable sleep lock with spinlock or critical = section held (sleep mutex) vm rendezvous lock @ = /usr/src/sys/amd64/vmm/vmm.c:2508 cpuid =3D 14 time =3D 1669035212 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe015f2dd530 vpanic() at vpanic+0x151/frame 0xfffffe015f2dd580 panic() at panic+0x43/frame 0xfffffe015f2dd5e0 witness_checkorder() at witness_checkorder+0xd3e/frame = 0xfffffe015f2dd7a0 __mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe015f2dd7f0 vm_start_cpus() at vm_start_cpus+0x31/frame 0xfffffe015f2dd820 vlapic_icrlo_write_handler() at vlapic_icrlo_write_handler+0x30c/frame = 0xfffffe015f2dd8a0 vmx_run() at vmx_run+0x20ed/frame 0xfffffe015f2dd9c0 vm_run() at vm_run+0x1d2/frame 0xfffffe015f2ddaa0 vmmdev_ioctl() at vmmdev_ioctl+0x644/frame 0xfffffe015f2ddb40 devfs_ioctl() at devfs_ioctl+0xcd/frame 0xfffffe015f2ddb90 vn_ioctl() at vn_ioctl+0x131/frame 0xfffffe015f2ddca0 devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe015f2ddcc0 kern_ioctl() at kern_ioctl+0x202/frame 0xfffffe015f2ddd30 sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe015f2dde00 amd64_syscall() at amd64_syscall+0x12e/frame 0xfffffe015f2ddf30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe015f2ddf30 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x3dd9cbd7f94a, rsp =3D = 0x3ddc1d44ae58, rbp =3D 0x3ddc1d44af10 --- And kgdb=E2=80=99s backtrace: (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:59 #1 dump_savectx () at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80bec678 in dumpsys (di=3D0x0) at = /usr/src/sys/x86/include/dump.h:87 #3 doadump (textdump=3Dtextdump@entry=3D0) at = /usr/src/sys/kern/kern_shutdown.c:434 #4 0xffffffff804b403a in db_dump (dummy=3D, = dummy2=3D, dummy3=3D, dummy4=3D) a= t = /usr/src/sys/ddb/db_command.c:593 #5 0xffffffff804b3e40 in db_command (last_cmdp=3D, = cmd_table=3D, dopager=3Dtrue) at = /usr/src/sys/ddb/db_command.c:506 #6 0xffffffff804b3b0d in db_command_loop () at = /usr/src/sys/ddb/db_command.c:553 #7 0xffffffff804b71a6 in db_trap (type=3D, = code=3D) at /usr/src/sys/ddb/db_main.c:270 #8 0xffffffff80c3b89e in kdb_trap (type=3Dtype@entry=3D3, = code=3D, code@entry=3D0, tf=3Dtf@entry=3D0xfffffe015f2dd470)= at = /usr/src/sys/kern/subr_kdb.c:745 #9 0xffffffff810ce577 in trap (frame=3D0xfffffe015f2dd470) at = /usr/src/sys/amd64/amd64/trap.c:611 #10 #11 kdb_enter (why=3D, msg=3D) at = /usr/src/sys/kern/subr_kdb.c:509 #12 0xffffffff80bec822 in vpanic (fmt=3D, = ap=3Dap@entry=3D0xfffffe015f2dd5c0) at /usr/src/sys/kern/kern_shutdown.c:= 967 #13 0xffffffff80bec5c3 in panic (fmt=3D0xffffffff81e8ce70 = "\314:)\201\377\377\377\377") at /usr/src/sys/kern/kern_shutdown.c:903 #14 0xffffffff80c5ea4e in witness_checkorder (lock=3D0xfffff804d585d138,= = flags=3D, file=3D, line=3D2508, = interlock=3D) at /usr/src/sys/kern/subr_witness.c:1202 #15 0xffffffff80bc6d04 in __mtx_lock_flags (c=3D0xfffff804d585d150, = opts=3D0, file=3D0xffffffff8322542d "/usr/src/sys/amd64/vmm/vmm.c", = line=3D2508) at /usr/src/sys/kern/kern_mutex.c:278 #16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, = tostart=3Dtostart@entry=3D0xfffffe015f2dd858) at = /usr/src/sys/amd64/vmm/vmm.c:2508 #17 0xffffffff83211b5c in vlapic_icrlo_write_handler = (vlapic=3Dvlapic@entry=3D0xfffff8048dc14a80, = retu=3Dretu@entry=3D0xfffffe015f2dd950) at = /usr/src/sys/amd64/vmm/io/vlapic.c:1172 #18 0xffffffff8321977d in vmx_handle_apic_write = (vcpu=3D0xfffff8048db3ac00, vlapic=3D0xfffff8048dc14a80, qual=3D768) at = /usr/src/sys/amd64/vmm/intel/vmx.c:2184 #19 vmx_exit_process (vmx=3D0xfffff804d585b000, vcpu=3D0xfffff8048db3ac0= 0, = vmexit=3D0xfffff804a7c1a688) at /usr/src/sys/amd64/vmm/intel/vmx.c:2767 #20 vmx_run (vcpui=3D0xfffff8048db3ac00, rip=3D, = pmap=3D0xfffffe003dce1530, evinfo=3D0xfffffe015f2dd9d8) at = /usr/src/sys/amd64/vmm/intel/vmx.c:3174 #21 0xffffffff83202572 in vm_run (vcpu=3Dvcpu@entry=3D0xfffff804a7c1a600= , = vme_user=3Dvme_user@entry=3D0xfffff80032601b08) at = /usr/src/sys/amd64/vmm/vmm.c:1873 #22 0xffffffff83205ab4 in vmmdev_ioctl (cdev=3D, = cmd=3D3230692865, data=3D, fflag=3D, = td=3D) at /usr/src/sys/amd64/vmm/vmm_dev.c:565 #23 0xffffffff80a7be7d in devfs_ioctl (ap=3D0xfffffe015f2ddba8) at = /usr/src/sys/fs/devfs/devfs_vnops.c:933 #24 0xffffffff80cf6421 in vn_ioctl (fp=3D0xfffff8000ce2cb40, = com=3D, data=3D0xfffff80032601b00, = active_cred=3D0xfffff8000c4d0800, td=3D0x10) at = /usr/src/sys/kern/vfs_vnops.c:1699 #25 0xffffffff80a7c52e in devfs_ioctl_f (fp=3D0xffffffff81e8ce70 = , com=3D0, data=3D0xffffffff812a0000, cred=3D0x1, td=3D0x10) = at = /usr/src/sys/fs/devfs/devfs_vnops.c:864 #26 0xffffffff80c64672 in fo_ioctl (fp=3D0xfffff8000ce2cb40, = com=3D3230692865, data=3D0x1d0, active_cred=3D0x1, td=3D) = at = /usr/src/sys/sys/file.h:365 #27 kern_ioctl (td=3Dtd@entry=3D0xfffffe0160936000, fd=3D= , = com=3Dcom@entry=3D3230692865, data=3D0x1d0 , data@entry=3D0xfffff80032601b00 "") at /usr/src/sys/kern/sys_generic.c:803 #28 0xffffffff80c643ba in sys_ioctl (td=3D0xfffffe0160936000, = uap=3D0xfffffe01609363f8) at /usr/src/sys/kern/sys_generic.c:711 #29 0xffffffff810cf3be in syscallenter (td=3D) at = /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189 #30 amd64_syscall (td=3D0xfffffe0160936000, traced=3D0) at = /usr/src/sys/amd64/amd64/trap.c:1200 #31 #32 0x00003dd9cbd7f94a in ?? () Backtrace stopped: Cannot access memory at address 0x3ddc1d44ae58 (kgdb) fr 16 #16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, = tostart=3Dtostart@entry=3D0xfffffe015f2dd858) at = /usr/src/sys/amd64/vmm/vmm.c:2508 2508 mtx_lock(&vm->rendezvous_mtx); I believe WITNESS is upset that we=E2=80=99re going to potentially sleep = doing = mtx_lock(&vm->rendezvous_mtx); in vm_start_cpus() when we=E2=80=99ve done= = critical_enter() in vm_run(). Best regards, Kristof --=_MailMate_9195AF87-15B8-46D2-B75E-5791331F0D7F_= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On 18 Nov 2022, at 19:26, John Baldwin wrote:

The branch main has been updated by= jhb:

URL: https://cgit.FreeBSD.org/src/co= mmit/?id=3Dc0f35dbf19c3c8825bd2b321d8efd582807d1940

commit c0f35dbf19c3c8825bd2b321d8efd582807d1940
Author: John Baldwin <jhb@FreeBSD.org>
AuthorDate: 2022-11-18 18:05:10 +0000
Commit: John Baldwin <jhb@FreeBSD.org>
CommitDate: 2022-11-18 18:25:38 +0000

vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPI= s.

Retire the boot_state member of struct vlapic and ins= tead use a cpuset
in the VM to track vCPUs waiting for STARTUP IPIs. INIT IPIs add
vCPUs to this set, and STARTUP IPIs remove vCPUs from the set.
STARTUP IPIs are only reported to userland for vCPUs that were remove= d
from the set.

In particular, this permits a subsequent change to al= locate vCPUs on
demand when the vCPU may not be allocated until after a STARTUP IPI i= s
reported to userland.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37173
---
sys/amd64/include/vmm.h | 3 +++
sys/amd64/vmm/io/vlapic.c | 46 ++++++++++--------------------------= ------
sys/amd64/vmm/io/vlapic_priv.h | 7 -------
sys/amd64/vmm/vmm.c | 27 +++++++++++++++++++++++++
4 files changed, 41 insertions(+), 42 deletions(-)


I=E2=80=99m seeing a panic starting a bhyve guest, and I = think this commit might be the responsible one:

lo=
gin: panic: acquiring blockable sleep lock with spinlock or critical sect=
ion held (sleep mutex) vm rendezvous lock @ /usr/src/sys/amd64/vmm/vmm.c:=
2508
cpuid =3D 14
time =3D 1669035212
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe015f2=
dd530
vpanic() at vpanic+0x151/frame 0xfffffe015f2dd580
panic() at panic+0x43/frame 0xfffffe015f2dd5e0
witness_checkorder() at witness_checkorder+0xd3e/frame 0xfffffe015f2dd7a0=

__mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe015f2dd7f0
vm_start_cpus() at vm_start_cpus+0x31/frame 0xfffffe015f2dd820
vlapic_icrlo_write_handler() at vlapic_icrlo_write_handler+0x30c/frame 0x=
fffffe015f2dd8a0
vmx_run() at vmx_run+0x20ed/frame 0xfffffe015f2dd9c0
vm_run() at vm_run+0x1d2/frame 0xfffffe015f2ddaa0
vmmdev_ioctl() at vmmdev_ioctl+0x644/frame 0xfffffe015f2ddb40
devfs_ioctl() at devfs_ioctl+0xcd/frame 0xfffffe015f2ddb90
vn_ioctl() at vn_ioctl+0x131/frame 0xfffffe015f2ddca0
devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe015f2ddcc0
kern_ioctl() at kern_ioctl+0x202/frame 0xfffffe015f2ddd30
sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe015f2dde00
amd64_syscall() at amd64_syscall+0x12e/frame 0xfffffe015f2ddf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe015f2ddf3=
0
--- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x3dd9cbd7f94a, rsp =3D 0=
x3ddc1d44ae58, rbp =3D 0x3ddc1d44af10 ---

And kgdb=E2=80=99s backtrace:

(k=
gdb) bt
#0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:59
#1  dump_savectx () at /usr/src/sys/kern/kern_shutdown.c:405
#2  0xffffffff80bec678 in dumpsys (di=3D0x0) at /usr/src/sys/x86/include/=
dump.h:87
#3  doadump (textdump=3Dtextdump@entry=3D0) at /usr/src/sys/kern/kern_shu=
tdown.c:434
#4  0xffffffff804b403a in db_dump (dummy=3D<optimized out>, dummy2=3D=
<unavailable>, dummy3=3D<unavailable>, dummy4=3D<unavailab=
le>) at /usr/src/sys/ddb/db_command.c:593
#5  0xffffffff804b3e40 in db_command (last_cmdp=3D<optimized out>, =
cmd_table=3D<optimized out>, dopager=3Dtrue) at /usr/src/sys/ddb/db=
_command.c:506
#6  0xffffffff804b3b0d in db_command_loop () at /usr/src/sys/ddb/db_comma=
nd.c:553
#7  0xffffffff804b71a6 in db_trap (type=3D<optimized out>, code=3D&=
lt;optimized out>) at /usr/src/sys/ddb/db_main.c:270
#8  0xffffffff80c3b89e in kdb_trap (type=3Dtype@entry=3D3, code=3D<una=
vailable>, code@entry=3D0, tf=3Dtf@entry=3D0xfffffe015f2dd470) at /usr=
/src/sys/kern/subr_kdb.c:745
#9  0xffffffff810ce577 in trap (frame=3D0xfffffe015f2dd470) at /usr/src/s=
ys/amd64/amd64/trap.c:611
#10 <signal handler called>
#11 kdb_enter (why=3D<optimized out>, msg=3D<optimized out>) =
at /usr/src/sys/kern/subr_kdb.c:509
#12 0xffffffff80bec822 in vpanic (fmt=3D<optimized out>, ap=3Dap@en=
try=3D0xfffffe015f2dd5c0) at /usr/src/sys/kern/kern_shutdown.c:967
#13 0xffffffff80bec5c3 in panic (fmt=3D0xffffffff81e8ce70 <cnputs_mtx&=
gt; "\314:)\201\377\377\377\377") at /usr/src/sys/kern/kern_shu=
tdown.c:903
#14 0xffffffff80c5ea4e in witness_checkorder (lock=3D0xfffff804d585d138, =
flags=3D<optimized out>, file=3D<optimized out>, line=3D2508,=
 interlock=3D<optimized out>)
    at /usr/src/sys/kern/subr_witness.c:1202
#15 0xffffffff80bc6d04 in __mtx_lock_flags (c=3D0xfffff804d585d150, opts=3D=
0, file=3D0xffffffff8322542d "/usr/src/sys/amd64/vmm/vmm.c", li=
ne=3D2508) at /usr/src/sys/kern/kern_mutex.c:278
#16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, tostart=
=3Dtostart@entry=3D0xfffffe015f2dd858) at /usr/src/sys/amd64/vmm/vmm.c:25=
08
#17 0xffffffff83211b5c in vlapic_icrlo_write_handler (vlapic=3Dvlapic@ent=
ry=3D0xfffff8048dc14a80, retu=3Dretu@entry=3D0xfffffe015f2dd950) at /usr/=
src/sys/amd64/vmm/io/vlapic.c:1172
#18 0xffffffff8321977d in vmx_handle_apic_write (vcpu=3D0xfffff8048db3ac0=
0, vlapic=3D0xfffff8048dc14a80, qual=3D768) at /usr/src/sys/amd64/vmm/int=
el/vmx.c:2184
#19 vmx_exit_process (vmx=3D0xfffff804d585b000, vcpu=3D0xfffff8048db3ac00=
, vmexit=3D0xfffff804a7c1a688) at /usr/src/sys/amd64/vmm/intel/vmx.c:2767=

#20 vmx_run (vcpui=3D0xfffff8048db3ac00, rip=3D<optimized out>, pma=
p=3D0xfffffe003dce1530, evinfo=3D0xfffffe015f2dd9d8) at /usr/src/sys/amd6=
4/vmm/intel/vmx.c:3174
#21 0xffffffff83202572 in vm_run (vcpu=3Dvcpu@entry=3D0xfffff804a7c1a600,=
 vme_user=3Dvme_user@entry=3D0xfffff80032601b08) at /usr/src/sys/amd64/vm=
m/vmm.c:1873
#22 0xffffffff83205ab4 in vmmdev_ioctl (cdev=3D<optimized out>, cmd=
=3D3230692865, data=3D<optimized out>, fflag=3D<optimized out>=
;, td=3D<optimized out>) at /usr/src/sys/amd64/vmm/vmm_dev.c:565
#23 0xffffffff80a7be7d in devfs_ioctl (ap=3D0xfffffe015f2ddba8) at /usr/s=
rc/sys/fs/devfs/devfs_vnops.c:933
#24 0xffffffff80cf6421 in vn_ioctl (fp=3D0xfffff8000ce2cb40, com=3D<op=
timized out>, data=3D0xfffff80032601b00, active_cred=3D0xfffff8000c4d0=
800, td=3D0x10) at /usr/src/sys/kern/vfs_vnops.c:1699
#25 0xffffffff80a7c52e in devfs_ioctl_f (fp=3D0xffffffff81e8ce70 <cnpu=
ts_mtx>, com=3D0, data=3D0xffffffff812a0000, cred=3D0x1, td=3D0x10) at=
 /usr/src/sys/fs/devfs/devfs_vnops.c:864
#26 0xffffffff80c64672 in fo_ioctl (fp=3D0xfffff8000ce2cb40, com=3D323069=
2865, data=3D0x1d0, active_cred=3D0x1, td=3D<optimized out>) at /us=
r/src/sys/sys/file.h:365
#27 kern_ioctl (td=3Dtd@entry=3D0xfffffe0160936000, fd=3D<optimized ou=
t>, com=3Dcom@entry=3D3230692865, data=3D0x1d0 <error: Cannot acces=
s memory at address 0x1d0>, data@entry=3D0xfffff80032601b00 "&quo=
t;)
    at /usr/src/sys/kern/sys_generic.c:803
#28 0xffffffff80c643ba in sys_ioctl (td=3D0xfffffe0160936000, uap=3D0xfff=
ffe01609363f8) at /usr/src/sys/kern/sys_generic.c:711
#29 0xffffffff810cf3be in syscallenter (td=3D<optimized out>) at /u=
sr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189
#30 amd64_syscall (td=3D0xfffffe0160936000, traced=3D0) at /usr/src/sys/a=
md64/amd64/trap.c:1200
#31 <signal handler called>
#32 0x00003dd9cbd7f94a in ?? ()
Backtrace stopped: Cannot access memory at address 0x3ddc1d44ae58
(kgdb) fr 16
#16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, tostart=
=3Dtostart@entry=3D0xfffffe015f2dd858) at /usr/src/sys/amd64/vmm/vmm.c:25=
08
2508		mtx_lock(&vm->rendezvous_mtx);

I believe WITNESS is upset that we=E2=80=99re going to po= tentially sleep doing mtx_lock(&vm->rendezvous_mtx); in vm_start_c= pus() when we=E2=80=99ve done critical_enter() in vm_run().

Best regards,
Kristof

--=_MailMate_9195AF87-15B8-46D2-B75E-5791331F0D7F_=--