FreeBSD-main-amd64-test - Build #19947 - Failure

FreeBSD-main-amd64-test - Build #19947 (0d2224733e970aaa67a4e1af7b340044adda92f6) - Failure

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19947/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19947/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19947/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

8bc792b3845ecf64145fda7964a7b5cc54dea777 by mhorne:
i386: take pcb and fpu area into account in GET_STACK_USAGE

b02908b051c897117ebbc3dcb55848383b17c435 by mhorne:
arm64, powerpc: fix calculation of 'used' in GET_STACK_USAGE

0d2224733e970aaa67a4e1af7b340044adda92f6 by mhorne:
Implement GET_STACK_USAGE on remaining archs



The end of the build log:

[...truncated 3.92 MB...]

Tracing command soaiod1 pid 2203 tid 100138 td 0xfffffe00a6a40ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5db9d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5db9d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5db9dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5db9e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5db9eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5db9ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5db9f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5db9f30
--- trap 0xc, rip = 0x258549bd70ea, rsp = 0x7fffffa06058, rbp = 0x7fffffa06070 ---

Tracing command soaiod2 pid 2204 tid 100137 td 0xfffffe00a6a411e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dbed70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dbed90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dbedd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dbee10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dbeeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dbeef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dbef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dbef30
--- trap 0xc, rip = 0x392c1ea890ea, rsp = 0x7fffff5e4a58, rbp = 0x7fffff5e4a70 ---

Tracing command soaiod3 pid 2205 tid 100136 td 0xfffffe00a6a41900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc3d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc3d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc3dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc3e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc3eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dc3ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc3f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc3f30
--- trap 0xc, rip = 0x1eef290030ea, rsp = 0x7fffff966828, rbp = 0x7fffff966840 ---

Tracing command soaiod4 pid 2206 tid 100135 td 0xfffffe00a6a42020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc8d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc8d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc8dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc8e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc8eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dc8ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc8f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc8f30
--- trap 0xc, rip = 0xccfee7f50ea, rsp = 0x7fffff95d758, rbp = 0x7fffff95d770 ---

Tracing command devd pid 473 tid 100103 td 0xfffffe00767dd740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d9ea60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d9ea80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d9eac0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5d9eb10
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00a5d9eb50
_cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe00a5d9ebd0
kern_select() at kern_select+0xa6f/frame 0xfffffe00a5d9edc0
sys_select() at sys_select+0x56/frame 0xfffffe00a5d9ee00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5d9ef30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5d9ef30
--- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28b2ba, rsp = 0x7fffffe23688, rbp = 0x7fffffe25820 ---

Tracing command auditd pid 673 tid 100100 td 0xfffffe009db1e720
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5da8b50
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5da8b70
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5da8bb0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5da8c00
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5da8c10
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5da8cb0
audit_read() at audit_read+0x9c/frame 0xfffffe00a5da8ce0
devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe00a5da8d40
dofileread() at dofileread+0x81/frame 0xfffffe00a5da8d90
sys_read() at sys_read+0xc0/frame 0xfffffe00a5da8e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5da8f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5da8f30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x27db7a48a4aa, rsp = 0x7fffffe0c178, rbp = 0x7fffffe0c1c0 ---

Tracing command sh pid 725 tid 100089 td 0xfffffe00767b5ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b90a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b90a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b90a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b90ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b90ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b90b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b90c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b90e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b90f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b90f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x215a6883668a, rsp = 0x7fffff445e28, rbp = 0x7fffff445ee0 ---

Tracing command sh pid 738 tid 100111 td 0xfffffe009db211e0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bf4a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bf4a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bf4a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063bf4ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063bf4ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063bf4b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063bf4c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063bf4e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063bf4f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063bf4f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2f7947be68a, rsp = 0x7fffff147448, rbp = 0x7fffff147500 ---

Tracing command sh pid 745 tid 100091 td 0xfffffe00767b4c80
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b86a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b86a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b86a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b86ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b86ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b86b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b86c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b86e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b86f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b86f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x19cf58e0668a, rsp = 0x7fffffe0dd78, rbp = 0x7fffffe0de30 ---

Tracing command kyua pid 746 tid 100093 td 0xfffffe00767b3e40
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b7ca10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b7ca30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b7ca70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b7cac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b7cad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b7cb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b7cc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b7ce00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b7cf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b7cf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x16a3395468a, rsp = 0x7ffffffbd208, rbp = 0x7ffffffbd240 ---

Tracing command KTLS pid 47869 tid 100118 td 0xfffffe009dbd63a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00765e7d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00765e7d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765e7d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00765e7e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00765e7ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765e7f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765e7f30
--- trap 0xc, rip = 0x23a8cddb00ea, rsp = 0x7fffff409ec8, rbp = 0x7fffff409ee0 ---

Tracing command KTLS pid 47869 tid 100850 td 0xfffffe00a6b14720
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c3bd00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c3bd20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c3bd60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c3be00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a6c3bef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c3bf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c3bf30
--- trap 0x16, rip = 0x3024a3e390ea, rsp = 0x202a88, rbp = 0x202aa0 ---

Tracing command KTLS pid 47869 tid 100851 td 0xfffffe00a6a6f020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c31d40
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c31d60
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c31da0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c31e40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe00a6c31ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c31f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c31f30
--- trap 0x16, rip = 0x3024a3e390ea, rsp = 0xc07a88, rbp = 0xc07aa0 ---

Tracing command kyua pid 1137 tid 100915 td 0xfffffe00ab170ac0 (CPU 1)
cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00097cde00
ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe00097cde10
trap() at trap+0x56/frame 0xfffffe00097cdf20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00097cdf20
--- trap 0x13, rip = 0xffffffff80f7b9be, rsp = 0xfffffe00a6d11790, rbp = 0xfffffe00a6d11790 ---
vm_radix_lookup() at vm_radix_lookup+0x1e/frame 0xfffffe00a6d11790
vm_object_collapse() at vm_object_collapse+0x36f/frame 0xfffffe00a6d11810
vm_object_deallocate() at vm_object_deallocate+0x312/frame 0xfffffe00a6d11840
vm_map_process_deferred() at vm_map_process_deferred+0xa9/frame 0xfffffe00a6d11860
vm_map_remove() at vm_map_remove+0xc9/frame 0xfffffe00a6d11890
exec_new_vmspace() at exec_new_vmspace+0x219/frame 0xfffffe00a6d11900
exec_elf64_imgact() at exec_elf64_imgact+0x877/frame 0xfffffe00a6d119f0
kern_execve() at kern_execve+0x70d/frame 0xfffffe00a6d11d80
sys_execve() at sys_execve+0x5a/frame 0xfffffe00a6d11e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a6d11f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a6d11f30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x16a3395176a, rsp = 0x7ffffffbcd78, rbp = 0x7ffffffbcd80 ---

Tracing command ng_queue pid 52799 tid 100087 td 0xfffffe006a452560
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b9ad80
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b9ada0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b9ade0
_sleep() at _sleep+0x2e7/frame 0xfffffe0063b9ae80
ngthread() at ngthread+0x88/frame 0xfffffe0063b9aef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b9af30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b9af30
--- trap 0xc, rip = 0xcfb79cf60ea, rsp = 0x7fffff3ac5c8, rbp = 0x7fffff3ac6c0 ---

Tracing command ng_queue pid 52799 tid 100882 td 0xfffffe00a6a61900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c1dd80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c1dda0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c1dde0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c1de80
ngthread() at ngthread+0x88/frame 0xfffffe00a6c1def0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c1df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c1df30
--- trap 0x80808080, rip = 0x8080808080808080, rsp = 0x2a5b6b24c720, rbp = 0x1 ---

Tracing command g_mp_kt pid 44885 tid 100151 td 0xfffffe00a6a5f560
sched_switch() at sched_switch+0x559/frame 0xfffffe009d987da0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d987dc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d987e00
_sleep() at _sleep+0x2e7/frame 0xfffffe009d987ea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe009d987ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d987f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d987f30
--- trap 0xc, rip = 0x18c62f70b0ea, rsp = 0x7fffff83bc68, rbp = 0x7fffff83bd60 ---
db:0:kdb.enter.panic> dump
Dumping 480 out of 4060 MB:..4%..14%..24%..34%..44%..54%..64%..74%..84%..94%
Dump complete
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19947' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./run-kyua.sh
x ./run.sh
x ./disable-zfs-tests.sh
x ./disable-notyet-tests.sh
x ./auto-shutdown
x ./disable-dtrace-tests.sh
+ rm -f test-report.txt test-report.xml
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins5010486139867192656.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '0d2224733e970aaa67a4e1af7b340044adda92f6', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../0d2224733e970aaa67a4e1af7b340044adda92f6/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address