FreeBSD-main-amd64-test - Build #19943 - Failure

FreeBSD-main-amd64-test - Build #19943 (aaa4cd7f3c933950582dfb4f0eac23a200dafe53) - Failure

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19943/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19943/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19943/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

aaa4cd7f3c933950582dfb4f0eac23a200dafe53 by otis:
Makefile.inc1: Silence an error in make packages



The end of the build log:

[...truncated 3.97 MB...]
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5ddad60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5ddad80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5ddadc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5ddae00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5ddaea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5ddaef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5ddaf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5ddaf30
--- trap 0xc, rip = 0x1d324d2b20ea, rsp = 0x7fffffa49868, rbp = 0x7fffffa49880 ---

Tracing command aiod3 pid 16786 tid 100099 td 0xfffffe00a6949720
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5de9d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5de9d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5de9dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5de9e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5de9ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5de9ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5de9f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5de9f30
--- trap 0xc, rip = 0x3423e8ea00ea, rsp = 0x7fffff74bcc8, rbp = 0x7fffff74bce0 ---

Tracing command aiod4 pid 16787 tid 100108 td 0xfffffe00a694d020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc1d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc1d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc1dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc1e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc1ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5dc1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc1f30
--- trap 0xc, rip = 0x1bb4ad8cd0ea, rsp = 0x7fffffba1c48, rbp = 0x7fffffba1c60 ---

Tracing command devd pid 473 tid 100103 td 0xfffffe00a6943900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dd5a60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dd5a80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dd5ac0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5dd5b10
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe00a5dd5b50
_cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe00a5dd5bd0
kern_select() at kern_select+0xa6f/frame 0xfffffe00a5dd5dc0
sys_select() at sys_select+0x56/frame 0xfffffe00a5dd5e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5dd5f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5dd5f30
--- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28b2ba, rsp = 0x7fffffdff798, rbp = 0x7fffffe01930 ---

Tracing command KTLS pid 76424 tid 100733 td 0xfffffe00a71d1c80
sched_switch() at sched_switch+0x559/frame 0xfffffe00a728bd00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a728bd20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a728bd60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a728be00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a728bef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a728bf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a728bf30
--- trap 0xc, rip = 0x14dfea2ea0ea, rsp = 0x7fffff466398, rbp = 0x7fffff4663b0 ---

Tracing command KTLS pid 76424 tid 101452 td 0xfffffe00a7705740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a7209d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a7209d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a7209d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a7209e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a7209ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a7209f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a7209f30
--- trap 0xc, rip = 0x37c650aa4b6c, rsp = 0x7fffff957898, rbp = 0x7fffff9578c0 ---

Tracing command KTLS pid 76424 tid 101453 td 0xfffffe00a7701000
sched_switch() at sched_switch+0x559/frame 0xfffffe00a7349d40
mi_switch() at mi_switch+0x157/frame 0xfffffe00a7349d60
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a7349da0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a7349e40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe00a7349ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a7349f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a7349f30
--- trap 0x16, rip = 0x2d691d49e0ea, rsp = 0x403a88, rbp = 0x403aa0 ---

Tracing command auditd pid 673 tid 100105 td 0xfffffe00a69431e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dd0b50
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dd0b70
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dd0bb0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5dd0c00
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5dd0c10
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5dd0cb0
audit_read() at audit_read+0x9c/frame 0xfffffe00a5dd0ce0
devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe00a5dd0d40
dofileread() at dofileread+0x81/frame 0xfffffe00a5dd0d90
sys_read() at sys_read+0xc0/frame 0xfffffe00a5dd0e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5dd0f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5dd0f30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x1ca9ec7164aa, rsp = 0x7fffff8a5d38, rbp = 0x7fffff8a5d80 ---

Tracing command sh pid 725 tid 100089 td 0xfffffe00767b53a0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b8ba10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b8ba30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b8ba70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b8bac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b8bad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b8bb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b8bc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b8be00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b8bf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b8bf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x212e928a868a, rsp = 0x7fffffe0cba8, rbp = 0x7fffffe0cc60 ---

Tracing command sh pid 738 tid 100111 td 0xfffffe00a694bac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5db2a10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5db2a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5db2a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5db2ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5db2ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5db2b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5db2c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5db2e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5db2f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5db2f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0xb1ddc5a268a, rsp = 0x7ffffff76498, rbp = 0x7ffffff76550 ---

Tracing command sh pid 745 tid 100113 td 0xfffffe00a694ac80
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5da8a10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5da8a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5da8a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5da8ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5da8ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5da8b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5da8c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5da8e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5da8f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5da8f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x24149b3c968a, rsp = 0x7fffffb5c068, rbp = 0x7fffffb5c120 ---

Tracing command kyua pid 746 tid 100109 td 0xfffffe00a694c900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dbca10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dbca30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dbca70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5dbcac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5dbcad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5dbcb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5dbcc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5dbce00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5dbcf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5dbcf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x397d8f28868a, rsp = 0x7fffffad5008, rbp = 0x7fffffad5040 ---

Tracing command md0 pid 17684 tid 100120 td 0xfffffe009dbf73a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00765ebda0
mi_switch() at mi_switch+0x157/frame 0xfffffe00765ebdc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765ebe00
_sleep() at _sleep+0x2e7/frame 0xfffffe00765ebea0
md_kthread() at md_kthread+0xd5/frame 0xfffffe00765ebef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765ebf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765ebf30
--- trap 0xc, rip = 0x1932a21cb0ea, rsp = 0x7fffff5182d8, rbp = 0x7fffff5183d0 ---

Tracing command kyua pid 30019 tid 101708 td 0xfffffe00ac37c900 (CPU 0)
cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81fe9310
ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81fe9320
trap() at trap+0x56/frame 0xffffffff81fe9430
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81fe9430
--- trap 0x13, rip = 0xffffffff80f6a03a, rsp = 0xfffffe00ac747740, rbp = 0xfffffe00ac747750 ---
vm_page_object_remove() at vm_page_object_remove+0xba/frame 0xfffffe00ac747750
vm_page_rename() at vm_page_rename+0x83/frame 0xfffffe00ac747790
vm_object_collapse() at vm_object_collapse+0x4d3/frame 0xfffffe00ac747810
vm_object_deallocate() at vm_object_deallocate+0x312/frame 0xfffffe00ac747840
vm_map_process_deferred() at vm_map_process_deferred+0xa9/frame 0xfffffe00ac747860
vm_map_remove() at vm_map_remove+0xc9/frame 0xfffffe00ac747890
exec_new_vmspace() at exec_new_vmspace+0x219/frame 0xfffffe00ac747900
exec_elf64_imgact() at exec_elf64_imgact+0x877/frame 0xfffffe00ac7479f0
kern_execve() at kern_execve+0x70d/frame 0xfffffe00ac747d80
sys_execve() at sys_execve+0x5a/frame 0xfffffe00ac747e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00ac747f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00ac747f30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x397d8f28576a, rsp = 0x7fffffad4b78, rbp = 0x7fffffad4b80 ---

Tracing command ng_queue pid 81354 tid 101463 td 0xfffffe00a751b900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a72dbd80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a72dbda0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a72dbde0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a72dbe80
ngthread() at ngthread+0x88/frame 0xfffffe00a72dbef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a72dbf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a72dbf30
--- trap 0xc, rip = 0x1e0890cf70ea, rsp = 0x7fffffa56c98, rbp = 0x7fffffa56d90 ---

Tracing command ng_queue pid 81354 tid 101484 td 0xfffffe00a71e1ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a72e0d80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a72e0da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a72e0de0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a72e0e80
ngthread() at ngthread+0x88/frame 0xfffffe00a72e0ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a72e0f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a72e0f30
--- trap 0x16, rip = 0x252acd3120ea, rsp = 0x202a88, rbp = 0x202aa0 ---

Tracing command g_mp_kt pid 73440 tid 100741 td 0xfffffe00a7519560
sched_switch() at sched_switch+0x559/frame 0xfffffe009d973da0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d973dc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d973e00
_sleep() at _sleep+0x2e7/frame 0xfffffe009d973ea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe009d973ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d973f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d973f30
--- trap 0xc, rip = 0xa31e42d00ea, rsp = 0x7fffffe42a18, rbp = 0x7fffffe42a30 ---
db:0:kdb.enter.panic> dump
Dumping 472 out of 4060 MB:..4%..11%..21%..31%..41%..51%..61%..72%..82%..92%
Dump complete
db:0:kdb.enter.panic> reset
cpu_reset: Restarting BSP
cpu_reset_proxy: Stopped CPU 1
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19943' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-zfs-tests.sh
x ./run.sh
x ./disable-dtrace-tests.sh
x ./auto-shutdown
x ./disable-notyet-tests.sh
x ./run-kyua.sh
+ rm -f test-report.txt test-report.xml
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins4463052152784776506.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'aaa4cd7f3c933950582dfb4f0eac23a200dafe53', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../aaa4cd7f3c933950582dfb4f0eac23a200dafe53/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address