FreeBSD-main-amd64-test - Build #19904 - Still Failing

FreeBSD-main-amd64-test - Build #19904 (913c07a04998b8cd11dc428f5d00fae7dd392244) - Still Failing

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19904/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19904/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19904/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

913c07a04998b8cd11dc428f5d00fae7dd392244 by mav:
hwpmc: Add IDs for Intel Comet/Ice/Tiger/Rocketlake CPUs.



The end of the build log:

[...truncated 3.92 MB...]
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5d8af30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5d8af30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x3bdfc241f15a, rsp = 0x7fffffa99f38, rbp = 0x7fffffa99f80 ---

Tracing command sh pid 725 tid 100090 td 0xfffffe00767b4c80
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b86a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b86a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b86a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b86ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b86ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b86b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b86c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b86e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b86f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b86f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x1332e4b8633a, rsp = 0x7fffffd8bc58, rbp = 0x7fffffd8bd10 ---

Tracing command ng_queue pid 51936 tid 100111 td 0xfffffe00767d1720
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bc7d80
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bc7da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bc7de0
_sleep() at _sleep+0x2e7/frame 0xfffffe0063bc7e80
ngthread() at ngthread+0x88/frame 0xfffffe0063bc7ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bc7f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bc7f30
--- trap 0xc, rip = 0x17d78b97cd9a, rsp = 0x7fffffff3458, rbp = 0x7fffffff3550 ---

Tracing command ng_queue pid 51936 tid 100865 td 0xfffffe00a6a66900
sched_switch() at sched_switch+0x559/frame 0xfffffe009d9d7d80
mi_switch() at mi_switch+0x157/frame 0xfffffe009d9d7da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d9d7de0
_sleep() at _sleep+0x2e7/frame 0xfffffe009d9d7e80
ngthread() at ngthread+0x88/frame 0xfffffe009d9d7ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d9d7f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d9d7f30
--- trap 0xc, rip = 0x2346c305bd9a, rsp = 0x202f48, rbp = 0x202f60 ---

Tracing command sh pid 738 tid 100109 td 0xfffffe009db54900
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bf9a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bf9a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bf9a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063bf9ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063bf9ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063bf9b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063bf9c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063bf9e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063bf9f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063bf9f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x1b2e9a8c633a, rsp = 0x7fffff7c3b18, rbp = 0x7fffff7c3bd0 ---

Tracing command sh pid 745 tid 100101 td 0xfffffe00767d5740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d9ea10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d9ea30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d9ea70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5d9eac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5d9ead0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5d9eb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5d9ec00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5d9ee00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5d9ef30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5d9ef30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x37bd60c1e33a, rsp = 0x7fffff4225b8, rbp = 0x7fffff422670 ---

Tracing command kyua pid 746 tid 100104 td 0xfffffe00767d4900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d94a10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d94a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d94a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5d94ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5d94ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5d94b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5d94c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5d94e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5d94f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5d94f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x3970f45a433a, rsp = 0x7fffff673308, rbp = 0x7fffff673340 ---

Tracing command g_mp_kt pid 44022 tid 100136 td 0xfffffe00a6a18020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc6da0
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc6dc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc6e00
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5dc6ea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe00a5dc6ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc6f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc6f30
--- trap 0xc, rip = 0x71ef2315d9a, rsp = 0x7fffff2bb108, rbp = 0x7fffff2bb120 ---

Tracing command aiod3 pid 1305 tid 100094 td 0xfffffe00767b3000
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b72d60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b72d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b72dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063b72e00
_sleep() at _sleep+0x2ce/frame 0xfffffe0063b72ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063b72ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b72f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b72f30
--- trap 0xc, rip = 0x105c9c71d9a, rsp = 0x7fffff51c5e8, rbp = 0x7fffff51c600 ---

Tracing command aiod10 pid 1312 tid 100087 td 0xfffffe00767b61e0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b95d60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b95d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b95dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063b95e00
_sleep() at _sleep+0x2ce/frame 0xfffffe0063b95ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063b95ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b95f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b95f30
--- trap 0xc, rip = 0x101873941d9a, rsp = 0x7fffffa70238, rbp = 0x7fffffa70250 ---

Tracing command aiod13 pid 1315 tid 100120 td 0xfffffe009dbe4e40
sched_switch() at sched_switch+0x559/frame 0xfffffe00765d8d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00765d8d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765d8dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00765d8e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00765d8ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00765d8ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765d8f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765d8f30
--- trap 0xc, rip = 0x16205d3b3d9a, rsp = 0x7fffff6f7f48, rbp = 0x7fffff6f7f60 ---

Tracing command aiod19 pid 1321 tid 100092 td 0xfffffe00767b3e40
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b7cd60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b7cd80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b7cdc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063b7ce00
_sleep() at _sleep+0x2ce/frame 0xfffffe0063b7cea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063b7cef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b7cf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b7cf30
--- trap 0xc, rip = 0x121e426d3d9a, rsp = 0x7fffff554148, rbp = 0x7fffff554160 ---

Tracing command soaiod1 pid 1341 tid 100134 td 0xfffffe00a6a47000
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dd0d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dd0d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dd0dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dd0e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dd0eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dd0ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dd0f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dd0f30
--- trap 0xc, rip = 0x28eda08d1d9a, rsp = 0x7fffffc0a928, rbp = 0x7fffffc0a940 ---

Tracing command soaiod2 pid 1342 tid 100133 td 0xfffffe009dbe71e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dd5d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dd5d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dd5dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dd5e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dd5eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dd5ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dd5f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dd5f30
--- trap 0xc, rip = 0x35087cc21d9a, rsp = 0x7fffff896ee8, rbp = 0x7fffff896f00 ---

Tracing command soaiod3 pid 1343 tid 100132 td 0xfffffe009dbe7900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5ddad70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5ddad90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5ddadd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5ddae10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5ddaeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5ddaef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5ddaf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5ddaf30
--- trap 0xc, rip = 0x213f50b77d9a, rsp = 0x7fffffe74d68, rbp = 0x7fffffe74d80 ---

Tracing command soaiod4 pid 1344 tid 100131 td 0xfffffe009dbe8020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5ddfd70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5ddfd90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5ddfdd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5ddfe10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5ddfeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5ddfef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5ddff30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5ddff30
--- trap 0xc, rip = 0x2f21b6aa5d9a, rsp = 0x7fffffc5fd88, rbp = 0x7fffffc5fda0 ---

Tracing command KTLS pid 47006 tid 100138 td 0xfffffe00a6a171e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dbcd00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dbcd20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dbcd60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5dbce00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a5dbcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dbcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dbcf30
--- trap 0xc, rip = 0xd16bc1f0d9a, rsp = 0x7fffff38fee8, rbp = 0x7fffff38ff00 ---

Tracing command KTLS pid 47006 tid 100833 td 0xfffffe00a6a163a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5db2d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5db2d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5db2d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5db2e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a5db2ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5db2f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5db2f30
--- trap 0xc, rip = 0x2808c52e8b6c, rsp = 0x7fffffa6d578, rbp = 0x7fffffa6d5a0 ---

Tracing command KTLS pid 47006 tid 100834 td 0xfffffe00a6a67020
sched_switch() at sched_switch+0x559/frame 0xfffffe009d9dcd40
mi_switch() at mi_switch+0x157/frame 0xfffffe009d9dcd60
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d9dcda0
_sleep() at _sleep+0x2e7/frame 0xfffffe009d9dce40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe009d9dcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d9dcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d9dcf30
--- trap 0xc, rip = 0x2808c52e8b6c, rsp = 0x202ee8, rbp = 0x202f10 ---
db:0:kdb.enter.panic> dump
Dumping 464 out of 4060 MB:..4%..11%..21%..31%..42%..52%..62%..73%..83%..93%
Dump complete
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19904' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-zfs-tests.sh
x ./disable-notyet-tests.sh
x ./auto-shutdown
x ./run.sh
x ./run-kyua.sh
x ./disable-dtrace-tests.sh
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins5324629643882917862.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '913c07a04998b8cd11dc428f5d00fae7dd392244', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../913c07a04998b8cd11dc428f5d00fae7dd392244/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address