FreeBSD-main-amd64-test - Build #19880 - Failure

FreeBSD-main-amd64-test - Build #19880 (319e9fc642a13ac20e89d0bc05f8a09d1da5a502) - Failure

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19880/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19880/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19880/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

c36f90417b734e56ef938cc2d19601001c709f40 by kp:
pf: remove unused variables

18d04cd2d42c16209c8f615bb9735dded328915b by kp:
pf: align IPv6 dummynet handling with IPv4

90c55481b2731224f9e3a013bb7957dfa3251ea8 by kp:
pf: fix netpfil.common.dummynet:pf_nat test

76fe9df7a40c7f1e185534565f7470b50110acd8 by kp:
RELNOTES: mention 6rd support in if_stf

319e9fc642a13ac20e89d0bc05f8a09d1da5a502 by glebius:
ng_bridge: count item allocation failures as memory failures



The end of the build log:

[...truncated 4.24 MB...]
Tracing command zfskern pid 21559 tid 100145 td 0xfffffe00a77a01e0
sched_switch() at sched_switch+0x559/frame 0xfffffe009d992dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d992de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d992e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d992ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d992ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d992f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d992f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100146 td 0xfffffe00a779fac0
sched_switch() at sched_switch+0x559/frame 0xfffffe009d98ddc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d98dde0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d98de20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d98dec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d98def0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d98df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d98df30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100147 td 0xfffffe00a779f3a0
sched_switch() at sched_switch+0x559/frame 0xfffffe009d988dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d988de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d988e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d988ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d988ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d988f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d988f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100148 td 0xfffffe00a779ec80
sched_switch() at sched_switch+0x559/frame 0xfffffe009d983dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d983de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d983e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d983ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d983ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d983f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d983f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100149 td 0xfffffe00a779e560
sched_switch() at sched_switch+0x559/frame 0xfffffe009d97edc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d97ede0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d97ee20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d97eec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d97eef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d97ef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d97ef30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100150 td 0xfffffe00a779de40
sched_switch() at sched_switch+0x559/frame 0xfffffe009d979dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d979de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d979e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d979ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d979ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d979f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d979f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100151 td 0xfffffe00a779d720
sched_switch() at sched_switch+0x559/frame 0xfffffe009d974dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d974de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d974e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d974ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d974ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d974f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d974f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100152 td 0xfffffe00a77e23a0
sched_switch() at sched_switch+0x559/frame 0xfffffe009d96fdc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d96fde0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d96fe20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d96fec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d96fef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d96ff30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d96ff30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100153 td 0xfffffe00a77e1c80
sched_switch() at sched_switch+0x559/frame 0xfffffe009d96ac20
mi_switch() at mi_switch+0x157/frame 0xfffffe009d96ac40
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d96ac80
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d96acc0
_cv_timedwait_sbt() at _cv_timedwait_sbt+0x19a/frame 0xfffffe009d96ad40
l2arc_feed_thread() at l2arc_feed_thread+0x274/frame 0xfffffe009d96aef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d96af30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d96af30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100154 td 0xfffffe00a77e1560
sched_switch() at sched_switch+0x559/frame 0xfffffe009d965dc0
mi_switch() at mi_switch+0x157/frame 0xfffffe009d965de0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d965e20
_sleep() at _sleep+0x2e7/frame 0xfffffe009d965ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe009d965ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d965f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d965f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command zfskern pid 21559 tid 100155 td 0xfffffe00a77e0e40
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bfbcc0
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bfbce0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bfbd20
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063bfbd70
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063bfbd80
_cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe0063bfbde0
zfs_zevent_wait() at zfs_zevent_wait+0x47/frame 0xfffffe0063bfbe00
sysevent_worker() at sysevent_worker+0x49/frame 0xfffffe0063bfbef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bfbf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bfbf30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod2 pid 21568 tid 100115 td 0xfffffe009db5e1e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00773f2d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00773f2d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00773f2dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00773f2e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00773f2ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00773f2ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00773f2f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00773f2f30
--- trap 0xc, rip = 0x3205f33d8d9a, rsp = 0x7fffff2bf5f8, rbp = 0x7fffff2bf610 ---

Tracing command aiod3 pid 21569 tid 100113 td 0xfffffe009db5f020
sched_switch() at sched_switch+0x559/frame 0xfffffe00773fcd60
mi_switch() at mi_switch+0x157/frame 0xfffffe00773fcd80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00773fcdc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00773fce00
_sleep() at _sleep+0x2ce/frame 0xfffffe00773fcea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00773fcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00773fcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00773fcf30
--- trap 0xc, rip = 0x32a4e950dd9a, rsp = 0x7ffffff363f8, rbp = 0x7ffffff36410 ---

Tracing command aiod4 pid 21570 tid 100122 td 0xfffffe009dbc63a0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bcfd60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bcfd80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bcfdc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063bcfe00
_sleep() at _sleep+0x2ce/frame 0xfffffe0063bcfea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063bcfef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bcff30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bcff30
--- trap 0xc, rip = 0x34ead2461d9a, rsp = 0x7fffff1a2708, rbp = 0x7fffff1a2720 ---

Tracing command KTLS pid 30037 tid 102171 td 0xfffffe00a8af2e40
sched_switch() at sched_switch+0x559/frame 0xfffffe00a8c40d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a8c40d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a8c40d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a8c40e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a8c40ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a8c40f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a8c40f30
--- trap 0xc, rip = 0x32e674716d9a, rsp = 0x7fffffabcbc8, rbp = 0x7fffffabcbe0 ---

Tracing command KTLS pid 30037 tid 102182 td 0xfffffe00a8abd740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a782cd00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a782cd20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a782cd60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a782ce00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a782cef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a782cf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a782cf30
--- trap 0xc, rip = 0x2d84b63f3d9a, rsp = 0x202ea8, rbp = 0x202ec0 ---

Tracing command KTLS pid 30037 tid 102183 td 0xfffffe00a8ace560
sched_switch() at sched_switch+0x559/frame 0xfffffe00a8c3bd40
mi_switch() at mi_switch+0x157/frame 0xfffffe00a8c3bd60
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a8c3bda0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a8c3be40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe00a8c3bef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a8c3bf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a8c3bf30
--- trap 0xc, rip = 0x1d6bd95e6b6c, rsp = 0x202ee8, rbp = 0x202f10 ---

Tracing command kyua pid 83695 tid 101321 td 0xfffffe00a8aa21e0 (CPU 0)
cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff81ffc690
ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xffffffff81ffc6a0
trap() at trap+0x56/frame 0xffffffff81ffc7b0
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81ffc7b0
--- trap 0x13, rip = 0xffffffff810c993d, rsp = 0xfffffe00a8d08750, rbp = 0xfffffe00a8d08890 ---
pmap_remove_pages() at pmap_remove_pages+0x51d/frame 0xfffffe00a8d08890
exec_new_vmspace() at exec_new_vmspace+0x209/frame 0xfffffe00a8d08900
exec_elf64_imgact() at exec_elf64_imgact+0x877/frame 0xfffffe00a8d089f0
kern_execve() at kern_execve+0x70d/frame 0xfffffe00a8d08d80
sys_execve() at sys_execve+0x5a/frame 0xfffffe00a8d08e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a8d08f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a8d08f30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0xbb30187a41a, rsp = 0x7fffffa445a8, rbp = 0x7fffffa445b0 ---

Tracing command md0 pid 22437 tid 100116 td 0xfffffe009db5dac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00773edda0
mi_switch() at mi_switch+0x157/frame 0xfffffe00773eddc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00773ede00
_sleep() at _sleep+0x2e7/frame 0xfffffe00773edea0
md_kthread() at md_kthread+0xd5/frame 0xfffffe00773edef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00773edf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00773edf30
--- trap 0xc, rip = 0x9f292e75d9a, rsp = 0x7fffff6042e8, rbp = 0x7fffff6043e0 ---
db:0:kdb.enter.panic> dump
Dumping 478 out of 4058 MB:..4%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Dump complete
db:0:kdb.enter.panic> reset
cpu_reset: Restarting BSP
cpu_reset_proxy: Stopped CPU 1
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19880' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-zfs-tests.sh
x ./disable-notyet-tests.sh
x ./run-kyua.sh
x ./auto-shutdown
x ./run.sh
x ./disable-dtrace-tests.sh
+ rm -f test-report.txt test-report.xml
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins7095460515432773649.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '319e9fc642a13ac20e89d0bc05f8a09d1da5a502', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../319e9fc642a13ac20e89d0bc05f8a09d1da5a502/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address