FreeBSD-main-amd64-test - Build #21211 - Failure

From: <jenkins-admin_at_FreeBSD.org>
Date: Fri, 22 Apr 2022 16:50:35 UTC
FreeBSD-main-amd64-test - Build #21211 (f010b9c2b9b0a550656de8a5b5ad1469630c3e2c) - Failure

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21211/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21211/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21211/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

f010b9c2b9b0a550656de8a5b5ad1469630c3e2c by se:
pci: recognize "non-essential instrumentation" devices



The end of the build log:

[...truncated 4.74 MB...]
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
vmnet0: Ethernet address: 58:9c:fc:10:ff:a6
passed  [10.088s]
sys/net/if_epair:params  ->  epair0a: Ethernet address: 02:68:e6:3b:3f:0a
epair0b: Ethernet address: 02:68:e6:3b:3f:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.024s]
sys/net/if_gif:basic  ->  epair0a: Ethernet address: 02:e4:f1:84:e3:0a
epair0b: Ethernet address: 02:e4:f1:84:e3:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
gif0: link state changed to UP
gif0: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.194s]
sys/net/if_lagg_test:create  ->  tap0: Ethernet address: 58:9c:fc:10:d3:3e
tap1: Ethernet address: 58:9c:fc:00:0a:44
lagg0: link state changed to DOWN
lock order reversal:
 1st 0xffffffff81f9fdf0 in_control (in_control, sx) @ /usr/src/sys/netinet/in.c:316
 2nd 0xfffff80011920028 if_lagg sx (if_lagg sx, sx) @ /usr/src/sys/net/if_lagg.c:1259
lock order in_control -> if_lagg sx attempted at:
#0 0xffffffff80c7ffdd at witness_checkorder+0xbdd
#1 0xffffffff80c190a3 at _sx_xlock+0x63
#2 0xffffffff82717a4e at lagg_init+0x2e
#3 0xffffffff80d3fd0e at ether_ioctl+0x1be
#4 0xffffffff80dcb428 at in_control+0x9a8
#5 0xffffffff80d3a074 at ifioctl+0x7f4
#6 0xffffffff80c85d42 at kern_ioctl+0x202
#7 0xffffffff80c85a96 at sys_ioctl+0x126
#8 0xffffffff810f2b16 at amd64_syscall+0x6a6
#9 0xffffffff810c2efb at fast_syscall_common+0xf8
lock order reversal:
 1st 0xffffffff81f9fe68 in_multi_sx (in_multi_sx, sx) @ /usr/src/sys/netinet/in_mcast.c:1206
 2nd 0xfffff80011920028 if_lagg sx (if_lagg sx, sx) @ /usr/src/sys/net/if_lagg.c:1676
lock order in_multi_sx -> if_lagg sx attempted at:
#0 0xffffffff80c7ffdd at witness_checkorder+0xbdd
#1 0xffffffff80c190a3 at _sx_xlock+0x63
#2 0xffffffff82717c64 at lagg_ioctl+0xd4
#3 0xffffffff80d394be at if_addmulti+0x2be
#4 0xffffffff80dd0f6d at in_joingroup_locked+0x27d
#5 0xffffffff80dd0cc2 at in_joingroup+0x42
#6 0xffffffff80dcb46e at in_control+0x9ee
#7 0xffffffff80d3a074 at ifioctl+0x7f4
#8 0xffffffff80c85d42 at kern_ioctl+0x202
#9 0xffffffff80c85a96 at sys_ioctl+0x126
#10 0xffffffff810f2b16 at amd64_syscall+0x6a6
#11 0xffffffff810c2efb at fast_syscall_common+0xf8
passed  [0.113s]
sys/net/if_lagg_test:create_destroy_stress  ->  skipped: Skipping this test because it easily panics the machine  [0.024s]
sys/net/if_lagg_test:lacp_linkstate_destroy_stress  ->  skipped: https://bugs.freebsd.org/244168  [0.019s]
sys/net/if_lagg_test:set_ether  ->  tap0: Ethernet address: 58:9c:fc:10:d3:3e
tap1: Ethernet address: 58:9c:fc:00:0a:44
lagg0: link state changed to DOWN
passed  [0.063s]
sys/net/if_lagg_test:status_stress  ->  skipped: Skipping this test because it panics the machine fairly often  [0.020s]
sys/net/if_lagg_test:up_destroy_stress  ->  skipped: Skipping this test because it panics the machine fairly often  [0.019s]
sys/net/if_lagg_test:updown  ->  tap0: Ethernet address: 58:9c:fc:10:d3:3e
tap1: Ethernet address: 58:9c:fc:00:0a:44
lagg0: link state changed to DOWN
expected_failure: PR 226144 Upping a lagg interrface should automatically up its children: atf-check failed; see the output of the test for details  [0.055s]
sys/net/if_lagg_test:witness  ->  skipped: https://bugs.freebsd.org/244163 and https://bugs.freebsd.org/251726  [0.016s]
sys/net/if_stf:6rd  ->  epair0a: Ethernet address: 02:49:61:25:80:0a
epair0b: Ethernet address: 02:49:61:25:80:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
lo0: link state changed to UP
gif0: link state changed to UP
lo0: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.181s]
sys/net/if_stf:6rd_peer  ->  epair0a: Ethernet address: 02:7a:5b:83:49:0a
epair0b: Ethernet address: 02:7a:5b:83:49:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
lo0: link state changed to UP
lo0: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.301s]
sys/net/if_stf:6to4  ->  epair0a: Ethernet address: 02:f0:11:5c:f3:0a
epair0b: Ethernet address: 02:f0:11:5c:f3:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
lo0: link state changed to UP
gif0: link state changed to UP
lo0: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.176s]
sys/net/if_tun_test:235704  ->  tun0: changing name to 'foo'
passed  [0.077s]
sys/net/if_vlan:basic  ->  epair0a: Ethernet address: 02:e4:86:33:df:0a
epair0b: Ethernet address: 02:e4:86:33:df:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
vlan0: link state changed to DOWN
epair0b: link state changed to DOWN
vlan0: link state changed to DOWN
passed  [0.149s]
sys/net/if_vlan:bpf_pcp  ->  epair0a: Ethernet address: 02:13:d3:84:85:0a
epair0b: Ethernet address: 02:13:d3:84:85:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
gif0: link state changed to DOWN
gif0: link state changed to DOWN
epair0a: promiscuous mode enabled
epair0a: promiscuous mode disabled
epair0a: promiscuous mode enabled
epair0a: promiscuous mode disabled
epair0b: link state changed to DOWN
epair0a: link state changed to DOWN
vlan0: link state changed to DOWN
passed  [28.121s]
sys/net/if_vlan:qinq_deep  ->  epair0a: Ethernet address: 02:93:20:00:0e:0a
epair0b: Ethernet address: 02:93:20:00:0e:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
vlan0: link state changed to DOWN
vlan0: link state changed to DOWN
vlan1: link state changed to DOWN
vlan1: link state changed to DOWN
vlan2: link state changed to DOWN
vlan2: link state changed to DOWN
passed  [0.197s]
sys/net/if_vlan:qinq_dot  ->  epair0a: Ethernet address: 02:ed:a7:da:8c:0a
epair0b: Ethernet address: 02:ed:a7:da:8c:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
vlan5: link state changed to DOWN
vlan0: link state changed to DOWN
vlan5.42: link state changed to DOWN
vlan1: link state changed to DOWN
passed  [0.184s]
sys/net/if_vlan:qinq_legacy  ->  epair0a: Ethernet address: 02:a0:2c:f1:4b:0a
epair0b: Ethernet address: 02:a0:2c:f1:4b:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
Memory modified after free 0xfffff801f37c0000(248) val=deadc0dd @ 0xfffff801f37c00c0
panic: Most recently used by pf_ifnet

cpuid = 0
time = 1650646233
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00d2e409a0
vpanic() at vpanic+0x17f/frame 0xfffffe00d2e409f0
panic() at panic+0x43/frame 0xfffffe00d2e40a50
mtrash_ctor() at mtrash_ctor+0x81/frame 0xfffffe00d2e40a70
item_ctor() at item_ctor+0x2cb/frame 0xfffffe00d2e40ac0
malloc() at malloc+0x86/frame 0xfffffe00d2e40b00
vnet_ipsec_init() at vnet_ipsec_init+0xd7/frame 0xfffffe00d2e40b20
vnet_alloc() at vnet_alloc+0x144/frame 0xfffffe00d2e40b50
kern_jail_set() at kern_jail_set+0x177c/frame 0xfffffe00d2e40dd0
sys_jail_set() at sys_jail_set+0x40/frame 0xfffffe00d2e40e00
amd64_syscall() at amd64_syscall+0x6a6/frame 0xfffffe00d2e40f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00d2e40f30
--- syscall (507, FreeBSD ELF64, sys_jail_set), rip = 0x78e4e184e4a, rsp = 0x78e4b7eac68, rbp = 0x78e4b7ead50 ---
KDB: enter: panic
[ thread pid 42910 tid 102598 ]
Stopped at      kdb_enter+0x32: movq    $0,0x127cea3(%rip)
db:0:kdb.enter.panic> show pcpu
cpuid        = 0
dynamic pcpu = 0x7c4180
curthread    = 0xfffffe00cd22e000: pid 42910 tid 102598 critnest 1 "jail"
curpcb       = 0xfffffe00cd22e510
fpcurthread  = 0xfffffe00cd22e000: pid 42910 "jail"
idlethread   = 0xfffffe001122f3a0: tid 100003 "idle: cpu0"
self         = 0xffffffff82410000
curpmap      = 0xfffffe00cd5acb20
tssp         = 0xffffffff82410384
rsp0         = 0xfffffe00d2e41000
kcr3         = 0x80000001ebf7077e
ucr3         = 0x80000001e8039f7e
scr3         = 0x1e8039f7e
gs32p        = 0xffffffff82410404
ldt          = 0xffffffff82410444
tss          = 0xffffffff82410434
curvnet      = 0xfffff801ec9002c0
spin locks held:
db:0:kdb.enter.panic>  reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-21211' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-notyet-tests.sh
x ./run-kyua.sh
x ./run.sh
x ./disable-dtrace-tests.sh
x ./disable-zfs-tests.sh
x ./auto-shutdown
+ rm -f test-report.txt test-report.xml
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins3275450391456024125.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'f010b9c2b9b0a550656de8a5b5ad1469630c3e2c', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../f010b9c2b9b0a550656de8a5b5ad1469630c3e2c/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address