FreeBSD-main-amd64-test - Build #25631 - Still Failing

Reply: jenkins-admin_a_FreeBSD.org: "FreeBSD-main-amd64-test - Build #25632 - Fixed"
In reply to: jenkins-admin_a_FreeBSD.org: "FreeBSD-main-amd64-test - Build #25630 - Failure"
Go to: [ bottom of page ] [ top of archives ] [ this month ]
From: <jenkins-admin_at_FreeBSD.org>
Date: Thu, 24 Oct 2024 20:05:20 UTC
FreeBSD-main-amd64-test - Build #25631 (d021d3b3c67573822b01d27cb05ca937fc966843) - Still Failing

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/25631/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/25631/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/25631/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

f3a097d0312cbadefa9bbb00cf3c6af784f9fbb9 by kevans:
netstat: switch to using the sysctl-exported stats for live stats

656991b0c629038beddf1847a6c20318d3ac4181 by glebius:
locks: augment lock_class with lc_trylock method

efcb2ec8cb81ea44c58e41082c6d965f9493099f by glebius:
callout: provide CALLOUT_TRYLOCK flag

bffebc336f4ece4d18774c1ab8f555802cebf961 by glebius:
tcp: use CALLOUT_TRYLOCK for the TCP callout

d021d3b3c67573822b01d27cb05ca937fc966843 by glebius:
tcp: get rid of TDP_INTCPCALLOUT



The end of the build log:

[...truncated 4.16 MiB...]
epair0b: link state changed to UP
epair1a: Ethernet address: 02:06:69:f0:c9:0a
epair1b: Ethernet address: 02:06:69:f0:c9:0b
epair1a: link state changed to UP
epair1b: link state changed to UP
bridge0: link state changed to UP
epair0a: promiscuous mode enabled
epair1a: promiscuous mode enabled
epair0b: promiscuous mode enabled
carp: 1@epair0b: INIT -> BACKUP (initialization complete)
epair1b: promiscuous mode enabled
carp: 1@epair1b: INIT -> BACKUP (initialization complete)
carp: 1@epair0b: BACKUP -> MASTER (master timed out)
epair1a: promiscuous mode disabled
epair0a: promiscuous mode disabled
bridge0: link state changed to DOWN
epair0a: link state changed to DOWN
carp: 1@epair0b: MASTER -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair0b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair0b: promiscuous mode disabled
epair1a: link state changed to DOWN
carp: 1@epair1b: BACKUP -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair1b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair1b: promiscuous mode disabled
passed  [6.553s]
sys/netinet/carp:vrrp_v6  ->  bridge0: Ethernet address: 58:9c:fc:10:16:70
epair0a: Ethernet address: 02:55:0b:cb:7d:0a
epair0b: Ethernet address: 02:55:0b:cb:7d:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair1a: Ethernet address: 02:6e:9d:41:7a:0a
epair1b: Ethernet address: 02:6e:9d:41:7a:0b
epair1a: link state changed to UP
epair1b: link state changed to UP
bridge0: link state changed to UP
epair0a: promiscuous mode enabled
epair1a: promiscuous mode enabled
epair0b: promiscuous mode enabled
carp: 1@epair0b: INIT -> BACKUP (initialization complete)
epair1b: promiscuous mode enabled
carp: 1@epair1b: INIT -> BACKUP (initialization complete)
carp: 1@epair0b: BACKUP -> MASTER (master timed out)
epair1a: promiscuous mode disabled
epair0a: promiscuous mode disabled
bridge0: link state changed to DOWN
epair0a: link state changed to DOWN
carp: 1@epair0b: MASTER -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair0b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair0b: promiscuous mode disabled
epair1a: link state changed to DOWN
carp: 1@epair1b: BACKUP -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair1b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair1b: promiscuous mode disabled
passed  [6.483s]
sys/netinet/carp.py:TestCarp::test_source_mac  ->  epair0a: Ethernet address: 02:48:f1:33:4a:0a
epair0b: Ethernet address: 02:48:f1:33:4a:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
lo0: link state changed to UP
epair0a: promiscuous mode enabled
carp: 1@epair0a: INIT -> BACKUP (initialization complete)
carp: 1@epair0a: BACKUP -> MASTER (master timed out)
carp: 1@epair0a: MASTER -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair0a: promiscuous mode disabled
passed  [4.647s]
sys/netinet/carp.py:TestCarp::test_source_mac_vrrp  ->  epair0a: Ethernet address: 02:21:34:b8:d9:0a
epair0b: Ethernet address: 02:21:34:b8:d9:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
lo0: link state changed to UP
epair0a: promiscuous mode enabled
carp: 1@epair0a: INIT -> BACKUP (initialization complete)
carp: 1@epair0a: BACKUP -> MASTER (master timed out)
carp: 1@epair0a: MASTER -> INIT (hardware interface down)
carp: demoted by 240 to 240 (interface down)
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
carp: demoted by -240 to 0 (vhid removed)
epair0a: promiscuous mode disabled
passed  [5.195s]
sys/netinet/divert:ipdivert_ip_input_local_success  ->  epair0a: Ethernet address: 02:ce:e7:a8:f1:0a
epair0b: Ethernet address: 02:ce:e7:a8:f1:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
tun0: loop detected
epair0b: link state changed to DOWN
epair0a: link state changed to DOWN
passed  [1.205s]
sys/netinet/divert:ipdivert_ip_output_remote_success  ->  epair0a: Ethernet address: 02:0f:c4:51:8c:0a
epair0b: Ethernet address: 02:0f:c4:51:8c:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0b: link state changed to DOWN
epair0a: link state changed to DOWN
passed  [1.239s]
sys/netinet/fibs:fibs_ifroutes1_success  ->  epair0a: Ethernet address: 02:a8:8f:78:54:0a
epair0b: Ethernet address: 02:a8:8f:78:54:0b
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.134s]
sys/netinet/fibs_test:arpresolve_checks_interface_fib  ->  skipped: firewalls interfere with this test  [0.043s]
sys/netinet/fibs_test:default_route_with_multiple_fibs_on_same_subnet  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
passed  [0.060s]
sys/netinet/fibs_test:default_route_with_multiple_fibs_on_same_subnet_inet6  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
passed  [0.070s]
sys/netinet/fibs_test:loopback_and_network_routes_on_nondefault_fib  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
passed  [0.048s]
sys/netinet/fibs_test:loopback_and_network_routes_on_nondefault_fib_inet6  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
passed  [0.059s]
sys/netinet/fibs_test:same_ip_multiple_ifaces  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
tap2: Ethernet address: 58:9c:fc:10:3b:4f
tap3: Ethernet address: 58:9c:fc:10:ff:ed
expected_failure: kern/189088 Assigning the same IP to multiple interfaces in different FIBs creates a host route for only one: atf-check failed; see the output of the test for details  [0.071s]
sys/netinet/fibs_test:same_ip_multiple_ifaces_fib0  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
tap2: Ethernet address: 58:9c:fc:10:3b:4f
tap3: Ethernet address: 58:9c:fc:10:ff:ed
passed  [0.056s]
sys/netinet/fibs_test:same_ip_multiple_ifaces_inet6  ->  tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
tap0: Ethernet address: 58:9c:fc:10:35:51
tap1: Ethernet address: 58:9c:fc:10:51:40
passed  [0.115s]
sys/netinet/fibs_test:slaac_on_nondefault_fib6  ->  skipped: firewalls interfere with this test  [0.040s]
sys/netinet/fibs_test:subnet_route_with_multiple_fibs_on_same_subnet  ->  tap0: Ethernet address: 58:9c:fc:10:35:51


Fatal trap 9: general protection fault while in kernel mode
cpuid = 1; apic id = 01
instruction pointer	= 0x20:0xffffffff82912bc6
stack pointer	        = 0x28:0xfffffe008e7a5830
frame pointer	        = 0x28:0xfffffe008e7a5840
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 0 (netlink_socket (PID)
rdi: fffffe00111df818 rsi: 0000000000000004 rdx: ffffffff829484b8
rcx: 000000000000017f  r8: fffff80168127000  r9: fffff80168127000
rax: deadc0dedeadc0de rbx: fffff8021b348e80 rbp: fffffe008e7a5840
r10: 0000000000000003 r11: 0000000000000000 r12: fffff80168127000
r13: fffff80148542500 r14: fffff8003e65d500 r15: fffff8021b348e80
trap number		= 9
panic: general protection fault
cpuid = 1
time = 1729800319
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008e7a5570
vpanic() at vpanic+0x136/frame 0xfffffe008e7a56a0
panic() at panic+0x43/frame 0xfffffe008e7a5700
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe008e7a5760
calltrap() at calltrap+0x8/frame 0xfffffe008e7a5760
--- trap 0x9, rip = 0xffffffff82912bc6, rsp = 0xfffffe008e7a5830, rbp = 0xfffffe008e7a5840 ---
sctp_add_ifa_to_ifn() at sctp_add_ifa_to_ifn+0x46/frame 0xfffffe008e7a5840
sctp_add_addr_to_vrf() at sctp_add_addr_to_vrf+0x456/frame 0xfffffe008e7a58c0
sctp_addr_change() at sctp_addr_change+0xde/frame 0xfffffe008e7a5910
rt_addrmsg() at rt_addrmsg+0x121/frame 0xfffffe008e7a59e0
in_addprefix() at in_addprefix+0x157/frame 0xfffffe008e7a5a30
in_control_ioctl() at in_control_ioctl+0xac7/frame 0xfffffe008e7a5ad0
rtnl_handle_addr() at rtnl_handle_addr+0x412/frame 0xfffffe008e7a5cd0
rtnl_handle_message() at rtnl_handle_message+0x195/frame 0xfffffe008e7a5d30
nl_taskqueue_handler() at nl_taskqueue_handler+0x588/frame 0xfffffe008e7a5e40
taskqueue_run_locked() at taskqueue_run_locked+0x1c2/frame 0xfffffe008e7a5ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xd3/frame 0xfffffe008e7a5ef0
fork_exit() at fork_exit+0x82/frame 0xfffffe008e7a5f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008e7a5f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
[ thread pid 0 tid 105131 ]
Stopped at      kdb_enter+0x33: movq    $0,0x1055fe2(%rip)
db:0:kdb.enter.panic> show pcpu
cpuid        = 1
dynamic pcpu = 0xfffffe008f7c2e00
curthread    = 0xfffff80168127000: pid 0 tid 105131 critnest 1 "netlink_socket (PID"
curpcb       = 0xfffff80168127520
fpcurthread  = none
idlethread   = 0xfffff8010260a000: tid 100004 "idle: cpu1"
self         = 0xffffffff82411000
curpmap      = 0xffffffff81b9be30
tssp         = 0xffffffff82411384
rsp0         = 0xfffffe008e7a6000
kcr3         = 0x800000000238c003
ucr3         = 0xffffffffffffffff
scr3         = 0x164c40c17
gs32p        = 0xffffffff82411404
ldt          = 0xffffffff82411444
tss          = 0xffffffff82411434
curvnet      = 0xfffff800031a6f40
spin locks held:
db:0:kdb.enter.panic>  reset
Uptime: 41m31s
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-25631' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-dtrace-tests.sh
x ./run.sh
x ./disable-notyet-tests.sh
x ./run-kyua.sh
x ./auto-shutdown
x ./disable-zfs-tests.sh
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins2210665535127140970.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'd021d3b3c67573822b01d27cb05ca937fc966843', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../d021d3b3c67573822b01d27cb05ca937fc966843/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address