aarch64 main [so: 15] panic's in kyua's sys/net/if_lagg_test:status_stress
Date: Mon, 11 Sep 2023 08:13:18 UTC
It will be some time before I can try this with
an official snapshot instead of a personal build.
The build is based on b6ce41118bb1 :
# uname -apKU
FreeBSD CA78C-WDK23-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 1500000 #17 main-n265279-b6ce41118bb1-dirty: Sun Sep 10 14:36:47 PDT 2023 root@CA78C-WDK23-ZFS:/usr/obj/BUILDs/main-CA78C-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA78C arm64 aarch64 1500000 1500000
So it was a non-debug build, although I do not
strip symbols and such in my builds.
. . .
sys/net/if_lagg_test:create -> passed [0.105s]
sys/net/if_lagg_test:create_destroy_stress -> skipped: Skipping this test because it easily panics the machine [0.019s]
sys/net/if_lagg_test:lacp_linkstate_destroy_stress -> passed [60.045s]
sys/net/if_lagg_test:set_ether -> passed [0.066s]
sys/net/if_lagg_test:status_stress ->
The core.txt.5 is not great, unfortunately:
panic: vm_fault failed: 0xffff0000006b96dc error 1
GNU gdb (GDB) 13.1 [GDB v13.1 for FreeBSD]
. . .
Reading symbols from /boot/kernel/kernel...
Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...
Unread portion of the kernel message buffer:
(dump_iface + 0x2c0)
elr: 0xffff0000006b96dc (dump_sa + 0x1c)
spsr: 0x0000000000400045
far: 0x44572d4338374144
esr: 0x0000000096000004
panic: vm_fault failed: 0xffff0000006b96dc error 1
cpuid = 2
time = 1694414226
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x30
vpanic() at vpanic+0x1a0
panic() at panic+0x44
data_abort() at data_abort+0x304
handle_el1h_sync() at handle_el1h_sync+0x14
--- exception, esr 0x96000004
dump_sa() at dump_sa+0x1c
dump_iface() at dump_iface+0x2bc
dump_cb() at dump_cb+0x18
if_foreach_sleep() at if_foreach_sleep+0x244
rtnl_handle_getlink() at rtnl_handle_getlink+0xec
rtnl_handle_message() at rtnl_handle_message+0x19c
nl_taskqueue_handler() at nl_taskqueue_handler+0x674
taskqueue_run_locked() at taskqueue_run_locked+0x194
taskqueue_thread_loop() at taskqueue_thread_loop+0xcc
fork_exit() at fork_exit+0x88
fork_trampoline() at fork_trampoline+0x14
KDB: enter: panic
get_curthread () at /usr/main-src/sys/arm64/include/pcpu.h:77
77 __asm __volatile("ldr %0, [x18]" : "=&r"(td));
(kgdb) #0 get_curthread () at /usr/main-src/sys/arm64/include/pcpu.h:77
#1 doadump (textdump=0, textdump@entry=4003518992)
at /usr/main-src/sys/kern/kern_shutdown.c:405
#2 0xffff0000000f7704 in db_dump (dummy=<optimized out>, dummy2=<optimized out>, dummy3=<optimized out>, dummy4=<optimized out>)
at /usr/main-src/sys/ddb/db_command.c:591
#3 0xffff0000000f74e0 in db_command (last_cmdp=<optimized out>, cmd_table=<optimized out>, dopager=true)
at /usr/main-src/sys/ddb/db_command.c:504
#4 0xffff0000000f71b8 in db_command_loop ()
at /usr/main-src/sys/ddb/db_command.c:551
#5 0xffff0000000fad9c in db_trap (type=<optimized out>, code=<optimized out>)
at /usr/main-src/sys/ddb/db_main.c:268
#6 0xffff0000004f4ec4 in kdb_trap (type=60, code=0, tf=<optimized out>)
at /usr/main-src/sys/kern/subr_kdb.c:790
#7 <signal handler called>
#8 <signal handler called>
#9 <signal handler called>
#10 <signal handler called>
#11 <signal handler called>
#12 <signal handler called>
#13 <signal handler called>
#14 <signal handler called>
#15 <signal handler called>
#16 <signal handler called>
#17 <signal handler called>
#18 <signal handler called>
#19 <signal handler called>
#20 <signal handler called>
#21 <signal handler called>
#22 <signal handler called>
Backtrace stopped: Cannot access memory at address 0x10
(kgdb)
So some transcribing of a picture in order to
show register values that were reported:
Fatal data abort:
x0: 0xffff000leea0e7f0 (_DYNAMIC * 0x6d816648)
x1: 0x0000000000000001
x2: 0x44572d4338374143
x3: 0xffff0000005d3f90 (ifdead_ioctl + 0x0)
x4: 0xffffa00b7f0d185e
x5: 0xffffa0023fe4b992
x6: 0x000000006767616c
x7: 0x00706174016f7575
x8: 0x00000000000001a4
x9: 0x0000000000210005
x10: 0×0000000000000800
x11: 0xfefefefefefefeff
x12: 0x0000000000000008
x13: 0x0000000000000000
x14: 0x00000000000000ff
x15: 0x0000000000000700
x16: 0x0000000000000008
x17: 0x0000000000000007
x18: 0xffff0001eea0e500 (_DYNAMIC + 0x6d816358)
x19: 0xffff000leea0e7f0 (_DYNAMIC * 0x6d816648)
x20: 0xffffa00b7f0d1800
x21: 0xffffa00b7f0d1858
x22: 0x000000000000000c
x23: 0X0000000000000005
x24: 0×0000000000000000
x25: 0xffff000000c68000 (sysctl___kern_features_netlink + 0x10)
x26: 0x0000000000000000
x27: 0xffff000000ce9000 (cap_linkat_source_rights + 0x8)
x28: 0xffff0000006bb0a0 (dump_cb + 0x0)
x29: 0xffff0001eea0e520 (_DYNAMIC + 0x6d816378)
sp: 0xffff0001eea0e500
lr: 0xffff0000006b8fe0 (dump_iface + 0x2c0)
elr: 0xffff0000006b96dc (dump_sa + 0x1c)
spsr: 0x0000000000400045
far: 0x44572d4338374144
esr: 0x0000000096000004
panic: m_fault failed: 0xffff0000006b96dc error 1
I expect that this is similar to reports I'd made
back in 14.0-CURRENT days. As I remember, snapshot
builds of the time also got the panic.
I will note that an earlier 14.0-BETA1 snapshot
kernel test run did not panic at this point in the
sequence (or at any point). But I do not know how
repeatable the panics are in the various contexts.
I'll note that I've tried to have the various ports
installed (poudriere built) that are listed at:
https://github.com/freebsd/freebsd-ci/blob/master/scripts/build/build-test_image-head.sh#L69-L84
(The ones that build for aarch64, anyway.)
I had in /etc/kyua/kyua.conf :
test_suites.FreeBSD.disks = '/dev/md0 /dev/md1 /dev/md2 /dev/md3 /dev/md4 /dev/md5'
and used:
# more ~/prekyua-aarch64-mdconfig.sh
#! /bin/sh
truncate -s 4g /var/tmp/for-md0.dat
truncate -s 4g /var/tmp/for-md1.dat
truncate -s 4g /var/tmp/for-md2.dat
truncate -s 4g /var/tmp/for-md3.dat
truncate -s 4g /var/tmp/for-md4.dat
truncate -s 4g /var/tmp/for-md5.dat
mdconfig -f /var/tmp/for-md0.dat -u md0
mdconfig -f /var/tmp/for-md1.dat -u md1
mdconfig -f /var/tmp/for-md2.dat -u md2
mdconfig -f /var/tmp/for-md3.dat -u md3
mdconfig -f /var/tmp/for-md4.dat -u md4
mdconfig -f /var/tmp/for-md5.dat -u md5
I also did a:
# kldload linux64
before doing:
# /usr/bin/kyua test -k /usr/tests/Kyuafile
(Not true of linux64.ko in 14.0-CURRENT days.)
===
Mark Millard
marklmi at yahoo.com