FYI: Pine64+ 2GB (so A64) booting and non-debug vs. debug kernel: nondebug+INVARIANTS+INVARIANT_SUPPORT sufficient to boot

[Mark Millard]

[Back to nooptions for INVARIANTS and INVARIANT_SUPPORT
but now verbose booting. taskqgroup_adjust_softirq(0)...
is the one to not get a "done." before failure.]

On 2017-Sep-12, at 7:19 PM, Mark Millard <markmi at dsl-only.net> wrote:

> I took my normal GENERIC-NODBG (that includes GENERIC)
> and changed INVARIANTS and INVARIANT_SUPPORT to have
> "options" status instead of "nooptions" status. The
> result boots (so far no counterexamples). (This is
> head -r323246 .)
> 
> So it appears that one or more INVARIANT tests are
> "fixing" the Pine64+ 2GB boot problem. I've no clue
> which. But other debug options are not required.
> 
> FYI. . .
> 
> # more /usr/src/sys/arm64/conf/GENERIC-NODBG                                                                                                                                               
> #
> # GENERIC -- Custom configuration for the arm64/aarch64
> #
> 
> include "GENERIC"
> 
> ident   GENERIC-NODBG
> 
> makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols
> 
> options         ALT_BREAK_TO_DEBUGGER
> 
> options         KDB                     # Enable kernel debugger support
> 
> # For minimum debugger support (stable branch) use:
> #options        KDB_TRACE               # Print a stack trace for a panic
> options         DDB                     # Enable the kernel debugger
> 
> # Extra stuff:
> #options        VERBOSE_SYSINIT         # Enable verbose sysinit messages
> #options        BOOTVERBOSE=1
> #options        BOOTHOWTO=RB_VERBOSE
> #options        KTR
> #options        KTR_MASK=KTR_TRAP
> ##options       KTR_CPUMASK=0xF
> #options        KTR_VERBOSE
> 
> # Disable any extra checking for. . .
> nooptions       DEADLKRES               # Enable the deadlock resolver
> options         INVARIANTS              # Enable calls of extra sanity checking
> options         INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS
> nooptions       WITNESS                 # Enable checks to detect deadlocks and cycles
> nooptions       WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed
> nooptions       DIAGNOSTIC
> nooptions       MALLOC_DEBUG_MAXZONES   # Separate malloc(9) zones
> nooptions       BUF_TRACKING
> nooptions       FULL_BUF_TRACKING

I've changed to have:

options        VERBOSE_SYSINIT         # Enable verbose sysinit messages
options        BOOTVERBOSE=1
options        BOOTHOWTO=RB_VERBOSE

and:

nooptions         INVARIANTS              # Enable calls of extra sanity checking
nooptions         INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS

The tail of the verbose failing boot looks like:

. . .
   vt_upgrade(&vt_consdev)... done.
subsystem b000000
   nfs_rootconf(0)... done.
   fhanew_init(0)... done.
subsystem d000000
   proc0_post(0)... done.
subsystem d800000
   sctp_syscalls_init(0)... done.
   selectinit(0)... done.
subsystem dffff9c
   linker_preload_finish(0)... done.
subsystem e000000
   kick_init(0)... done.
   kstack_cache_init(0)... done.
subsystem e400000
   vm_pageout_init(0)... done.
   $x.1(&page_kp)... done.
subsystem e800000
   $x.1(&vm_kp)... done.
subsystem ea00000
   $x.1(&bufspace_kp)... done.
   $x.1(&buf_kp)... done.
subsystem ec00000
   $x.1(&vnlru_kp)... done.
   $x.1(&up_kp)... done.
subsystem ee00000
   acpi_acad_ac_only(0)... done.
   nfsiod_setup(0)... done.
subsystem f000000
   release_aps(0)... Release APs
APs not started
done.
   tmr_setup_user_access(0)... done.
   intr_irq_shuffle(0)... done.
   tqg_record_smp_started(0)... done.
   netisr_start(0)... done.
   cpuset_init(0)... done.
   taskqgroup_adjust_if_config_tqg(0)... done.
   identify_cpu_sysinit(0)... CPU  0: ARM Cortex-A53 r0p4 affinity:  0
 Instruction Set Attributes 0 = <AES+PMULL,SHA1,SHA2,CRC32>
 Instruction Set Attributes 1 = <0>
         Processor Features 0 = <AdvSIMD,Float,EL3 32,EL2 32,EL1 32,EL0 32>
         Processor Features 1 = <0>
      Memory Model Features 0 = <4k Granule,64k Granule,MixedEndian,S/NS Mem,16bit ASID,1TB PA>
      Memory Model Features 1 = <>
             Debug Features 0 = <2 CTX Breakpoints,4 Watchpoints,6 Breakpoints,PMUv3,Debug v8>
             Debug Features 1 = <0>
         Auxiliary Features 0 = <0>
         Auxiliary Features 1 = <0>
CPU  1: (null) (null) r0p0 affinity:  0
CPU  2: (null) (null) r0p0 affinity:  0
CPU  3: (null) (null) r0p0 affinity:  0
done.
   taskqgroup_adjust_softirq(0)...   x0: ffff000000a1c080
  x1: fffffd0001031a80
  x2:                3
[ thread pid 0 tid 100055 ]
Stopped at      thread_lock_flags_+0x298:       ldr     w4, [x3, #156]
db> 

taskqgroup_adjust_softirq seems to be from:

/usr/src/sys/kern/subr_gtaskqueue.c :

TASKQGROUP_DEFINE(softirq, mp_ncpus, 1);





===
Mark Millard
markmi at dsl-only.net