Re: "Khelp module "ertt" can't unload until its refcount drops from 1 to 0." after "All buffers synced."?

> On 19. Nov 2021, at 00:11, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On 2021-Nov-18, at 12:31, tuexen@freebsd.org <tuexen@FreeBSD.org> wrote:
>> 
>>> On 17. Nov 2021, at 21:13, Mark Millard via freebsd-current <freebsd-current@freebsd.org> wrote:
>>> 
>>> I've not noticed the ertt message before in:
>>> 
>>> . . .
>>> Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop... done
>>> All buffers synced.
>>> Uptime: 1d9h57m18s
>>> Khelp module "ertt" can't unload until its refcount drops from 1 to 0.
>> Hi Mark,
>> 
>> what kernel configuration are you using? What kernel modules are loaded?
> 
> The shutdown was of my ZFS boot media but the machine is
> currently doing builds on the UFS media. (The ZFS media is
> present but not mounted). For now I provide information
> from the booted UFS system. The UFS context is intended
> to be nearly a copy of the brctl selection for main [so: 14]
> from the ZFS media. Both systems have been doing the same
> poudriere builds for various comparison/contrast purposes.
> The current build activity will likely take 16+ hrs.
Hi Mark,

thanks a lot for the information. I was contemplating whether this message
was related to a recent change in the TCP congestion module handling, but
since it was already there in February, this is not the case. Will try to
reproduce this, but wasn't able up to now.

Best regards
Michael
> 
> For reference for now (from UFS context):
> 
> # uname -apKU
> FreeBSD HC_CA72_UFS 14.0-CURRENT FreeBSD 14.0-CURRENT #19 main-n250667-20aa359773be-dirty: Sun Nov 14 02:57:32 PST 2021     root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72  arm64 aarch64 1400042 1400042
> 
> This environment was built with -mcpu=cortex-a72 in use and the
> system is a 16 core Cortex-A72 system. (I've caught a FreeBSD
> weak memory model error in the past with such -mcpu=cortext-a72
> use on a Cortext-A72 system, not that I have evidence of such
> here.)
> 
> # kldstat
> Id Refs Address                Size Name
> 1   18 0xffff000000000000  12b1660 kernel
> 2    1 0xffff0000012b2000   41e070 zfs.ko
> 3    1 0xffff0000016d1000    26af8 cryptodev.ko
> 4    1 0xffff00019a400000    27000 nullfs.ko
> 5    1 0xffff00019a427000    25000 fdescfs.ko
> 
> # more /usr/main-src/sys/arm64/conf/GENERIC-NODBG-CA72
> include "GENERIC-NODBG"
> 
> ident   GENERIC-NODBG-CA72
> 
> makeoptions     CONF_CFLAGS="-mcpu=cortex-a72"
> 
> # more /usr/main-src/sys/arm64/conf/GENERIC-NODBG
> #
> # GENERIC -- Custom configuration for the arm64/aarch64
> #
> 
> include "GENERIC"
> 
> ident   GENERIC-NODBG
> 
> makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols
> 
> options         ALT_BREAK_TO_DEBUGGER
> 
> options         KDB                     # Enable kernel debugger support
> 
> # For minimum debugger support (stable branch) use:
> #options        KDB_TRACE               # Print a stack trace for a panic
> options         DDB                     # Enable the kernel debugger
> 
> # Extra stuff:
> #options        VERBOSE_SYSINIT=0       # Enable verbose sysinit messages
> #options        BOOTVERBOSE=1
> #options        BOOTHOWTO=RB_VERBOSE
> #options        KTR
> #options        KTR_MASK=KTR_TRAP
> ##options       KTR_CPUMASK=0xF
> #options        KTR_VERBOSE
> 
> # Disable any extra checking for. . .
> nooptions       DEADLKRES               # Enable the deadlock resolver
> nooptions       INVARIANTS              # Enable calls of extra sanity checking
> nooptions       INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS
> nooptions       WITNESS                 # Enable checks to detect deadlocks and cycles
> nooptions       WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed
> nooptions       DIAGNOSTIC
> nooptions       MALLOC_DEBUG_MAXZONES   # Separate malloc(9) zones
> nooptions       BUF_TRACKING
> nooptions       FULL_BUF_TRACKING
> 
> 
> The builds do include a bulk for targeting armv7
> (so 32-bit compatibility) but that bulk has not
> started yet in the UFS context.
> 
> # poudriere jail -jmain-CA7 -i
> Jail name:         main-CA7
> Jail version:      14.0-CURRENT
> Jail arch:         arm.armv7
> Jail method:       null
> Jail mount:        /usr/obj/DESTDIRs/main-CA7-poud
> Jail fs:           
> Jail updated:      2021-11-16 02:24:57
> Jail pkgbase:      disabled
> 
> My networking context is simple context does not provide
> services outside the local network. The services are normal
> ones, such as nfs, ssh, and whatever default things. ntpd
> is in use. distfiles are downloaded during builds. It is not
> a desktop environment, not even a video card.
> 
> I've noticed a 2021-Feb-22 report in:
> 
> https://forums.freebsd.org/threads/whats-the-error-khelp-module-ertt-unload.79009/
> 
> so getting the message does not appear to be unique to my
> context.
> 
> ===
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>