From nobody Fri Jul 12 11:36:09 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WL8gP0ljxz5QCvl for ; Fri, 12 Jul 2024 11:36:17 +0000 (UTC) (envelope-from SRS0=8UWh=OM=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WL8gN58XDz4rYQ for ; Fri, 12 Jul 2024 11:36:16 +0000 (UTC) (envelope-from SRS0=8UWh=OM=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 12 Jul 2024 13:36:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1720784169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WHZEM55BlARUhsX/MiCmx3W4iQcdkpAy1JoSHMTsheQ=; b=A+6oNRPGie3+dXWUOwtrM3r9wbpZra0jI+sn5yLRGkTJ/+bC65F9sBiGJzmiIJZvd4QhU5 FuT8fnTpfRGEbAvQYBinREojgrAWOByrmUBlByK05+P732HKQJMNr6Hx+i8AkaGo3P3/Qm kb17qWbl1nfIzlsMj7yaxx3pWDS+KtFP9cGBfm/aC4E/LwFR9HQNj5dG6GT52Ip81DPzt9 xo3Jj5DpWXp+ZvB4+qaOi9RaezYvB36b4KSZbF1xZ7Vsy1cZ6iTTjdC67mNMtr3U3pwgl/ gsqexD/fs0d7qm2S8nTUYxDCRh21p7FHdn+OHVDvsBMn/eb4Ba17lgD2suv/lg== From: Ronald Klop To: Konstantin Belousov Cc: FreeBSD Current Message-ID: <298825087.4944.1720784169535@localhost> In-Reply-To: References: <5bb1505d-65c5-43cb-878c-38ec02271a50@app.fastmail.com> Subject: Re: exited on signal 11 (no core dump - other error) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4943_285023450.1720784169415" X-Mailer: Realworks (710.8) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4WL8gN58XDz4rYQ ------=_Part_4943_285023450.1720784169415 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Konstantin Belousov Datum: vrijdag, 12 juli 2024 13:15 Aan: FreeBSD Current Onderwerp: Re: exited on signal 11 (no core dump - other error) > > On Fri, Jul 12, 2024 at 10:45:31AM +0000, Dave Cottlehuber wrote: > > On Fri, 12 Jul 2024, at 03:39, Zhenlei Huang wrote: > > > Hi > > > > > > I observed something weird on Release 14.1. > > > > > > When rebooting my dev machine, I got > > > ... > > > IIUC all processes will get signal to quit on system reboot. But what does the > > > signal 11 mean ? Is it EDEADLK in sys/sys/errno.h ? > > > > > > If yes, then why they get dead locked ? > > > > I see the same on 15.0-CURRENT too here. In my case this is just after syslog-ng is stopped. > > > > <6>[1920] pid 6090 (wezterm-gui), jid 0, uid 1002: exited on signal 11 (no core dump - other error) > > <6>[1920] pid 6039 (polkitd), jid 0, uid 565: exited on signal 11 (no core dump - bad address) > > <6>[1920] pid 4306 (dbus-daemon), jid 0, uid 556: exited on signal 11 (no core dump - bad address) > > Most natural cause for SIGSEGV during shutdown is because root is unmounted > while the processes are still handling signals (SIGTERM) from init. The > text vnodes for the process binary and shared libraries are force-reclaimed, > and any page-in request results in the unhandled fault. > > I regularly see these SIGSEGVs on nfs-booted crash boxes. > > > > I can also easily reproduce this on my RPI4/15-CURRENT using 2 ZFS disks via USB. Just did a shutdown -r now to check and appended the serial output here. FreeBSD/arm64 (rpi4) (ttyu0) login: Jul 7 23:47:46 rpi4 shuStopping jails: jail14 jail13 jenkins monitoring loghost. Stopping node_exporter. Stopping sshd. Waiting for PIDS: 1910. Stopping cron. Waiting for PIDS: 1863. Stopping powerd. Waiting for PIDS: 1832. Stopping rtsold. Waiting for PIDS: 1484. Stopping devd. Waiting for PIDS: 1475. Writing RTC file: /var/db/fakertc. Writing entropy file: . Writing early boot entropy file: . . Terminated Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 0 0 done All buffers synced. pid 23288 (sshd), jid 0, uid 1001: exited on signal 4 (no core dump - bad address) pid 23329 (bash), uid (0): Path `/var/tmp/0.bash.0.23329.core' failed on initial open test, error = 2 pid 23286 (sshd), jid 0, uid 0: exited on signal 4 (no core dump - bad address) pid 23329 (bash), jid 0, uid 0: exited on signal 4 (no core dump - other error) pid 23328 (su), jid 0, uid 0: exited on signal 4 (no core dump - bad address) pid 23289 (bash), uid (1001): Path `/var/tmp/1001.bash.0.23289.core' failed on initial open test, error = 2 pid 23289 (bash), jid 0, uid 1001: exited on signal 4 (no core dump - other error) Uptime: 23h48m34s Resetting system ... pid 1769 (syslogd), uid (0): Path `/var/tmp/0.syslogd.0.1769.core' failed on initial open test, error = 2 pid 1769 (syslogd), jid 0, uid 0: exited on signal 4 (no core dump - other error) To me it looks like the sshd process in which I typed 'shutdown -r now' is still available somehow. Regards, Ronald. ------=_Part_4943_285023450.1720784169415 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Van: Konstantin Belousov <kostikbel@gmail.com> Datum: vrijdag, 12 juli 2024 13:15
Aan: FreeBSD Current <current@freebsd.org>
Onderwerp: Re: exited on signal 11 (no core dump - other e= rror)

On Fri, Jul 12, 2024 at 10:45:31A= M +0000, Dave Cottlehuber wrote:
> On Fri, 12 Jul 2024, at 03:39, Zhenlei Huang wrote:
> > Hi
> >
> > I observed something weird on Release 14.1.
> >
> > When rebooting my dev machine, I got
> > ...
> > IIUC all processes will get signal to quit on system reboot. But = what does the
> > signal 11 mean ? Is it EDEADLK in sys/sys/errno.h ?
> >
> > If yes, then why they get dead locked ?
>
> I see the same on 15.0-CURRENT too here. In my case this is just after= syslog-ng is stopped.
>
> <6>[1920] pid 6090 (wezterm-gui), jid 0, uid 1002: exited on sig= nal 11 (no core dump - other error)
> <6>[1920] pid 6039 (polkitd), jid 0, uid 565: exited on signal 1= 1 (no core dump - bad address)
> <6>[1920] pid 4306 (dbus-daemon), jid 0, uid 556: exited on sign= al 11 (no core dump - bad address)

Most natural cause for SIGSEGV during shutdown is because root is unmounted=
while the processes are still handling signals (SIGTERM) from init.  T= he
text vnodes for the process binary and shared libraries are force-reclaimed= ,
and any page-in request results in the unhandled fault.

I regularly see these SIGSEGVs on nfs-booted crash boxes.
 


I can also easily reproduce this on my RPI4/15-CURRENT using 2 ZFS disks vi= a USB.
Just did a shutdown -r now to check and appended the serial output here.
FreeBSD/arm64 (rpi4) (ttyu0)        = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;     
            &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;            =     
login: Jul  7 23:47:46 rpi4 shuStopping jails: jail14 jail13 jenkins m= onitoring loghost.         &nb= sp;            =             &nb= sp;            =             &nb= sp;            =             &nb= sp;      
Stopping node_exporter.        &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;         
Stopping sshd.          &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;    
Waiting for PIDS: 1910.        &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;         
Stopping cron.          &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;    
Waiting for PIDS: 1863.        &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;         
Stopping powerd.          = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;   
Waiting for PIDS: 1832.
Stopping rtsold.
Waiting for PIDS: 1484.
Stopping devd.
Waiting for PIDS: 1475.
Writing RTC file: /var/db/fakertc.
Writing entropy file: .
Writing early boot entropy file: .
.
Terminated
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 done
All buffers synced.
pid 23288 (sshd), jid 0, uid 1001: exited on signal 4 (no core dump - bad a= ddress)
pid 23329 (bash), uid (0):  Path `/var/tmp/0.bash.0.23329.core' failed= on initial open test, error =3D 2
pid 23286 (sshd), jid 0, uid 0: exited on signal 4 (no core dump - bad addr= ess)
pid 23329 (bash), jid 0, uid 0: exited on signal 4 (no core dump - other er= ror)
pid 23328 (su), jid 0, uid 0: exited on signal 4 (no core dump - bad addres= s)
pid 23289 (bash), uid (1001):  Path `/var/tmp/1001.bash.0.23289.core' = failed on initial open test, error =3D 2
pid 23289 (bash), jid 0, uid 1001: exited on signal 4 (no core dump - other= error)
Uptime: 23h48m34s
Resetting system ... pid 1769 (syslogd), uid (0):  Path `/var/tmp/0.sy= slogd.0.1769.core' failed on initial open test, error =3D 2
pid 1769 (syslogd), jid 0, uid 0: exited on signal 4 (no core dump - other = error)

To me it looks like the sshd process in which I typed 'shutdown -r now' is = still available somehow.

Regards,
Ronald.
  ------=_Part_4943_285023450.1720784169415--