From nobody Mon Jan 15 20:29:14 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TDNzG6wHcz57Vtp for ; Mon, 15 Jan 2024 20:29:30 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TDNzG3YJtz4kbl for ; Mon, 15 Jan 2024 20:29:30 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-28e0786edb3so2691212a91.2 for ; Mon, 15 Jan 2024 12:29:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705350568; x=1705955368; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4bBSzZMfiGcUeSr3kTOHu/8/DuhYKJ3HGe7EpiUm7AM=; b=g5Jgkv32lVNhdM3s2ZbteYBQxZ68EKTNDs8pxow5tWccZ2BkwSjK/bvRR8Ro5A7UjX vhcDDC5/IdyqHaZuyckhCcRCiwqJfSUNeQTHFuqaTSNyqr4uwZEgZLYwPr/fHaxAbsVm gIhzOSxd02IU/S2DQxhSLNsNHTT/C3nQA6I5nHG/BB6toDjka7G0VNFZnrzRVsnIHbMp ZT1/kfRpMQULSJzF4vuEspPHvvK/qE/Z04taTFmZ8hJ/2RF3lpF5AmhW/SESsqT5v3b3 ELCq5FfErIb+DO4b8SpRa8BVxu/0+L3tacIoHSmjjok8U+vt0hFI1AlXVeBXmSQLxL96 8IsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705350568; x=1705955368; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4bBSzZMfiGcUeSr3kTOHu/8/DuhYKJ3HGe7EpiUm7AM=; b=HuNHGv2qJLipwbhrh1tC0w38xdrU1zJ76u/WpIibown1upqUrX0RDtE/kaa0Pm5ZT7 vTUWtiN2M/NiWw97h1BtZUiH/UC3v51H76C9JdwIx75uVz8WjCpZhB55mmFZcYCShjk9 1tLUh4v4+63I1UCNBAnh3QB8D5ACVR1xT9PMvBmxd7oEvSzinCa5jc8hAYpa6Lfys11B PX5VUjHEpYT0kMEfP5KoouVoBxIqw3YljucugT0XV6mr+WqlUmfHgZe30YHzPIXMdYOC FUwkIdwHEGbzgjrGe9lXyLOGamxOMxBVwMw4pGxyAiEwUpVHscY0y5KD6YMplJHF++Ru meXw== X-Gm-Message-State: AOJu0Yxnod4RQThYFFztW8dzjHrbRx65uQQ88iYnPh8TNaVt6GME85yL 9X1McFvN4TocmJRtrxCptBm66PAUQN+Grjx05w== X-Google-Smtp-Source: AGHT+IEetDO6bQKoc1JNDZ5G79Je+MIFLmurDZuSDC2PQwDC/AMaydIdZk6aRO2mymq0CBwEpKk/jSLmf5Z5QfehgZw= X-Received: by 2002:a17:90b:1e43:b0:28c:f07a:2986 with SMTP id pi3-20020a17090b1e4300b0028cf07a2986mr3964628pjb.85.1705350567911; Mon, 15 Jan 2024 12:29:27 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20240113193324.3fd54295@thor.intern.walstatt.dynvpn.de> <1369645989.13766.1705178331205@localhost> <20240115043412.B6998C8@slippy.cwsent.com> <20240115064704.611fe0c4@thor.intern.walstatt.dynvpn.de> <683EF50F-6665-4664-A7CE-1EFE50076FB0@bsd4all.org> <20240115173611.2b8e76d6@thor.intern.walstatt.dynvpn.de> In-Reply-To: <20240115173611.2b8e76d6@thor.intern.walstatt.dynvpn.de> From: Rick Macklem Date: Mon, 15 Jan 2024 12:29:14 -0800 Message-ID: Subject: Re: NFSv4 crash of CURRENT To: FreeBSD User Cc: Peter Blok , Cy Schubert , Ronald Klop , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4TDNzG3YJtz4kbl X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On Mon, Jan 15, 2024 at 11:03=E2=80=AFAM FreeBSD User wrote: > > Am Mon, 15 Jan 2024 11:53:31 +0100 > Peter Blok schrieb: > > > Hi, > > > > Forgot to mention I=E2=80=99m on 13-stable. The fix that is causing the= crash with automounted NFS > > is: > > > > commit cc5cda1dbaa907ce52074f47264cc45b5a7d6c8b > > Author: Konstantin Belousov > > Date: Tue Jan 2 00:22:44 2024 +0200 > > > > nfsclient: limit situations when we do unlocked read-ahead by nfsio= d > > > > (cherry picked from commit 70dc6b2ce314a0f32755005ad02802fca7ed186e= ) > > > > When I remove the fix, the problem is gone. Add it back and the crash h= appens. > > > > Peter > > > > > On 15 Jan 2024, at 09:31, Peter Blok wrote: > > > > > > Hi, > > > > > > I do have a crash on a NFS client with stable of today > > > (4c4633fdffbe8e4b6d328c2bc9bb3edacc9ab50a). It is also autofs related= . Maybe it is the > > > same problem. > > > > > > I have ports automounted on /am/ports. When I do cd /am/ports/sys and= type tab to > > > autocomplete it crashes with the below stack trace. If I plainly moun= t ports on /usr/ports > > > and do the same everything works. I am using NFSv3 > > > > > > Peter > > > > > > > > > > > > > > > Fatal trap 12: page fault while in kernel mode > > > cpuid =3D 2; apic id =3D 04 > > > fault virtual address =3D 0x89 > > > fault code =3D supervisor read data, page not present > > > instruction pointer =3D 0x20:0xffffffff809645d4 > > > stack pointer =3D 0x28:0xfffffe00acadb830 > > > frame pointer =3D 0x28:0xfffffe00acadb830 > > > code segment =3D base 0x0, limit 0xfffff, type 0x1b > > > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > > > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > > > current process =3D 6869 (csh) > > > trap number =3D 12 > > > panic: page fault > > > cpuid =3D 2 > > > time =3D 1705306940 > > > KDB: stack backtrace: > > > #0 0xffffffff806232f5 at kdb_backtrace+0x65 > > > #1 0xffffffff805d7a02 at vpanic+0x152 > > > #2 0xffffffff805d78a3 at panic+0x43 > > > #3 0xffffffff809d58ad at trap_fatal+0x38d > > > #4 0xffffffff809d58ff at trap_pfault+0x4f > > > #5 0xffffffff809af048 at calltrap+0x8 > > > #6 0xffffffff804c7a7e at ncl_bioread+0xb7e > > > #7 0xffffffff804b9d90 at nfs_readdir+0x1f0 > > > #8 0xffffffff8069c61a at vop_sigdefer+0x2a > > > #9 0xffffffff809f8ae0 at VOP_READDIR_APV+0x20 > > > #10 0xffffffff81ce75de at autofs_readdir+0x2ce > > > #11 0xffffffff809f8ae0 at VOP_READDIR_APV+0x20 > > > #12 0xffffffff806c3002 at kern_getdirentries+0x222 > > > #13 0xffffffff806c33a9 at sys_getdirentries+0x29 > > > #14 0xffffffff809d6180 at amd64_syscall+0x110 > > > #15 0xffffffff809af95b at fast_syscall_common+0xf8 > > > > > > > > > > > >> On 15 Jan 2024, at 06:46, FreeBSD User > >> > wrote: > > >> > > >> Am Sun, 14 Jan 2024 20:34:12 -0800 > > >> Cy Schubert > schrieb: > > >> > > >>> In message > >>> > > >>> om> > > >>> , Rick Macklem writes: > > >>>> On Sat, Jan 13, 2024 at 12:39=3DE2=3D80=3DAFPM Ronald Klop > >>>> >=3D wrote: > > >>>>> > > >>>>> > > >>>>> Van: FreeBSD User > > > >>>>> Datum: 13 januari 2024 19:34 > > >>>>> Aan: FreeBSD CURRENT > > > >>>>> Onderwerp: NFSv4 crash of CURRENT > > >>>>> > > >>>>> Hello, > > >>>>> > > >>>>> running CURRENT client (FreeBSD 15.0-CURRENT #4 main-n267556-6974= 8e62e82a=3D > > >>>> : Sat Jan 13 18:08:32 > > >>>>> CET 2024 amd64). One NFSv4 server is same OS revision as the ment= ioned cl=3D > > >>>> ient, other is FreeBSD > > >>>>> 13.2-RELEASE-p8. Both offer NFSv4 filesystems, non-kerberized. > > >>>>> > > >>>>> I can crash the client reproducable by accessing the one or other= NFSv4 F=3D > > >>>> S (a simple ls -la). > > >>>>> The NFSv4 FS is backed by ZFS (if this matters). I do not have ph= ysicla a=3D > > >>>> ccess to the client > > >>>>> host, luckily the box recovers. > > >>>> Did you rebuild both the nfscommon and nfscl modules from the same= sources? > > >>>> I did a commit to main that changes the interface between these tw= o > > >>>> modules and did bump the > > >>>> __FreeBSD_version to 1500010, which should cause both to be rebuil= t. > > >>>> (If you have "options NFSCL" in your kernel config, both should ha= ve > > >>>> been rebuilt as a part of > > >>>> the kernel build.) > > >>>> > > >>> > > >>> Is anyone by chance seeing autofs in the backtrace too? > > >>> > > >>> > > >> > > >> Hello Cy Shubert, > > >> > > >> I forgot to mention that those crashes occur with autofs mounted fil= esystems. Good > > >> question, by the way, I will check whether crashes also happen when = mounting the > > >> tradidional way. > > >> > > >> Kind regards, > > >> > > >> oh > > >> > > >> -- > > >> O. Hartmann > > > > > > > good catch! Don't thank me, thank Kostik. He's already committed the patch. Btw, I didn't look at fixing this because I knew Kostik would fix it before I had it figured out;-) Thanks everyone for reporting it, rick > > -- > O. Hartmann