From nobody Tue Nov 23 09:19:21 2021 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8C7861895CAC; Tue, 23 Nov 2021 09:19:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hyz9h1j3Lz3Dp6; Tue, 23 Nov 2021 09:19:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1AN9JLWc094798 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 23 Nov 2021 11:19:24 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1AN9JLWc094798 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1AN9JLpK094797; Tue, 23 Nov 2021 11:19:21 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 23 Nov 2021 11:19:21 +0200 From: Konstantin Belousov To: Peter Jeremy Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 8ef0c11e7ce7 - main - nfsclient: upgrade vnode lock in VOP_OPEN()/VOP_CLOSE() if we need to flush buffers Message-ID: References: <202111161714.1AGHEtBA084291@gitrepo.freebsd.org> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4Hyz9h1j3Lz3Dp6 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, Nov 23, 2021 at 07:09:08PM +1100, Peter Jeremy wrote: > On 2021-Nov-16 17:14:55 +0000, Konstantin Belousov wrote: > > nfsclient: upgrade vnode lock in VOP_OPEN()/VOP_CLOSE() if we need to flush buffers > > > > VOP_FSYNC() asserts that the vnode is exclusively locked for NFS. > > If we try to execute file with recently modified content, the assert is > > triggered. > > I have a diskless arm64 system configured with swap over NFS and I'm > now consistently getting a panic during shutdown. I haven't > specificially confirmed that it's this commit but the content is > suggestive. > > panic: upgrade of unlocked lock (lockmgr) nfs @ /usr/src/sys/fs/nfsclient/nfs_clvnops.c:855 > cpuid = 3 > time = 1637166551 > KDB: stack backtrace: > db_trace_self() at db_trace_self > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > vpanic() at vpanic+0x178 > panic() at panic+0x44 > witness_upgrade() at witness_upgrade+0x104 > lockmgr_upgrade() at lockmgr_upgrade+0x164 > nfs_lock() at nfs_lock+0x2c > vop_sigdefer() at vop_sigdefer+0x30 > _vn_lock() at _vn_lock+0x54 > nfs_close() at nfs_close+0xc8 > vop_sigdefer() at vop_sigdefer+0x30 > VOP_CLOSE_APV() at VOP_CLOSE_APV+0x2c > swapdev_close() at swapdev_close+0x3c > swapoff_one() at swapoff_one+0x598 > sys_swapoff() at sys_swapoff+0x12c > do_el0_sync() at do_el0_sync+0x498 > handle_el0_sync() at handle_el0_sync+0x90 > --- exception, esr 0x56000000 > > I presume this isn't intended. Can you suggest where I should start > looking for the problem? Try this please. It might be also useful to enable DEBUG_VFS_LOCKS in your kernel config, to catch all related issues once. commit 815eb81d8bf24d62a4686673d83b7bf8ec9f7d90 Author: Konstantin Belousov Date: Tue Nov 23 11:16:53 2021 +0200 swap pager: lock vnode around VOP_CLOSE() Reported by: peterj MFC after: 1 week diff --git a/sys/vm/swap_pager.c b/sys/vm/swap_pager.c index 9bc506c9b6b8..395c0fb72639 100644 --- a/sys/vm/swap_pager.c +++ b/sys/vm/swap_pager.c @@ -3057,9 +3057,9 @@ swapdev_strategy(struct buf *bp, struct swdevt *sp) static void swapdev_close(struct thread *td, struct swdevt *sp) { - + vn_lock(sp->sw_vp, LK_EXCLUSIVE | LK_RETRY); VOP_CLOSE(sp->sw_vp, FREAD | FWRITE, td->td_ucred, td); - vrele(sp->sw_vp); + vput(sp->sw_vp); } static int