From nobody Mon Jul 22 01:39:23 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WS2y36kbfz5R14x; Mon, 22 Jul 2024 01:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WS2y367ksz4nJJ; Mon, 22 Jul 2024 01:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721612363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=usNLBVnzzkwrp6LjcPhyzC/Ty4DmIs2CeO58HcFhK4k=; b=jgV4rNYjsw9iHG5NjEnVGxSkfbWfbKXPlOqXszq+VUe7xz/9u8klfFYIcljd05QKauMGxM aQGiz9VDIYP+Ug/EmyQaYRJgR00nnpnduoevn4CnPpGLIPz3vLsRtZduFShBnduJYMKszm FN9JRxRCGEY5o+qCVMHqHfvq8kUstdggFFkNTi0T1r/EV5mYSbu2dzELOTZRh+5lbtb3ty NgkOx6vW3i+hibsYHD/5sO0nQOekoKMz2UbGxTaaBGtgGDPsqWpcTr4oQWWKsU/+WLCLlr Jro5lRi208eLgLrmd2QrYdyv1OBlt+ypOPwm02CrTR/zVwMUDorwKbYtI8eh+A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721612363; a=rsa-sha256; cv=none; b=xtmCv9sKNwwIf2ELLLUC+ZRl5efOnwEbN5GtVV54hRHn03hPiFJwPYAlWM6ayfbadHyhCW hjywsqdqZIkgTWqqaGeylGVABhoWz9jzzVDrlENBqsAsRCV+gRIvRTx77yKX2Ho5p7GSuE WRyHIwGvvUtSEcwKdJ/I66+hEYQmBARV0YzgiUy6H+i1gzKwIcnly3JeKLvR3b86cLvnXy NKWY2yRHjPvNr3OEqA/DAqTbIc0RxreG6f0jG6CIEHY6VoKsGajREwpWehgtiZYTMwsh29 qxXLdIHreoNd7B7MTq/lSG42wDrrqgOaH0sbctdsr88Yp96nqEVKQ/SEdvEk7Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721612363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=usNLBVnzzkwrp6LjcPhyzC/Ty4DmIs2CeO58HcFhK4k=; b=EjuOOPraiz/3XcWmPXhKw5zJLNzxGN9OfeNjT1L2ZAleBaw0VUEe1NEP6XaWe1MlX90UQ9 MNmP1qhnAaG6uv5PUO6yzNZLD+XGL7SCpdgZnlfPUYAFq+oKvQHc60p3OoDiMB6VRgZDWF CEDrhd4aHnfCOpXuPADUjiVz15HOLlPt9fDJJcMqjwt8SQqG35ExvaHrce17Khsa1NXOxi 38o0uQHAv/XkSrvHclvnhOebnj+PFT6OMD0mkSYSfvj7i0WH0nu3iU1BC1AZUZ7ocyJ1Kp hFDr3tLUmxITwygLq9w5fhD4rwVqYECzzzp2pizrtPqKaqudxeG8SJKMiTIGyw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WS2y35g6rzP2X; Mon, 22 Jul 2024 01:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46M1dNoP075773; Mon, 22 Jul 2024 01:39:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46M1dNas075770; Mon, 22 Jul 2024 01:39:23 GMT (envelope-from git) Date: Mon, 22 Jul 2024 01:39:23 GMT Message-Id: <202407220139.46M1dNas075770@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: c664b786ccd1 - stable/13 - nfsd: Fix nfsrv_cleanclient so that it can be called with a mutex List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: c664b786ccd18bd186c59279e26fd19c6a212be4 Auto-Submitted: auto-generated The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=c664b786ccd18bd186c59279e26fd19c6a212be4 commit c664b786ccd18bd186c59279e26fd19c6a212be4 Author: Rick Macklem AuthorDate: 2024-06-21 22:08:48 +0000 Commit: Rick Macklem CommitDate: 2024-07-21 23:30:11 +0000 nfsd: Fix nfsrv_cleanclient so that it can be called with a mutex On Feb. 28, a problem was reported on freebsd-stable@ where a nfsd thread processing an ExchangeID operation was blocked for a long time by another nfsd thread performing a copy_file_range. This occurred because the copy_file_range was taking a long time, but also because handling a clientID requires that all other nfsd threads be blocked via an exclusive lock, as required by ExchangeID. This patch adds two arguments to nfsv4_cleanclient() so that it can optionally be called with a mutex held. For this patch, the first of these arguments is "false" and, as such, there is no change in semantics. However, this change will allow a future commit to modify handling of the clientID so that it can be done with a mutex held while other nfsd threads continue to process NFS RPCs. (cherry picked from commit a7de51068502ad1e2851d4a855ed28b27573bb36) --- sys/fs/nfs/nfsrvstate.h | 2 +- sys/fs/nfsserver/nfs_nfsdsocket.c | 2 +- sys/fs/nfsserver/nfs_nfsdstate.c | 53 +++++++++++++++++++++++---------------- 3 files changed, 34 insertions(+), 23 deletions(-) diff --git a/sys/fs/nfs/nfsrvstate.h b/sys/fs/nfs/nfsrvstate.h index 3652e2c2db2f..c196f43aac04 100644 --- a/sys/fs/nfs/nfsrvstate.h +++ b/sys/fs/nfs/nfsrvstate.h @@ -331,7 +331,7 @@ struct nfsf_rec { u_int32_t numboots; /* Number of boottimes */ }; -void nfsrv_cleanclient(struct nfsclient *, NFSPROC_T *); +void nfsrv_cleanclient(struct nfsclient *, NFSPROC_T *, bool, SVCXPRT **); void nfsrv_freedeleglist(struct nfsstatehead *); /* diff --git a/sys/fs/nfsserver/nfs_nfsdsocket.c b/sys/fs/nfsserver/nfs_nfsdsocket.c index da92b7b0047c..94369c575e52 100644 --- a/sys/fs/nfsserver/nfs_nfsdsocket.c +++ b/sys/fs/nfsserver/nfs_nfsdsocket.c @@ -792,7 +792,7 @@ nfsrvd_compound(struct nfsrv_descript *nd, int isdgram, u_char *tag, !LIST_EMPTY(&clp->lc_deleg)) nfsrv_writestable(clp->lc_id, clp->lc_idlen, NFSNST_REVOKE, p); - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); LIST_REMOVE(clp, lc_hash); diff --git a/sys/fs/nfsserver/nfs_nfsdstate.c b/sys/fs/nfsserver/nfs_nfsdstate.c index 06d0b79f10d5..911f5053bd3c 100644 --- a/sys/fs/nfsserver/nfs_nfsdstate.c +++ b/sys/fs/nfsserver/nfs_nfsdstate.c @@ -203,7 +203,8 @@ static void nfsrv_locallock_commit(struct nfslockfile *lfp, int flags, static void nfsrv_locklf(struct nfslockfile *lfp); static void nfsrv_unlocklf(struct nfslockfile *lfp); static struct nfsdsession *nfsrv_findsession(uint8_t *sessionid); -static int nfsrv_freesession(struct nfsdsession *sep, uint8_t *sessionid); +static int nfsrv_freesession(struct nfsdsession *sep, uint8_t *sessionid, + bool locked, SVCXPRT **old_xprtp); static int nfsv4_setcbsequence(struct nfsrv_descript *nd, struct nfsclient *clp, int dont_replycache, struct nfsdsession **sepp, int *slotposp); static int nfsv4_getcbsession(struct nfsclient *clp, struct nfsdsession **sepp); @@ -329,7 +330,7 @@ nfsrv_setclient(struct nfsrv_descript *nd, struct nfsclient **new_clpp, */ if (i != nfsrv_clienthashsize) { LIST_REMOVE(clp, lc_hash); - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); zapit = 1; @@ -382,7 +383,7 @@ nfsrv_setclient(struct nfsrv_descript *nd, struct nfsclient **new_clpp, */ if (clp->lc_expiry < NFSD_MONOSEC && (!LIST_EMPTY(&clp->lc_open) || !LIST_EMPTY(&clp->lc_deleg))) { - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); } @@ -445,7 +446,7 @@ nfsrv_setclient(struct nfsrv_descript *nd, struct nfsclient **new_clpp, /* Get rid of all sessions on this clientid. */ LIST_FOREACH_SAFE(sep, &clp->lc_session, sess_list, nsep) { - ret = nfsrv_freesession(sep, NULL); + ret = nfsrv_freesession(sep, NULL, false, NULL); if (ret != 0) printf("nfsrv_setclient: verifier changed free" " session failed=%d\n", ret); @@ -715,7 +716,7 @@ nfsrv_getclient(nfsquad_t clientid, int opflags, struct nfsclient **clpp, * for an Open with CLAIM_DELEGATE_PREV unless in * grace, but get rid of the rest of the state. */ - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_olddeleg); if (nfsrv_checkgrace(nd, clp, 0)) { /* In grace, so just delete delegations */ @@ -874,7 +875,7 @@ nfsrv_destroyclient(nfsquad_t clientid, NFSPROC_T *p) } /* Destroy the clientid and return ok. */ - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); LIST_REMOVE(clp, lc_hash); @@ -943,7 +944,7 @@ nfsrv_adminrevoke(struct nfsd_clid *revokep, NFSPROC_T *p) */ clp->lc_flags &= ~LCL_CALLBACKSON; clp->lc_flags |= LCL_ADMINREVOKED; - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); NFSLOCKV4ROOTMUTEX(); @@ -1363,7 +1364,8 @@ nfsrv_servertimer(void *arg __unused) * there are no other active nfsd threads. */ void -nfsrv_cleanclient(struct nfsclient *clp, NFSPROC_T *p) +nfsrv_cleanclient(struct nfsclient *clp, NFSPROC_T *p, bool locked, + SVCXPRT **old_xprtp) { struct nfsstate *stp, *nstp; struct nfsdsession *sep, *nsep; @@ -1372,7 +1374,7 @@ nfsrv_cleanclient(struct nfsclient *clp, NFSPROC_T *p) nfsrv_freeopenowner(stp, 1, p); if ((clp->lc_flags & LCL_ADMINREVOKED) == 0) LIST_FOREACH_SAFE(sep, &clp->lc_session, sess_list, nsep) - (void)nfsrv_freesession(sep, NULL); + (void)nfsrv_freesession(sep, NULL, locked, old_xprtp); } /* @@ -4603,7 +4605,7 @@ nfsrv_docallback(struct nfsclient *clp, int procnum, nfsv4stateid_t *stateidp, if (procnum != NFSV4PROC_CBNULL) nfsv4_freeslot(&sep->sess_cbsess, slotpos, true); - nfsrv_freesession(sep, NULL); + nfsrv_freesession(sep, NULL, false, NULL); } else if (nd->nd_procnum == NFSV4PROC_CBNULL) error = newnfs_connect(NULL, &clp->lc_req, cred, NULL, 1, dotls, &clp->lc_req.nr_client); @@ -4652,7 +4654,7 @@ nfsrv_docallback(struct nfsclient *clp, int procnum, nfsv4stateid_t *stateidp, nfsv4_freeslot(&sep->sess_cbsess, slotpos, true); } - nfsrv_freesession(sep, NULL); + nfsrv_freesession(sep, NULL, false, NULL); } else error = newnfs_request(nd, NULL, clp, &clp->lc_req, NULL, NULL, cred, clp->lc_program, @@ -5250,7 +5252,7 @@ nfsrv_clientconflict(struct nfsclient *clp, int *haslockp, vnode_t vp, */ nfsrv_writestable(clp->lc_id, clp->lc_idlen, NFSNST_REVOKE, p); nfsrv_backupstable(); - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); LIST_REMOVE(clp, lc_hash); @@ -5442,7 +5444,7 @@ nfsrv_delegconflict(struct nfsstate *stp, int *haslockp, NFSPROC_T *p, nfsrv_writestable(clp->lc_id, clp->lc_idlen, NFSNST_REVOKE, p); nfsrv_backupstable(); if (clp->lc_expiry < NFSD_MONOSEC) { - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); LIST_REMOVE(clp, lc_hash); @@ -6243,7 +6245,7 @@ nfsrv_throwawayallstate(NFSPROC_T *p) for (i = 0; i < nfsrv_clienthashsize; i++) { LIST_FOREACH_SAFE(clp, &NFSD_VNET(nfsclienthash)[i], lc_hash, nclp) { - nfsrv_cleanclient(clp, p); + nfsrv_cleanclient(clp, p, false, NULL); nfsrv_freedeleglist(&clp->lc_deleg); nfsrv_freedeleglist(&clp->lc_olddeleg); free(clp->lc_stateid, M_NFSDCLIENT); @@ -6456,7 +6458,7 @@ nfsrv_destroysession(struct nfsrv_descript *nd, uint8_t *sessionid) } while (igotlock == 0); NFSUNLOCKV4ROOTMUTEX(); - error = nfsrv_freesession(NULL, sessionid); + error = nfsrv_freesession(NULL, sessionid, false, NULL); if (error == 0 && samesess != 0) nd->nd_flag &= ~ND_HASSEQUENCE; @@ -6547,12 +6549,14 @@ nfsrv_bindconnsess(struct nfsrv_descript *nd, uint8_t *sessionid, int *foreaftp) * Free up a session structure. */ static int -nfsrv_freesession(struct nfsdsession *sep, uint8_t *sessionid) +nfsrv_freesession(struct nfsdsession *sep, uint8_t *sessionid, + bool locked, SVCXPRT **old_xprtp) { struct nfssessionhash *shp; int i; - NFSLOCKSTATE(); + if (!locked) + NFSLOCKSTATE(); if (sep == NULL) { shp = NFSSESSIONHASH(sessionid); NFSLOCKSESSION(shp); @@ -6565,21 +6569,28 @@ nfsrv_freesession(struct nfsdsession *sep, uint8_t *sessionid) sep->sess_refcnt--; if (sep->sess_refcnt > 0) { NFSUNLOCKSESSION(shp); - NFSUNLOCKSTATE(); + if (!locked) + NFSUNLOCKSTATE(); return (NFSERR_BACKCHANBUSY); } LIST_REMOVE(sep, sess_hash); LIST_REMOVE(sep, sess_list); } NFSUNLOCKSESSION(shp); - NFSUNLOCKSTATE(); + if (!locked) + NFSUNLOCKSTATE(); if (sep == NULL) return (NFSERR_BADSESSION); for (i = 0; i < NFSV4_SLOTS; i++) if (sep->sess_slots[i].nfssl_reply != NULL) m_freem(sep->sess_slots[i].nfssl_reply); - if (sep->sess_cbsess.nfsess_xprt != NULL) - SVC_RELEASE(sep->sess_cbsess.nfsess_xprt); + if (!locked) { + if (sep->sess_cbsess.nfsess_xprt != NULL) + SVC_RELEASE(sep->sess_cbsess.nfsess_xprt); + if (old_xprtp != NULL) + *old_xprtp = NULL; + } else if (old_xprtp != NULL) + *old_xprtp = sep->sess_cbsess.nfsess_xprt; free(sep, M_NFSDSESSION); return (0); }