From nobody Fri Feb 11 13:58:03 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8D35C19B0620; Fri, 11 Feb 2022 13:58:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JwFZ41Zj8z3sdp; Fri, 11 Feb 2022 13:58:04 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644587884; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+dwy0zK/IyxO9TAGFmeSBijMMrQEaYUDhnnp5VVacOI=; b=PFWDQqs2WueSjeETGNjkXNmOPlHyblxQLztDjUwrPJWvYZ5VkoJG83hwkO3ZWc23NH6pI0 K05BeMbjNYl4hzo+qu4sgvPxtlViEW/rNYP8yvd24R6TZ+e6C+5QpIBsb2zHU/bpfLvCMn LXW7iH1x9BiB+I55lWm+UMfpB/gxZunGMZE/KR71IoXAK3Z7WXerOG69ROovfdHq8mFtlN VDmJjfHSp/793CxYVf44g0f/XJ3mQa+Aer0i1zED/DCO6yCNfGBFSdZz7TR9Ud+Kx8F9+g TTmuOua5yjHO4PvAJJIZ6EvrtGuqND4mh+LYvQ2DjkiKyn3o/4xSa45j1D/Njg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DD32A154C7; Fri, 11 Feb 2022 13:58:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 21BDw3m0093131; Fri, 11 Feb 2022 13:58:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 21BDw3KS093130; Fri, 11 Feb 2022 13:58:03 GMT (envelope-from git) Date: Fri, 11 Feb 2022 13:58:03 GMT Message-Id: <202202111358.21BDw3KS093130@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mateusz Guzik Subject: git: 62849eef5b57 - main - fd: split fget_unlocked_seq depending on CAPABILITIES List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 62849eef5b573b9907257f20318f4bd48fcf7b3a Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644587884; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+dwy0zK/IyxO9TAGFmeSBijMMrQEaYUDhnnp5VVacOI=; b=pmVA9DzrEbjGzALD61wZ3tD1+dRojFlVCLXS6YTeVSEs2vEvhiGikl7Q8Ds2zzywUBlU0T v0NV0TOwEY9CuNmPNXjbPTWlOxwiNAqdzdccF2s2qLlE6AIkTl+nfn8wZDus1LXnYzpmIF zysuwKu9kPsVFUEXcuGAygwYthCu70gQ5WoFLGFKiSHqjy+0AJV3TMNQLqUTw0FTH3esfB OtTBjh5IrjIx/KrylGC7krAViBw69IEnPDNb9nzkQ+1hzHZU4cGp0mP661c0cJnuu1BYML x2FtSpmyXezaVb8VPMwSWo2JgiFMQumoXZ5lfJqxwU1jkPjmJU2ArxzVl4t6Sw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644587884; a=rsa-sha256; cv=none; b=oTQo2e012462GnmAfbXfXFCIXV6saXsO1AbED02M2bNd9sy8oe+x8e/vPbD7m2Mji/XCte q+KL5mFEE25rkPdfiv1qEn/ydS67bAVqq3h3iO9ThCfVA1dcnPZJNO4jMQucoLQmRfznJt Cd9JQuz0+EE+UelN+UzhyjNHDjSuLUwfPAw6sX8l3OM1ToqSIB+yfnYQMyZnxUOMUqkm9e lFm78H1cCSMi2zePeqtEFR+hJ5Y4kL0kHNb+uWqnNCedLlxr7XsXfo/s7pGndHOdcX/X/U BcCuxPYh3QHIZt1iV0C5ePALpZv5LOaTpHyVIPp+AB1WltXYemI8TwaGzvoU1g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=62849eef5b573b9907257f20318f4bd48fcf7b3a commit 62849eef5b573b9907257f20318f4bd48fcf7b3a Author: Mateusz Guzik AuthorDate: 2022-02-11 11:54:34 +0000 Commit: Mateusz Guzik CommitDate: 2022-02-11 12:27:22 +0000 fd: split fget_unlocked_seq depending on CAPABILITIES This will simplify an upcoming change. --- sys/kern/kern_descrip.c | 81 ++++++++++++++++++++++++++++++------------------- 1 file changed, 50 insertions(+), 31 deletions(-) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 84622e163503..c4f435002907 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -3027,56 +3027,47 @@ fgetvp_lookup_smr(int fd, struct nameidata *ndp, struct vnode **vpp, bool *fsear } #endif +/* + * Fetch the descriptor locklessly. + * + * We avoid fdrop() races by never raising a refcount above 0. To accomplish + * this we have to use a cmpset loop rather than an atomic_add. The descriptor + * must be re-verified once we acquire a reference to be certain that the + * identity is still correct and we did not lose a race due to preemption. + * + * Force a reload of fdt when looping. Another thread could reallocate + * the table before this fd was closed, so it is possible that there is + * a stale fp pointer in cached version. + */ +#ifdef CAPABILITIES static int fget_unlocked_seq(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp, seqc_t *seqp) { -#ifdef CAPABILITIES const struct filedescent *fde; -#endif const struct fdescenttbl *fdt; struct file *fp; -#ifdef CAPABILITIES seqc_t seq; cap_rights_t haverights; int error; -#endif fdt = fdp->fd_files; if (__predict_false((u_int)fd >= fdt->fdt_nfiles)) return (EBADF); - /* - * Fetch the descriptor locklessly. We avoid fdrop() races by - * never raising a refcount above 0. To accomplish this we have - * to use a cmpset loop rather than an atomic_add. The descriptor - * must be re-verified once we acquire a reference to be certain - * that the identity is still correct and we did not lose a race - * due to preemption. - */ + for (;;) { -#ifdef CAPABILITIES seq = seqc_read_notmodify(fd_seqc(fdt, fd)); fde = &fdt->fdt_ofiles[fd]; haverights = *cap_rights_fde_inline(fde); fp = fde->fde_file; if (!seqc_consistent(fd_seqc(fdt, fd), seq)) continue; -#else - fp = fdt->fdt_ofiles[fd].fde_file; -#endif - if (fp == NULL) + if (__predict_false(fp == NULL)) return (EBADF); -#ifdef CAPABILITIES error = cap_check_inline(&haverights, needrightsp); - if (error != 0) + if (__predict_false(error != 0)) return (error); -#endif if (__predict_false(!refcount_acquire_if_not_zero(&fp->f_count))) { - /* - * Force a reload. Other thread could reallocate the - * table before this fd was closed, so it is possible - * that there is a stale fp pointer in cached version. - */ fdt = atomic_load_ptr(&fdp->fd_files); continue; } @@ -3086,22 +3077,50 @@ fget_unlocked_seq(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, */ atomic_thread_fence_acq(); fdt = fdp->fd_files; -#ifdef CAPABILITIES if (seqc_consistent_nomb(fd_seqc(fdt, fd), seq)) -#else - if (fp == fdt->fdt_ofiles[fd].fde_file) -#endif break; fdrop(fp, curthread); } *fpp = fp; if (seqp != NULL) { -#ifdef CAPABILITIES *seqp = seq; -#endif } return (0); } +#else +static int +fget_unlocked_seq(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, + struct file **fpp, seqc_t *seqp __unused) +{ + const struct fdescenttbl *fdt; + struct file *fp; + + fdt = fdp->fd_files; + if (__predict_false((u_int)fd >= fdt->fdt_nfiles)) + return (EBADF); + + for (;;) { + fp = fdt->fdt_ofiles[fd].fde_file; + if (__predict_false(fp == NULL)) + return (EBADF); + if (__predict_false(!refcount_acquire_if_not_zero(&fp->f_count))) { + fdt = atomic_load_ptr(&fdp->fd_files); + continue; + } + /* + * Use an acquire barrier to force re-reading of fdt so it is + * refreshed for verification. + */ + atomic_thread_fence_acq(); + fdt = fdp->fd_files; + if (__predict_true(fp == fdt->fdt_ofiles[fd].fde_file)) + break; + fdrop(fp, curthread); + } + *fpp = fp; + return (0); +} +#endif /* * See the comments in fget_unlocked_seq for an explanation of how this works.