From nobody Wed Aug 21 15:21:41 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wpqn11gztz5V2tl; Wed, 21 Aug 2024 15:21:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wpqn116s2z4P47; Wed, 21 Aug 2024 15:21:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724253701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vJMqpqIK2LIotT86yvX1Nl8ux1i7iK6UXV5HAJb26dM=; b=VOoP1d0n1OnYkNoho9GfkVv7NIaFfIlW2xkov6oOK+YUMRjThxeaXR+rT6biO/CuRHP4zX d+88r83TCfccy2qUQ/b1qQq+S/eQH85Gpcb652na3xL56/KIBQDYSDT59sZCzdBPnyWTsN AB2+jE2hwKvELhpRqRAB3ndJ4UwkDSNpf8gqEGqoxK1T0yhfUSSZkN4Ep0DsDQsab/8y1z xfmcSuSWRAfTVeF8i7lEjzXH3N7XFUxxV/y6CCGfFT9jJ8eqDdSY3gjA+eHDHabw1BtbMA jM8Bmtw66wXx1KSSxdCjN3vF2RV53/9syNyC33IFRzmUUskp4ohTyEnKTJV4zA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724253701; a=rsa-sha256; cv=none; b=U28oYfEtXeXeR8xtPr4/yCKfcbZKspC3s8aHMdkkRh4q0lUiEtW8jjjc/DBTnKypOztwTB dQF1joL+zzaxZjgJRPHs/tTRsjeQwbhBx5VHT4wozDbLli16jtuQFuEc8QRpgJia23kHn7 tMSFEQAdQayJwxKpzNHj+fYSNyvV7k3otMO3XtbfUNwQIbdfsCkuNZlhgrzbJSaFxs/nkT yEGsby3KX22oP1Vx7wHJDrmPHguJQVdA3EwU8LWV5iiEkCLUuhxz97Y8YYy0OAqUgznGyz Vcg2XTGgYV6bx0Ype8bE9Anbwcw/G0vdDfr8Beacm68yTSpPfCLIatkJXoL0nw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724253701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vJMqpqIK2LIotT86yvX1Nl8ux1i7iK6UXV5HAJb26dM=; b=AUqt0B7IN9RaW4ApstDxeRqM9jVICqx76enqHyWLLQIccx0i54r/2PZWqnuEgsemlYmqV9 RmGoxGCCglYkviPkoF9T8+rnhnaLsIxnsXQWUE1RAPyBubByY95Xc9OakjNvK691O6Vw0X 3m9KZMd7jwgz0DpzELCULVAASyFyGsZX+PPp/WH4hFOM1m3Zd3rpcWvHFEXiVnW5UgtZYW BrG1lBBSdXG2yd3K70esdOU/M1PJyAl8VJKcGyGn82c1Z10wpmDiXQoSDP6P/mijuAY8jh 1sA2nK3DaYIQnpAfZxS/ebcvkmWcZ1fUWeVFgdCHwAW+M763jYHdtbDQmUUGxA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wpqn10kShzq6L; Wed, 21 Aug 2024 15:21:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47LFLfod067139; Wed, 21 Aug 2024 15:21:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47LFLfNY067136; Wed, 21 Aug 2024 15:21:41 GMT (envelope-from git) Date: Wed, 21 Aug 2024 15:21:41 GMT Message-Id: <202408211521.47LFLfNY067136@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 8a5b2db3d81d - main - ranglelock_destroy(): do not remove lock entries from under live lock acquirer List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 8a5b2db3d81db16e9e6aaea82cc071bdc766e360 Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=8a5b2db3d81db16e9e6aaea82cc071bdc766e360 commit 8a5b2db3d81db16e9e6aaea82cc071bdc766e360 Author: Konstantin Belousov AuthorDate: 2024-08-09 22:55:36 +0000 Commit: Konstantin Belousov CommitDate: 2024-08-21 15:18:39 +0000 ranglelock_destroy(): do not remove lock entries from under live lock acquirer Tested by: markj, pho Sponsored by: The FreeBSD Foundation --- sys/kern/kern_rangelock.c | 56 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/sys/kern/kern_rangelock.c b/sys/kern/kern_rangelock.c index c2f1f2d762bb..c01ed05e4a6a 100644 --- a/sys/kern/kern_rangelock.c +++ b/sys/kern/kern_rangelock.c @@ -289,6 +289,7 @@ static uma_zone_t rl_entry_zone; static smr_t rl_smr; static void rangelock_free_free(struct rl_q_entry *free); +static void rangelock_noncheating_destroy(struct rangelock *lock); static void rangelock_sys_init(void) @@ -340,16 +341,9 @@ rangelock_init(struct rangelock *lock) void rangelock_destroy(struct rangelock *lock) { - struct rl_q_entry *e, *ep; - MPASS(!lock->sleepers); - if (rangelock_cheat_destroy(lock)) - return; - for (e = (struct rl_q_entry *)atomic_load_ptr(&lock->head); - e != NULL; e = rl_e_unmark(ep)) { - ep = atomic_load_ptr(&e->rl_q_next); - uma_zfree_smr(rl_entry_zone, e); - } + if (!rangelock_cheat_destroy(lock)) + rangelock_noncheating_destroy(lock); } static bool @@ -487,6 +481,50 @@ rl_q_cas(struct rl_q_entry **prev, struct rl_q_entry *old, (uintptr_t)new) != 0); } +static void +rangelock_noncheating_destroy(struct rangelock *lock) +{ + struct rl_q_entry *cur, *free, *next, **prev; + + free = NULL; +again: + smr_enter(rl_smr); + prev = (struct rl_q_entry **)&lock->head; + cur = rl_q_load(prev); + MPASS(!rl_e_is_marked(cur)); + + for (;;) { + if (cur == NULL) + break; + if (rl_e_is_marked(cur)) + goto again; + + next = rl_q_load(&cur->rl_q_next); + if (rl_e_is_marked(next)) { + next = rl_e_unmark(next); + if (rl_q_cas(prev, cur, next)) { +#ifdef INVARIANTS + cur->rl_q_owner = NULL; +#endif + cur->rl_q_free = free; + free = cur; + cur = next; + continue; + } + smr_exit(rl_smr); + goto again; + } + + sleepq_lock(&lock->sleepers); + if (!rl_e_is_marked(cur)) { + rl_insert_sleep(lock); + goto again; + } + } + smr_exit(rl_smr); + rangelock_free_free(free); +} + enum RL_INSERT_RES { RL_TRYLOCK_FAILED, RL_LOCK_SUCCESS,