From nobody Wed Apr 24 20:41:17 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VPrVl1f1lz5J3vS; Wed, 24 Apr 2024 20:41:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VPrVk0dK7z4MX2; Wed, 24 Apr 2024 20:41:18 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713991278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0hZejh25Way3YEaaApiPmdqX/4+SgWrAI/AQgx+vcas=; b=jBfqM9iMnQyu6nMMGo9q4RBGiej1V85Jq7qhchV3/vEucRP3uxEdNUF5DxlvRlCsfAp9ve yvlkrJGPpWOBFIcM0PXwMhYoCrzCE/Uh/fOmC/NezbWZ6I8uL+GGvb+RLendP3GXKQhSOY h+hmrSYDQv7eFQvLFwwOewheck6Hr0NoBYQx9TLAC+7l+Uw5jaDEpdeE0Ds7HDdgmB6PxF kx8EhkFqL/aEL6AnLy1XBql3COMAEEDQqJGfanfc+SIUpiTfA70oCEnxK6eHDUaIhTteWs daawo3r3iWr1UkIry/rCEYJ3v5Rqq/PW483nXRB2tFqdCOK+n7HT6rtKj5Zo1A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713991278; a=rsa-sha256; cv=none; b=pgTqy5e2e5XAZaeMtOdYLyCOQl7QSJ/HYj6bKEV48NXj8M+TOZJj1B0JZMHR1dnJ04kl5H qPS/3IeHy9pePt9PMSq28krSFI77j/uhdxbV1ZiuzwMgFKb90w0HiFaT5EBj/jvkpfcVbn X7Z1HeI24GkLHpWV6hrLDfRpP5AVbqCqUkc/HsEyKTh6YKavw+kIaNouS7y5+p0Vh7aC22 utTFa3N7PpSWVre0dgJHQAYwbUKRpnRL9JEmilJkS4zApcDEuZWncIl3Q6R76fnCA4YQx7 mcd/pm3fHpbvOtYxNrWbbAOtVgemuvwIxuEeBdUXXgJ+Rupv3iqJ8SD6oEMJRg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713991278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0hZejh25Way3YEaaApiPmdqX/4+SgWrAI/AQgx+vcas=; b=WqYLvtcAUaYvcPFhGXcYQ29UJaPJ9gMTmxIOIgUha2JenFEjRbOlVces38trX5S3Srenw7 Ggpn1m0uA3wREmPcDrCI3H3i4pVZf3sz3Gs5EHvSrpn98jpoLRSWuvlUlkIJneE1z/hLtP jeWqGElcTWr+XYt+8cIi3smSleubY2eDN+3iGKvNx6MNO4OrOrkC1RbfroEfGc9lS5pHC8 kHPSUac5E0hF+Eem0CfFgBdeyPa0seV/cptrcRW+3CBRdMKxj51a3tQ8gY2sZKseQH36RG D19DBwFFDsHERvLqBrTzhOGiziFAO4A/1hJVWUlLnoxC2S53Ux/5oc2b8CyR5A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VPrVk0CTHzWRS; Wed, 24 Apr 2024 20:41:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 43OKfH9J010493; Wed, 24 Apr 2024 20:41:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 43OKfH8c010490; Wed, 24 Apr 2024 20:41:17 GMT (envelope-from git) Date: Wed, 24 Apr 2024 20:41:17 GMT Message-Id: <202404242041.43OKfH8c010490@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Michael Tuexen Subject: git: 7fdef9cdb7ee - stable/14 - Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: tuexen X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 7fdef9cdb7ee3ff9b7bc7629ee2db7c808e8f1a5 Auto-Submitted: auto-generated The branch stable/14 has been updated by tuexen: URL: https://cgit.FreeBSD.org/src/commit/?id=7fdef9cdb7ee3ff9b7bc7629ee2db7c808e8f1a5 commit 7fdef9cdb7ee3ff9b7bc7629ee2db7c808e8f1a5 Author: Randall Stewart AuthorDate: 2024-03-28 12:12:37 +0000 Commit: Michael Tuexen CommitDate: 2024-04-24 20:37:40 +0000 Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold HPTS inserts a softclock for system call return that optimizes performance. However when no HPTS threads need the help (i.e. when they have less than 100 or so connections) then there should be little work done i.e. check the counter and return instead of running through all the threads getting locks etc.ptimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold. Reported by: eduardo Reviewed by: gallatin, glebius, tuexen Tested by: gallatin Differential Revision: https://reviews.freebsd.org/D44420 (cherry picked from commit b7b78c1c169dd2213b4cb3e14e19c045b2c5e5af) --- sys/kern/subr_trap.c | 2 ++ sys/netinet/tcp_hpts.c | 10 +++++++++- sys/sys/systm.h | 4 +++- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c index 42ce9e7988b5..5a89ba8b6500 100644 --- a/sys/kern/subr_trap.c +++ b/sys/kern/subr_trap.c @@ -76,6 +76,8 @@ #include #endif +volatile uint32_t __read_frequently hpts_that_need_softclock = 0; + void (*tcp_hpts_softclock)(void); /* diff --git a/sys/netinet/tcp_hpts.c b/sys/netinet/tcp_hpts.c index 07c5f04b907e..85341cab0750 100644 --- a/sys/netinet/tcp_hpts.c +++ b/sys/netinet/tcp_hpts.c @@ -193,7 +193,8 @@ struct tcp_hpts_entry { uint8_t p_direct_wake :1, /* boolean */ p_on_min_sleep:1, /* boolean */ p_hpts_wake_scheduled:1, /* boolean */ - p_avail:5; + hit_callout_thresh:1, + p_avail:4; uint8_t p_fill[3]; /* Fill to 32 bits */ /* Cache line 0x40 */ struct hptsh { @@ -1684,6 +1685,13 @@ tcp_hpts_thread(void *ctx) ticks_ran = tcp_hptsi(hpts, 1); tv.tv_sec = 0; tv.tv_usec = hpts->p_hpts_sleep_time * HPTS_TICKS_PER_SLOT; + if ((hpts->p_on_queue_cnt > conn_cnt_thresh) && (hpts->hit_callout_thresh == 0)) { + hpts->hit_callout_thresh = 1; + atomic_add_int(&hpts_that_need_softclock, 1); + } else if ((hpts->p_on_queue_cnt <= conn_cnt_thresh) && (hpts->hit_callout_thresh == 1)) { + hpts->hit_callout_thresh = 0; + atomic_subtract_int(&hpts_that_need_softclock, 1); + } if (hpts->p_on_queue_cnt >= conn_cnt_thresh) { if(hpts->p_direct_wake == 0) { /* diff --git a/sys/sys/systm.h b/sys/sys/systm.h index 3721db649a3d..eb690b1cf4b4 100644 --- a/sys/sys/systm.h +++ b/sys/sys/systm.h @@ -393,8 +393,10 @@ extern int cpu_disable_c2_sleep; extern int cpu_disable_c3_sleep; extern void (*tcp_hpts_softclock)(void); +extern volatile uint32_t __read_frequently hpts_that_need_softclock; + #define tcp_hpts_softclock() do { \ - if (tcp_hpts_softclock != NULL) \ + if (hpts_that_need_softclock > 0) \ tcp_hpts_softclock(); \ } while (0)