From nobody Thu Mar 28 12:13:17 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V52W16Plyz5Fl95; Thu, 28 Mar 2024 12:13:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V52W15w5xz4VJH; Thu, 28 Mar 2024 12:13:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711627997; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zDuBaP1HMPAZHiIzWeIK2tkDZ6TDosLZ7+ebw5ho/2I=; b=oH0r/tp0LwIvUXhfFy0+IwjpRhv82WZIHRPvsZPWBhlX49ds6zr1rOXgFALKAI5IEhkwoZ yfqg84hkwh7++d1GyAzMuJRmfxIWDMXTAEKLcqBbHKohJb2ktlJtibKHYgES2Sw7CxB1Zp +5InduIzFm5Xf1TRrxGior8cbecCTAyYdP09DT4eYBMWMc8/wjnZDWLR3etv+mFdkWcT49 v5xBG1HQOFgTARtq2/c74mD2IYTrQpyTA9IUzVCq5cDTttS30L0nwCxf+8kWcyEou8p2Ga 18x3NkvTv/eqU3dICwRpJyjYSp0r/mgLiIL/T7UvH1N0QpBUOHsaZ7x4+4oAYA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711627997; a=rsa-sha256; cv=none; b=j/LxL1wAddpqmD4yURZBB0vbrvsAoI9ylguSiCVAu9hRKQ45UKN1jyfKbRy0dbbouSY2to +Ij4gpdHdY1kKqwapFIBf3Fucyms5EyYJ9F14NjDn0hYjzocrqx6TZKBKvnVau70dAVvRb eaJGnburfKh1AJ/iTRnKtCa4HHuarcyuUxcEaqSCuhl328PcIfo3f0GdVoOYwefcRQilOb LcqlH87ub0gdIHpBNEjfjbrKhVrkTI6sVh9mAtUH1VweqbV9t7RURU8Cbyen8wGVSeHOo6 9XpVyGX4ADJkwP9EuC4/xdMbX9QeWhzFUF3Wc5bwfXgDfy46aBtgtTXvYXARSw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711627997; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zDuBaP1HMPAZHiIzWeIK2tkDZ6TDosLZ7+ebw5ho/2I=; b=eXlOcKmzQS0Ly4jsu7fHxV86ljz+yelTydab5wfAQbfCWKz69zWHYdgYCjCy7kyCg1xm/r qaRHNdMpu4ghaEDhA9loHyb4eKt/V3P/pPK4HWgo7o4SSUIkux4oKmWxfNFRlGHgnyyY5D znCN4hmiZLDA78nnZmhHkmKGcGgdgSXYu2ChHjDJhHpy7JpB9xQxESfAdWk7+3lgZvLPr8 k4TcW1Jt/dUrNHRrZxZAtK1HfpkywSYT29S+Br/7gKHo1QCSr0s6FMjdNNg2vmtn/fzmlW 8bHObreOlT3GK4jJeO8X3PsC4aRiZ0vT0d0PQiY7TuKwk1pXFsRHrZXicG4gUw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4V52W15D3QzRpC; Thu, 28 Mar 2024 12:13:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 42SCDHIK025955; Thu, 28 Mar 2024 12:13:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 42SCDH2P025953; Thu, 28 Mar 2024 12:13:17 GMT (envelope-from git) Date: Thu, 28 Mar 2024 12:13:17 GMT Message-Id: <202403281213.42SCDH2P025953@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Randall Stewart Subject: git: b7b78c1c169d - main - Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rrs X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b7b78c1c169dd2213b4cb3e14e19c045b2c5e5af Auto-Submitted: auto-generated The branch main has been updated by rrs: URL: https://cgit.FreeBSD.org/src/commit/?id=b7b78c1c169dd2213b4cb3e14e19c045b2c5e5af commit b7b78c1c169dd2213b4cb3e14e19c045b2c5e5af Author: Randall Stewart AuthorDate: 2024-03-28 12:12:37 +0000 Commit: Randall Stewart CommitDate: 2024-03-28 12:12:37 +0000 Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold HPTS inserts a softclock for system call return that optimizes performance. However when no HPTS threads need the help (i.e. when they have less than 100 or so connections) then there should be little work done i.e. check the counter and return instead of running through all the threads getting locks etc.ptimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold. Reported by: eduardo Reviewed by: gallatin, glebius, tuexen Tested by: gallatin Differential Revision: https://reviews.freebsd.org/D44420 --- sys/kern/subr_trap.c | 2 ++ sys/netinet/tcp_hpts.c | 10 +++++++++- sys/sys/systm.h | 4 +++- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c index e9a16cd0b36e..18388ae5f232 100644 --- a/sys/kern/subr_trap.c +++ b/sys/kern/subr_trap.c @@ -74,6 +74,8 @@ #include #endif +volatile uint32_t __read_frequently hpts_that_need_softclock = 0; + void (*tcp_hpts_softclock)(void); /* diff --git a/sys/netinet/tcp_hpts.c b/sys/netinet/tcp_hpts.c index 8c4d2d41a3eb..222f8e0229dd 100644 --- a/sys/netinet/tcp_hpts.c +++ b/sys/netinet/tcp_hpts.c @@ -193,7 +193,8 @@ struct tcp_hpts_entry { uint8_t p_direct_wake :1, /* boolean */ p_on_min_sleep:1, /* boolean */ p_hpts_wake_scheduled:1, /* boolean */ - p_avail:5; + hit_callout_thresh:1, + p_avail:4; uint8_t p_fill[3]; /* Fill to 32 bits */ /* Cache line 0x40 */ struct hptsh { @@ -1683,6 +1684,13 @@ tcp_hpts_thread(void *ctx) ticks_ran = tcp_hptsi(hpts, 1); tv.tv_sec = 0; tv.tv_usec = hpts->p_hpts_sleep_time * HPTS_TICKS_PER_SLOT; + if ((hpts->p_on_queue_cnt > conn_cnt_thresh) && (hpts->hit_callout_thresh == 0)) { + hpts->hit_callout_thresh = 1; + atomic_add_int(&hpts_that_need_softclock, 1); + } else if ((hpts->p_on_queue_cnt <= conn_cnt_thresh) && (hpts->hit_callout_thresh == 1)) { + hpts->hit_callout_thresh = 0; + atomic_subtract_int(&hpts_that_need_softclock, 1); + } if (hpts->p_on_queue_cnt >= conn_cnt_thresh) { if(hpts->p_direct_wake == 0) { /* diff --git a/sys/sys/systm.h b/sys/sys/systm.h index 435e6b61c904..6c2b466c4cfe 100644 --- a/sys/sys/systm.h +++ b/sys/sys/systm.h @@ -400,8 +400,10 @@ extern int cpu_disable_c2_sleep; extern int cpu_disable_c3_sleep; extern void (*tcp_hpts_softclock)(void); +extern volatile uint32_t __read_frequently hpts_that_need_softclock; + #define tcp_hpts_softclock() do { \ - if (tcp_hpts_softclock != NULL) \ + if (hpts_that_need_softclock > 0) \ tcp_hpts_softclock(); \ } while (0)