From nobody Tue Jan 16 19:05:23 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TDz3l4zNMz57qxs; Tue, 16 Jan 2024 19:05:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TDz3l2PN9z54wK; Tue, 16 Jan 2024 19:05:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1705431923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NfNIEo9ntyNHcjz0wJEfNj2/tg5419XWK200RUszlTo=; b=X4GLso8F5DN2SZQ6NKK2ApC+BsVozvoMmo6ZajXl3scRitVAD/dOBuzJjm1vm0oHuvWUO6 CNCvtxNh2BfrPAYq7W4BAxDZVgWZiGOniPg0qUV7EwRe2BJ5bFTP1/3dF/Z/dwbSvv0onG GJOs3xCvYyrmGpKXfc9d1czmy7YUL0AxgOii4YVnMPOEo3h6+JLLK/omneUWNYFGV4GwO2 ywPrtHalPjQugzPm9w8tAZIqZVUQsYlGPNo9YQONhq/XZ1DdeekkaIpb/PJ9st81dugnOu Qk4eM2n0EB6iijeDQk916rp5pMfA3MRSwDMRQ6Pt1dEyFkF6WZ+C+4u9k6IMLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1705431923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NfNIEo9ntyNHcjz0wJEfNj2/tg5419XWK200RUszlTo=; b=X26WI5Xp8/m+Pd6ybow04DL/uR0oqVshOo6ehYjsFAKiMkrI+zA/aro6ZbnHBpVL4hd04a n3mC66bI7kzHElCjb9mlvzbAR6jXWn0N1sgTGDfwlbW707BydUX4mSAhVDJI6sHjoybzZm r2k0X5MgpEZ4Ggp0JLaKBgE8F/AxWJkM7X/TibGo/eJo9kB2my3qNwwn7DGyXs8jdV+RIl B2pdvbLDCpfmikw0UTFve7WLvNSU0WNTW+DK4YDcu+5YS0ruG6vQ1BG7ptaHs39eszw1KH uSNwyHEY/gvsxJ9aaX3KAiRpP1vzYIfAdrg/yXVZaN3VwDqXE1eMUp4Pm3cVQw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1705431923; a=rsa-sha256; cv=none; b=Km7eooUDEvgAgANMXv4i7XBSy0G+xRxs5BTyRwuYmj+5LoQMfWgs0xouCakIzBRsYfMdVy 11oil1zEL7H6Tfesuv8F2VADiynyPBG8ScVuj4ZgWHm2skfZuZurs3Ys8b1gmA7V3lqKsk zXLwEAcFIlb6eTwTQ2F4AgH0DwbFoMmIPISrw8WaL6l/c/+0U+OVhPKB8dLqspUxR/SO31 vwKanhTkeU9hqZWKgz6+d/I0JVDH94BE+LFwr2ZsFQrHJl9+RKarKqqshSv463DSjDJmVR 27svXGRz8Kj8n9CitCh2tL9b1N+6ZhWhII2oTEb/tdfMb2nZ6UFlYmOrl9PYvQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TDz3l1KpMzsMm; Tue, 16 Jan 2024 19:05:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 40GJ5NIF010570; Tue, 16 Jan 2024 19:05:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 40GJ5NmJ010567; Tue, 16 Jan 2024 19:05:23 GMT (envelope-from git) Date: Tue, 16 Jan 2024 19:05:23 GMT Message-Id: <202401161905.40GJ5NmJ010567@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Gleb Smirnoff Subject: git: 2ae0ff8f11ac - stable/14 - tcp_hpts: make the module unloadable List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: glebius X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 2ae0ff8f11acf18247609f3abdaaa5ca64697d63 Auto-Submitted: auto-generated The branch stable/14 has been updated by glebius: URL: https://cgit.FreeBSD.org/src/commit/?id=2ae0ff8f11acf18247609f3abdaaa5ca64697d63 commit 2ae0ff8f11acf18247609f3abdaaa5ca64697d63 Author: Gleb Smirnoff AuthorDate: 2023-12-19 18:21:56 +0000 Commit: Gleb Smirnoff CommitDate: 2024-01-16 18:47:50 +0000 tcp_hpts: make the module unloadable Although the HPTS subsytem wasn't initially designed as a loadable module, now it is so. Make it possible to also unload it, but for safety reasons hide that under 'kldunload -f'. Reviewed by: tuexen Differential Revision: https://reviews.freebsd.org/D43092 (cherry picked from commit 48b55a7c7be4175998f9b26dfbec5a561acbd936) --- sys/netinet/tcp_hpts.c | 82 ++++++++++++++++++++++++++++++++++++++++++---- sys/netinet/tcp_hpts.h | 1 + sys/netinet/tcp_lro_hpts.c | 6 ++++ 3 files changed, 83 insertions(+), 6 deletions(-) diff --git a/sys/netinet/tcp_hpts.c b/sys/netinet/tcp_hpts.c index 50a30a2cff03..a6fa79a26949 100644 --- a/sys/netinet/tcp_hpts.c +++ b/sys/netinet/tcp_hpts.c @@ -229,7 +229,7 @@ static struct tcp_hptsi { uint32_t rp_num_hptss; /* Number of hpts threads */ } tcp_pace; -MALLOC_DEFINE(M_TCPHPTS, "tcp_hpts", "TCP hpts"); +static MALLOC_DEFINE(M_TCPHPTS, "tcp_hpts", "TCP hpts"); #ifdef RSS static int tcp_bind_threads = 1; #else @@ -240,7 +240,6 @@ static int hpts_does_tp_logging = 0; static int32_t tcp_hptsi(struct tcp_hpts_entry *hpts, int from_callout); static void tcp_hpts_thread(void *ctx); -static void tcp_init_hptsi(void *st); int32_t tcp_min_hptsi_time = DEFAULT_MIN_SLEEP; static int conn_cnt_thresh = DEFAULT_CONNECTION_THESHOLD; @@ -1794,7 +1793,7 @@ hpts_gather_grps(struct cpu_group **grps, int32_t *at, int32_t max, struct cpu_g } static void -tcp_init_hptsi(void *st) +tcp_hpts_mod_load(void) { struct cpu_group *cpu_top; int32_t error __diagused; @@ -2005,10 +2004,81 @@ tcp_init_hptsi(void *st) printf("TCP Hpts created %d swi interrupt threads and bound %d to %s\n", created, bound, tcp_bind_threads == 2 ? "NUMA domains" : "cpus"); -#ifdef INVARIANTS - printf("HPTS is in INVARIANT mode!!\n"); +} + +static void +tcp_hpts_mod_unload(void) +{ + int rv __diagused; + + tcp_lro_hpts_uninit(); + atomic_store_ptr(&tcp_hpts_softclock, NULL); + + for (int i = 0; i < tcp_pace.rp_num_hptss; i++) { + struct tcp_hpts_entry *hpts = tcp_pace.rp_ent[i]; + + rv = callout_drain(&hpts->co); + MPASS(rv != 0); + + rv = swi_remove(hpts->ie_cookie); + MPASS(rv == 0); + + rv = sysctl_ctx_free(&hpts->hpts_ctx); + MPASS(rv == 0); + + mtx_destroy(&hpts->p_mtx); + free(hpts->p_hptss, M_TCPHPTS); + free(hpts, M_TCPHPTS); + } + + free(tcp_pace.rp_ent, M_TCPHPTS); + free(tcp_pace.cts_last_ran, M_TCPHPTS); +#ifdef SMP + free(tcp_pace.grps, M_TCPHPTS); #endif + + counter_u64_free(hpts_hopelessly_behind); + counter_u64_free(hpts_loops); + counter_u64_free(back_tosleep); + counter_u64_free(combined_wheel_wrap); + counter_u64_free(wheel_wrap); + counter_u64_free(hpts_wake_timeout); + counter_u64_free(hpts_direct_awakening); + counter_u64_free(hpts_back_tosleep); + counter_u64_free(hpts_direct_call); + counter_u64_free(cpu_uses_flowid); + counter_u64_free(cpu_uses_random); +} + +static int +tcp_hpts_modevent(module_t mod, int what, void *arg) +{ + + switch (what) { + case MOD_LOAD: + tcp_hpts_mod_load(); + return (0); + case MOD_QUIESCE: + /* + * Since we are a dependency of TCP stack modules, they should + * already be unloaded, and the HPTS ring is empty. However, + * function pointer manipulations aren't 100% safe. Although, + * tcp_hpts_mod_unload() use atomic(9) the userret() doesn't. + * Thus, allow only forced unload of HPTS. + */ + return (EBUSY); + case MOD_UNLOAD: + tcp_hpts_mod_unload(); + return (0); + default: + return (EINVAL); + }; } -SYSINIT(tcphptsi, SI_SUB_SOFTINTR, SI_ORDER_ANY, tcp_init_hptsi, NULL); +static moduledata_t tcp_hpts_module = { + .name = "tcphpts", + .evhand = tcp_hpts_modevent, +}; + +DECLARE_MODULE(tcphpts, tcp_hpts_module, SI_SUB_SOFTINTR, SI_ORDER_ANY); MODULE_VERSION(tcphpts, 1); diff --git a/sys/netinet/tcp_hpts.h b/sys/netinet/tcp_hpts.h index 7eb1b2e08cb4..0c5cfac28a6c 100644 --- a/sys/netinet/tcp_hpts.h +++ b/sys/netinet/tcp_hpts.h @@ -153,6 +153,7 @@ void __tcp_set_hpts(struct tcpcb *tp, int32_t line); void tcp_set_inp_to_drop(struct inpcb *inp, uint16_t reason); void tcp_lro_hpts_init(void); +void tcp_lro_hpts_uninit(void); extern int32_t tcp_min_hptsi_time; diff --git a/sys/netinet/tcp_lro_hpts.c b/sys/netinet/tcp_lro_hpts.c index 769c82a32391..7724e727d5ce 100644 --- a/sys/netinet/tcp_lro_hpts.c +++ b/sys/netinet/tcp_lro_hpts.c @@ -584,3 +584,9 @@ tcp_lro_hpts_init(void) { tcp_lro_flush_tcphpts = _tcp_lro_flush_tcphpts; } + +void +tcp_lro_hpts_uninit(void) +{ + atomic_store_ptr(&tcp_lro_flush_tcphpts, NULL); +}