From nobody Mon Oct 16 07:37:51 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S888y3DC8z4x2tC; Mon, 16 Oct 2023 07:37:54 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4S888y2jcWz4YHq; Mon, 16 Oct 2023 07:37:54 +0000 (UTC) (envelope-from kp@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1697441874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l91kvURmSb9nHaSCNfegLEjxhfZcC+MMGZHOR2F6+6g=; b=w8sGGekJ86z/UreVMtGdWE4GC6sf1KupKNRSQhAG+9AERnFoqyzYIJ5U1IqWl74esYu9Rx ovnby7Bsbpaj6SSqmaB9EjsjqlQ6eXEL3PE7Mo/rF1mCCBbFryTyN0FlIrnTx4KTfZ+cex qpCauq4vbjHCbqC1qxcsnGMSHuYTobTOhNyiEMp/vQnx1Ppec2x4ng3YaIN7uMOtrC5jlO 33d4cjecaFcWrsgmcYWRKVtvkCdjr7NBSp58uJ6hXjaYCO5LYR2z/BmiSkQhxOuA1gUPLs 1WVsU2ReC8S+M7eih3cfV6SRMHSHBhR7Ruvisj3a78utcLMzUXiFf4DHWYaBZg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1697441874; a=rsa-sha256; cv=none; b=XIp6lFuMFFrQ/WX68irHc5TbRFhCLkQaabjC/6by8SVIeejlvVeiBeQ79bZ++h6tL8qcWI VbvD29qfttZDusP6OOCp7b0d5hvo+k4kYl+Vlaxfl0o3E7wluS7hVCAzF1NfTZfAXexTqX vRWq/xa9NX89CX+9x+n7cENqQW/n01KYBnZke81pY38TzAYrREkWYEMDn3SPSki0wPelLY XCfyr/t+/FIvvjb/Q+lcEMBZVDpy+pkH9owtrsTlVsaln2vEwX0IGAmKVfE7dHMEWdyUO/ wtSvLDiVoy9it4IKp1uCohjJ2zN2TkFBNTwlPxM1A7OUqT7WTRMSQ8toCxfQYg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1697441874; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l91kvURmSb9nHaSCNfegLEjxhfZcC+MMGZHOR2F6+6g=; b=Z1oBI9uVIF/mCG0YTEIpSredzbq+F8p3WVOSdeKMHBGdvVkfRuE1yShiJNhu0wpQRuye6n f6hqTLiOkuwOxj2trtMoNKjghZQn1BJqPsDTlTG8lAZbABHl7p3ViPDDm68u3Pc4+Yq4Xq ucTeHulWajGuBL1PIlgJK+k3sI4ctWRYoA4tOskKiynx8OQZIf+Gtih79A+C2asCrOcRhK RxYR71GpeLkRvP1AXBHf+udEe0GsZfeSn5GsPJ9UysS8br9JIsdwLgquR6aWE1u3S7I4oj 7y3+di35eMgiLVkK6mERJNbtSIyyX5pI39OCys/cGluJUJdWxweH6hr/y7rEaQ== Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 4S888y11mMz5Hd; Mon, 16 Oct 2023 07:37:54 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 3AAE94B26E; Mon, 16 Oct 2023 09:37:52 +0200 (CEST) From: Kristof Provost To: Shawn Webb Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 2cef62886dc7 - main - pf: convert state retrieval to netlink Date: Mon, 16 Oct 2023 09:37:51 +0200 X-Mailer: MailMate (1.14r5937) Message-ID: <92D6FE8C-50FF-4DC2-B0BF-29CBC7A4075C@FreeBSD.org> In-Reply-To: <20231015201139.zt7mfyss4ua2bkn3@mutt-hbsd> References: <202310100950.39A9oYuc029996@gitrepo.freebsd.org> <20231015201139.zt7mfyss4ua2bkn3@mutt-hbsd> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 15 Oct 2023, at 22:11, Shawn Webb wrote: > On Tue, Oct 10, 2023 at 09:50:34AM +0000, Kristof Provost wrote: >> The branch main has been updated by kp: >> >> URL: https://cgit.FreeBSD.org/src/commit/?id=3D2cef62886dc7c33ca01f70c= a712845da1e55b470 >> >> commit 2cef62886dc7c33ca01f70ca712845da1e55b470 >> Author: Alexander V. Chernikov >> AuthorDate: 2023-09-15 10:06:59 +0000 >> Commit: Kristof Provost >> CommitDate: 2023-10-10 09:48:21 +0000 >> >> pf: convert state retrieval to netlink >> >> Use netlink to export pf's state table. >> >> The primary motivation is to improve how we deal with very large s= tate >> stables. With the previous implementation we had to build the enti= re >> list (both in the kernel and in userspace) before we could start >> processing. With netlink we start to get data in userspace while t= he >> kernel is still generating more. This reduces peak memory consumpt= ion >> (which can get to the GB range once we hit millions of states). >> >> Netlink also makes future extension easier, in that we can easily = add >> fields to the state export without breaking userspace. In that reg= ard >> it's similar to an nvlist-based approach, except that it also deal= s >> with transport to userspace and that it performs significantly bet= ter >> than nvlists. Testing has failed to measure a performance differen= ce >> between the previous struct-copy based ioctl and the netlink appro= ach. >> >> Differential Revision: https://reviews.freebsd.org/D38888 >> --- >> include/Makefile | 3 +- >> lib/libpfctl/libpfctl.c | 214 +++++++++++++++++---------------- >> sys/conf/files | 1 + >> sys/modules/pf/Makefile | 2 +- >> sys/netpfil/pf/pf_ioctl.c | 5 + >> sys/netpfil/pf/pf_nl.c | 292 +++++++++++++++++++++++++++++++++++++= +++++++++ >> sys/netpfil/pf/pf_nl.h | 105 +++++++++++++++++ >> 7 files changed, 522 insertions(+), 100 deletions(-) > >> diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c >> index db8f481a1567..42c2aa9bfb01 100644 >> --- a/sys/netpfil/pf/pf_ioctl.c >> +++ b/sys/netpfil/pf/pf_ioctl.c >> @@ -83,6 +83,7 @@ >> #include >> #include >> #include >> +#include >> #include >> >> #ifdef INET6 >> @@ -6648,6 +6649,8 @@ pf_unload(void) >> } >> sx_xunlock(&pf_end_lock); >> >> + pf_nl_unregister(); >> + >> if (pf_dev !=3D NULL) >> destroy_dev(pf_dev); >> >> @@ -6683,6 +6686,7 @@ pf_modevent(module_t mod, int type, void *data) >> switch(type) { >> case MOD_LOAD: >> error =3D pf_load(); >> + pf_nl_register(); >> break; >> case MOD_UNLOAD: >> /* Handled in SYSUNINIT(pf_unload) to ensure it's done after >> @@ -6703,4 +6707,5 @@ static moduledata_t pf_mod =3D { >> }; >> >> DECLARE_MODULE(pf, pf_mod, SI_SUB_PROTO_FIREWALL, SI_ORDER_SECOND); >> +MODULE_DEPEND(pf, netlink, 1, 1, 1); >> MODULE_VERSION(pf, PF_MODVER); > > Hey Kristof, > > This causes a hard dependency on the netlink kernel module, which may > not be available in some configurations. > > For safety reasons, HardenedBSD prevents loading of netlink.ko by > default. The code is too new and too complex, with already a > not-so-nice security history, to be trusted. > > A lot (all?) of the other netlink integration code respects the > potential unavailability of netlink (or netlink.ko). Would it be > possible to do the same in pf? > No. I=E2=80=99m not maintaining multiple configuration paths for pf. There=E2= =80=99s currently fallback paths to support old binaries, but I intend to= remove that code as soon as possible. Having configuration paths that ar= e untested, practically untestable and unmaintained is not a situation we= want to maintain either. Kristof