From nobody Sun Mar 17 15:39:06 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TyMc01Cpbz5DbR7; Sun, 17 Mar 2024 15:39:28 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TyMc00gvVz3y66; Sun, 17 Mar 2024 15:39:28 +0000 (UTC) (envelope-from gallatin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710689968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CZ4XTc/b83IYGjOo4Wo6K+oSfrQ/+7OzSGN09jmHsVQ=; b=K/kt8AobXbde8wAKQpLmHzfLmebOQVOWm5VV6k0nYDNU23IZBN/XHTeez/exLVV7BxUajp NmxH1xyUoRjRlJbtS/XG1IkGgMQ6TfcdrAxdhqcXOLg5Dz7QZmR5vUXvUe1DaKCqmWC9lY aCSEYLiPOWA39l6U3JlEteLxj9OhFlz35PeihlICB8yhXFhq7emL6zZu5cvdvbFW/rO0hP 4guX/yeDWQRvGVUMk+bYqzh97ofR2HYtNrfAsnVy6cIri6xTR1Vqk37cJLEGrXmBmtSTHp 1xgBT7OZLX2YrNLsA0HWBLURMQXGIHjmC8qoM/pauFVr9JTD2kmNmLvxlYPyOw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710689968; a=rsa-sha256; cv=none; b=ysWEU/x13xCQ24KHrtnXbOMYa5VRa1w+upTd2/aJgzu+h4+sGRI1D+ax3dpvtmQ5kqvusc 4VhJFQUMJFLmdUW6m2QCM0nN7VQlBW7nh84v0r0ee5ELu76EstGfXfa41qsDHMJUtunPq0 4QAyAtssT6PSHOHtl5KnDpejkQImBqUkAP7bXgeqpb4oSDFMyjdu36GnH5HAgF5jvwFDGH TSj70+SkP0G7PRFWPOSenXPiX54w+X2tuEWKKnL9bQxlkxXBQMK1cc/QHggIe95BXBZB6Z wNYSGlOwk2KZDzWb4nSEC4BpydEfUyx7AedVv/zkeBwRKg+Kvob7Ok2Ae3DEQw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710689968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CZ4XTc/b83IYGjOo4Wo6K+oSfrQ/+7OzSGN09jmHsVQ=; b=cBAXs/LmtDz/lejwYAzrmV9J5l/v7qvgCxJ5YPD1dkYK5xPwmFSmWWy1qb0k1m2CMlxHvH JgymRooVCg6gEdJb1NEEdZT2fsc16EfKMQLzzOQA2RH1znbBAkcqfkzYUaj995WfM4eLj9 80J6EXfKVaEOHxu5VN9ww6tIavxcGd5XHp2FHa2hapbs7aLrCGMiQANxyIBqxc0L3fo2fC mIGKIyE7bZKKiryvfKu1MjAixHJHuTW4boyYNASpnXWDn1kj2Y6FMBSLhiacS2fQFw+v51 i/U3oNpRAz5AtBNAoCeCjKSyBMI0iVAM07p5SvYV3GZYYEytt5TJyYYLoKRJnA== Received: from fauth2-smtp.messagingengine.com (fauth2-smtp.messagingengine.com [103.168.172.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gallatin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TyMbz5gNRz14wM; Sun, 17 Mar 2024 15:39:27 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfauth.nyi.internal (Postfix) with ESMTP id DAA891200043; Sun, 17 Mar 2024 11:39:26 -0400 (EDT) Received: from imap53 ([10.202.2.103]) by compute5.internal (MEProxy); Sun, 17 Mar 2024 11:39:26 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrkeehgdehiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkjghffffhvfevufgtsegrtd erreerredtnecuhfhrohhmpedfffhrvgifucfirghllhgrthhinhdfuceoghgrlhhlrght ihhnsehfrhgvvggsshgurdhorhhgqeenucggtffrrghtthgvrhhnpeeggfeugeevuedtue dvleefffduteegtdffudeihefhgfegfeekffeiueevkeeuudenucevlhhushhtvghrufhi iigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrghllhgrthhinhdomhgvshhmth hprghuthhhphgvrhhsohhnrghlihhthidqudeffeehledvvdduiedqvdelhedtgedukeeg qdhgrghllhgrthhinheppehfrhgvvggsshgurdhorhhgsehfrghsthhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i41414658:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 9E7A136400AB; Sun, 17 Mar 2024 11:39:26 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-300-gdee1775a43-fm-20240315.001-gdee1775a List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Message-Id: <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <486915F0-456B-4B09-A8BC-93BBA79C4CA1@freebsd.org> <20240313080624.6c73908c@ernst.home> <508E3B47-8E1B-469F-97B1-2171A3098888@freebsd.org> <86a5n1i0xg.fsf@ltc.des.dev> <78D1FF09-71A3-4486-B934-D8332F54B237@freebsd.org> <20240316104053.20bef8c2@ernst.home> <20240316115128.33d11f7b@ernst.home> <7367F29A-D52B-4828-B79A-AA2667E81E7D@freebsd.org> <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> Date: Sun, 17 Mar 2024 11:39:06 -0400 From: "Drew Gallatin" To: "Nuno Teixeira" , tuexen@freebsd.org Cc: garyj@gmx.de, current@freebsd.org, net@freebsd.org, "Randall Stewart" Subject: Re: Request for Testing: TCP RACK Content-Type: multipart/alternative; boundary=49acd8c4f2f346cdb093c5dcbb4bfe48 --49acd8c4f2f346cdb093c5dcbb4bfe48 Content-Type: text/plain I don't have the full context, but it seems like the complaint is a performance regression in bonnie++ and perhaps other things when tcp_hpts is loaded, even when it is not used. Is that correct? If so, I suspect its because we drive the tcp_hpts_softclock() routine from userret(), in order to avoid tons of timer interrupts and context switches. To test this theory, you could apply a patch like: diff --git a/sys/kern/subr_trap.c b/sys/kern/subr_trap.c index e9a16cd0b36e..54b540c97123 100644 --- a/sys/kern/subr_trap.c +++ b/sys/kern/subr_trap.c @@ -138,7 +138,7 @@ userret(struct thread *td, struct trapframe *frame) * Software Timer Support for Network Processing" * by Mohit Aron and Peter Druschel. */ - tcp_hpts_softclock(); + /*tcp_hpts_softclock();*/ /* * Let the scheduler adjust our priority etc. */ If that fixes it, I suspect we should either make this hook optional for casual users of tcp_hpts(), or add some kind of "last called" timestamp to prevent it being called over and over and over on workloads which are syscall heavy. Note that for non-casual users of hpts (like Netflix, with hundreds of thousands of TCP connections managed by hpts), this call is a huge win, so I think we'd prefer that it remain in some form. Drew --49acd8c4f2f346cdb093c5dcbb4bfe48 Content-Type: text/html Content-Transfer-Encoding: quoted-printable
I don't have th= e full context, but it seems like the complaint is a performance regress= ion in bonnie++ and perhaps other things when tcp_hpts is loaded, even w= hen it is not used.  Is that correct?

= If so, I suspect its because we drive the tcp_hpts_softclock() routine f= rom userret(), in order to avoid tons of timer interrupts and context sw= itches.  To test this theory,  you could apply a patch like:

diff --git a/sys/kern/subr_trap.c b/sys/kern= /subr_trap.c
index e9a16cd0b36e..54b540c97123 100644
--- a/sys/kern/subr_trap.c
+++ b/sys/kern/subr_tr= ap.c
@@ -138,7 +138,7 @@ userret(struct thread *td, struct= trapframe *frame)
      &nb= sp;  * Software Timer Support for Network Processing"
         * by Mohit Aron and Pe= ter Druschel.
       &n= bsp; */
-       tcp_hpts_sof= tclock();
+       /*tcp_hpts= _softclock();*/
       = /*
         * Let= the scheduler adjust our priority etc.
   =       */


If that fixes it, I suspect we should either make this hook option= al for casual users of tcp_hpts(), or add some kind of "last called" tim= estamp to prevent it being called over and over and over on workloads wh= ich are syscall heavy.

Note that for non-ca= sual users of hpts (like Netflix, with hundreds of thousands of TCP conn= ections managed by hpts), this call is a huge win, so I think we'd prefe= r that it remain in some form.

Drew

--49acd8c4f2f346cdb093c5dcbb4bfe48--