From nobody Thu Mar 23 07:57:45 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PhyPj5j8tz4123x for ; Thu, 23 Mar 2023 07:58:01 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PhyPj2Crrz3rqx for ; Thu, 23 Mar 2023 07:58:01 +0000 (UTC) (envelope-from Alexander@leidinger.net) Authentication-Results: mx1.freebsd.org; none Received: from outgoing.leidinger.net (p5b165130.dip0.t-ipconnect.de [91.22.81.48]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) client-signature ECDSA (P-256)) (Client CN "outgoing.leidinger.net", Issuer "R3" (verified OK)) by mailgate.Leidinger.net (Postfix) with ESMTPSA id 6715424A29 for ; Thu, 23 Mar 2023 08:57:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1679558269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=H7jHQznVTEq0tvDILTaMIGWqRSkWGJa63f2tIPr1A9I=; b=odTX7QzortXqe0lOV3hH4tWiGaI6wC774UEcc5gTGDJ5Fcyei/APPI9keTZ07A+BVxCli2 0ZGl6atleFp0hPdJqbYYkvvPF4NWNPj4f6GbL+2djeNG6AUSfhGpnEShqwrsX6JGv8ugWs eiAO1cPsJMiZro9yEKCQ59Ee1VeddrKtevWzGkGDOVf8Em46K3xmeDNZHZthUVNlmaajVk 2NdG+6Yzc05/rjrYWuvp2wU29mv0vV5rF9AR7qPO/wsVo485H5sjZZ4qJZ80V2bhT5EiQS aKfRkxvvaJ8oAo1lCU0QXgo2KhNKP3ZU/wMDuDeMM/XBZKKM0uTCvcIBeQa9Yg== Received: from webmail.leidinger.net (localhost [127.0.0.1]) by outgoing.leidinger.net (Postfix) with ESMTP id 0B9AA370D for ; Thu, 23 Mar 2023 08:57:46 +0100 (CET) Received: from www (uid 80) (envelope-from Alexander@leidinger.net) id ac50c by webmail.leidinger.net (DragonFly Mail Agent v0.13+ on webmail.leidinger.net); Thu, 23 Mar 2023 08:57:45 +0100 Date: Thu, 23 Mar 2023 08:57:45 +0100 Message-ID: <20230323085745.Horde.eeJqWekvVqCybD-psTGfIRE@webmail.leidinger.net> From: Alexander Leidinger To: Mateusz Guzik Cc: sgk@troutmask.apl.washington.edu, Matthias Andree , freebsd-hackers@freebsd.org Subject: Re: Periodic rant about SCHED_ULE References: <8173cc7e-e934-dd5c-312a-1dfa886941aa@FreeBSD.org> <8cfdb951-9b1f-ecd3-2291-7a528e1b042c@m5p.com> In-Reply-To: Accept-Language: de,en Content-Type: multipart/signed; boundary="=_xdph0Tnv8P9I0E3lje7qzJ1"; protocol="application/pgp-signature"; micalg=pgp-sha256 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 X-Rspamd-Queue-Id: 4PhyPj2Crrz3rqx X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34240, ipnet:89.238.64.0/18, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N This message is in MIME format and has been PGP signed. --=_xdph0Tnv8P9I0E3lje7qzJ1 Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting Mateusz Guzik (from Wed, 22 Mar 2023=20=20 20:23:42=20+0100): > sched_4bsd *round robins* all workers across all CPUs, which comes at > a performance *hit* compared to ule if number of workers is <=3D CPU > count -- here ule maintains affinity, avoiding cache busting. If you > slap in $cpu_count + 1 workers, 4bsd keeps the round robin equally > penalizing everyone, while ule mostly penalizes a select victim. By > the end of it, you get lower total cpu time spent, but higher total > real time. See below for an example. > > Two massive problems with 4bsd, apart from mandatory round robin which > also happens to help by accident: > 1. it has one *global* lock, meaning the scheduler itself just does > not scale and this is visible at modest contemporary scales > 2. it does not understand topology -- no accounting done for ht nor > numa, but i concede the latter wont be a factor for most people > > That said, ULE definitely has performance bugs which need to be fixed. > At least for the case below, 4BSD just "lucked" into sucking less > simply because it is so primitive. IIRC ULE also has a semantic difference compared with 4BSD in terms of=20= =20 handling=20of the "idle" (and realtime) priority. What I remember (it's=20= =20 been=20a while since ULE was introduced) is, that with 4BSD the=20=20 idprio(1)=20case could result in such processes getting no CPU time,=20=20 whereas=20with ULE it is quaranteed that such processes will get "some"=20= =20 (no=20idea what a more specific description of "some" would be) CPU=20=20 time.=20BTW: someone may want to review the BUGS part of idprio(1) if=20=20 the=20part about "preempted" is still correct. I remember there was a lengthy discussion about the handing of process=20= =20 priority=20back then, but a quick googling didn't seems to have had the=20= =20 right=20keywords to find it. What I got was this: Original ULE design/scientific paper: https://web.cs.ucdavis.edu/~roper/ecs150/ULE.pdf Laymans-terms description: https://people.freebsd.org/~jeff/FBSD_14_10ULE_aj.jm_cx10_09.pdf ULE 2.0: =20=20=20=20=20=20 https://lists.freebsd.org/pipermail/freebsd-current/2007-January/068404.htm= l Discussion=20about ULE/prio/sche_preempt_threads sysctl: =20=20=20=20=20=20 https://forums.freebsd.org/threads/what-is-sysctl-kern-sched-preempt_thresh= .85601/page-2 Bye, Alexander. --=20 http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_xdph0Tnv8P9I0E3lje7qzJ1 Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIzBAABCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmQcBnkACgkQEg2wmwP4 2IZO1BAAjzrfS3XZNUxEOWSBQuU8UvYUr3IYPHU4fqzXYprydAe/c05Fu85DUA+M 8iLGMDg073yTjOx+CzD9j56stBffr9UWiIuw0DOvkV8XtIRveiyZQw2DNK+ppnSZ 2C0HI0EbARipsrQd+g0eCHVc0i+9u8Ntr8EIMlzdiXu6BEtw4Pc1dzf4qyCVq7Tc gLEg0812v3ErvEIQcP4bKlN0uWQQlSjjADXh9bk2IPRwW1a8fIWiM1hujK7qrFrL gK6PcjbPCqUxwFUR8ogX4wkwLn8l8FtTlwjlq5ZRabuJeaJPG6cHbTyStZ3qgDMt GjzqEmsh3mHJrE5OtBn2sonOJvMaSJqigK9Tf+1Y021EvySmkaeiytFcWKnWAyHE NRQjJJWzrf5caQC4hTX/BhSE4TpWAuZ4J6hsdxNr3uhOKMc8gNhpOqUA6RljXLfJ 5L9k3SQvihlysK1JW+fxu75MiCYCF5YOlH43XqRULwMwQ03i47KKyDbeEkvZTsHX siowP7jDeq0TtW2USADSGRqlhG7QIgRWgNjnGtGBg7qOVghuQNQKcKdHFEyj4cuT o1WV+7wIMPIC1riOmKEIxWlDlp6yvURfTulcQM+6xNFkmMeoHEUI4CjxI97WDlsE xU2b2QY09DfJFoE9Hn3KNcAUIbNLBBP28SmYd0z7r7OpOgxjHZQ= =gkYZ -----END PGP SIGNATURE----- --=_xdph0Tnv8P9I0E3lje7qzJ1--