From nobody Wed Jan 08 22:20:19 2025 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YT2RX6FzCz5k5Gm for ; Wed, 08 Jan 2025 22:20:24 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YT2RX3cR8z4gfg for ; Wed, 8 Jan 2025 22:20:24 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-844eac51429so19051139f.2 for ; Wed, 08 Jan 2025 14:20:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736374823; x=1736979623; darn=freebsd.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=4gH9jF7spXB+iY0BRsgo/evvPSo+p8EgbF3ACWuDLq8=; b=WbuOTg3hUF9xjEanBk/zHSsUgasdEevoEZhYfjD32vpPZGSiMFOFKEoc1GhoEfycem agQ0NS+msHp6LHI1rSe/Erz9/ZcjOMzb+wfeHz/ayfvu6TjKjR820dBkHOQ4XSmHazsi OnKAl18ku7+TbobkidgFm+C+uavtYtf/8/IhdUvwzzlap+sGbJCwojgOxqqnUMbO8d3q ivDOfYjjO8hKSBjhSkrY9OYBo+QuIO64KF9gjywkFkiJVEJm1+U0C5DmyM004Y0xAkKC otyo7tz0cfSqxhZ9+r1fUl3PXDZrJtlPQY3/YrJXgcA3IutUKm9OFGhKOAcMkgmGA/AZ mBnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736374823; x=1736979623; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4gH9jF7spXB+iY0BRsgo/evvPSo+p8EgbF3ACWuDLq8=; b=sPJg9YK1vaKpjHNKNcTJ+aKzfKk7Vk1SyipJcLw0mtOf7Rh/X675+59ZQq6Eua2Iqm Nxk2PtDJ3lLXmK31z3d6R7D/l6NI4EZrG1bHhOlOgQdtXS70JvjVm+vFqdeCu8/Ltr8E 0a6iGC6t4aVPbcjpKU7lZHz+V9t2xkqMApcpgwMpExk331kByqgIB4y5YFA0Oy62JS2i 6mOT3sLiyrxoOxqElyVoAKmTduzJQ819br2SsFVJXz55hOPU58Wk3bxoa/iw3pnkr9cS azi4b1wrVGqWPuPT5+vGOU88iWqMTTEDuQtm939DoNRyaSsjUVIjfZncOQWXPn7/MERS E9wQ== X-Gm-Message-State: AOJu0YyPpdOXfJkZKWa/AKaNIxPaIHIEeWu7exnIVH/Myly08Mu9VSE4 bHBnuzib/fLsyC6UePKarLmRIe1WLL7OCbLcY2SWktBjqDCqAJpBkVUcgg== X-Gm-Gg: ASbGnctdHWJ2ZHPl/nUHDhbgnKBFUtpp0Q456kImpePZrlDcLb33mhyL8c1ND0b8SnD XxNjzmQ6/H3l2fELp44GaildDqLFEWCOftPeBUVPa+u7VJWRJd5h/h+JQOd/I1KRn/g/0dOiX4R c/tvN2+JjOk4LOFOqKBqBxEi7Ix7yJaNnoE50MHSMEvRrOe9hM7WX/EUgWQFDTKi8d4silLXPTk jBGrd0+h6syo3sYhFB6pL+F5lZnXCPZjOZWHTuw5Wg7xrvYJC3OjW72Io+lACrGicN+0HA= X-Google-Smtp-Source: AGHT+IEa46/T+9O9G0GiO3p8KlE0ctnUhgJATGrdPS3/JStEtEYqBucc9dxagqAGKOw0Hz0HYTjvZQ== X-Received: by 2002:a92:cda6:0:b0:3a7:70a4:6877 with SMTP id e9e14a558f8ab-3ce3a87a690mr35967105ab.7.1736374823535; Wed, 08 Jan 2025 14:20:23 -0800 (PST) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3ce4af5649csm79875ab.64.2025.01.08.14.20.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 14:20:22 -0800 (PST) Date: Wed, 8 Jan 2025 17:20:19 -0500 From: Mark Johnston To: Warner Losh Cc: freebsd-hackers@freebsd.org Subject: Re: widening ticks Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 4YT2RX3cR8z4gfg X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On Wed, Jan 08, 2025 at 02:51:31PM -0700, Warner Losh wrote: > On Wed, Jan 8, 2025 at 2:31 PM Mark Johnston wrote: > > > The global "ticks" variable counts hardclock ticks, it's widely used in > > the kernel for low-precision timekeeping. The linuxkpi provides a very > > similar variable, "jiffies", but there's an incompatibility: the former > > is a signed int and the latter is an unsigned long. It's not > > particularly easy to paper over this difference, which has been > > responsible for some nasty bugs, and modifying drivers to store the > > jiffies value in a signed int is error-prone and a maintenance burden > > that the linuxkpi is supposed to avoid. > > > > It would be nice to provide a compatible implementation of jiffies. I > > can see a few approaches: > > - Define a 64-bit ticks variable, say ticks64, and make hardclock() > > update both ticks and ticks64. Then #define jiffies ticks64 on 64-bit > > platforms. This is the simplest to implement, but it adds extra work > > to hardclock() and is somewhat ugly. > > - Make ticks an int64_t or a long and convert our native code > > accordingly. This is cleaner but requires a lot of auditing to avoid > > introducing bugs, though perhaps some code could be left unmodified, > > implicitly truncating the value to an int. For example I think > > sched_pctcpu_update() is fine. I've gotten an amd64 kernel to compile > > and boot with this change, but it's hard to be confident in it. This > > approach also has the potential downside of bloating structures that > > store a ticks value, and it can't be MFCed. > > - Introduce a 64-bit ticks variable, ticks64, and > > #define ticks ((int)ticks64). This requires renaming any struct > > fields and local vars named "ticks", of which there's a decent number, > > but that can be done fairly mechanically. > > > > Is there another solution which avoids these pitfalls? If not, should > > we go ahead with one of these approaches? If so, which one? > > > > So solution (1) is MFC-able, I think, so I like it. > (2) Isn't, but is likely a better long-term solution. > (3) is a non-starter since ticks is too common a name to #define. Why is that a non-starter? This is just in the kernel, and as you note below, shadowing ticks isn't a great idea anyway. (I don't really want to go down this path in any case, but I'm wondering if I misunderstood something.) > I could easily see a situation where we do (1) and then convert all current > users of ticks to be ticks64. This could proceed one at a time with as much > haste or caution as we need. Once we convert all of them over, we could > delete ticks and then there'd be no extra work in hardclock. This too would > be MFC-able. > > sys/net/iflib.c: uint64_t this_tick = ticks; > sys/netinet/tcp_subr.c: < (u_int)ticks))) { > > look fun! We also shadow it in a lot of places. The TCP stack uses it a lot > with a bunch of different variables, struct entries, etc, including RACK > and BBR. > The 802.11 stack uses it a bunch. As to a bunch of drivers, sometimes > shadowing > other times not. > > It would be a lot to audit all this, so I think having the new API in place > might be > better, and incrementally converting / removing the shadowing (even if it > isn't > completely in scoe, using ticks as a local variable is begging for trouble). Yeah, looking some more, I think having a flag day will make this too painful. So then I guess the question is, do we provide an int64_t ticks64 or a long ticksl? Do we have any 32-bit platforms where a 64-bit cmpset in hardclock() would be a problem? > Warner > > Also I see both jiffies and jiffies_64 defined. Does that matter at all? They differ only on 32-bit systems I believe. On such systems there is a 64-bit tick counter, jiffies_64, but it might not be atomic.