From nobody Wed Dec 29 15:47:01 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C11AB1917D23; Wed, 29 Dec 2021 15:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JPG461wkRz4TZw; Wed, 29 Dec 2021 15:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0D04627987; Wed, 29 Dec 2021 15:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1BTFl1bS001471; Wed, 29 Dec 2021 15:47:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1BTFl1DZ001470; Wed, 29 Dec 2021 15:47:01 GMT (envelope-from git) Date: Wed, 29 Dec 2021 15:47:01 GMT Message-Id: <202112291547.1BTFl1DZ001470@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 13dbf9da686c - stable/13 - x86: Perform late TSC calibration before LAPIC timer calibration List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 13dbf9da686c1c3ea1a1e197b932ea3abdf27a43 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1640792822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=z3R7BcLOGUddLvDBeSJ4vIjJCbCl5AQ95B9r0wQfo6Q=; b=t3TlceWddG/cnCwJ6NKdRdJrym/eL28fYudXhBX3jJIdiiq8yMX+XdafsAcrn8Am1GYrTl vw1cA7+6lzo/NdESUF/QwQEmP/A80OUgugofpLqxW1BSP6ctfmZyYlKFKEid7YOwBlO7t3 zAIBCpVb4AZpuZ9GjxKPn9THM2506u9B6WTnnWHcMH/0Kp46LryedvjLfkuv8Q5eRnZtmv gS9y41Hzr3aLw+QN4KCsZRaVQ/BoAINhL13LOdNlT2ZOcPH7CVui9hVWXSuFboFtOS+qC/ m7nTJXFx1Sr1EkUjBJ0sWWdspyaoBHtaQvaMEmWTDN3wZr8MzWiGnl8lHJXaqA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1640792822; a=rsa-sha256; cv=none; b=cfaIZ9V3HbU9euSrIjgDkpOzCii1FnL24Ds+I58gfCPaGNkbbjDlQCcxXAxTMa0yCpx415 Avnk9Pr7bFLtw8uDaEE+nSP+jlEVsJQdZr517u9BK+MSsHsBleAJw4XCtix8Dwsat9eW74 NbVdXxGCJz9Z/fimPlKfNa7aX4+qrhFpx8FyOx+LlxxtmfIaeXvqD4gB53gIF4CyKCclzL Q+zUdIGYQmYcuXHsQ3wTqf0QQpsVS69dJfchXQ+lIY4k24hpeTb6+P7ffBh/IhRfM+O4J/ kcLzZDntSwhWrIMvZMY11/C0xqun+gK2Mqt9TJP4qV8eIeyw2BtRoP7KqVeaSA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=13dbf9da686c1c3ea1a1e197b932ea3abdf27a43 commit 13dbf9da686c1c3ea1a1e197b932ea3abdf27a43 Author: Mark Johnston AuthorDate: 2021-12-06 15:42:19 +0000 Commit: Mark Johnston CommitDate: 2021-12-29 15:39:22 +0000 x86: Perform late TSC calibration before LAPIC timer calibration This ensures that LAPIC calibration is done using the correct tsc_freq value, i.e., the one associated with the TSC timecounter. It does mean though that TSC calibration cannot use sbinuptime() to read the reference timecounter, as timehands are not yet set up. Reviewed by: kib, jhb Sponsored by: The FreeBSD Foundation (cherry picked from commit 553af8f1ec71d397b5b4fd5876622b9269936e63) --- sys/x86/include/clock.h | 1 + sys/x86/isa/clock.c | 2 ++ sys/x86/x86/tsc.c | 26 +++++++++++++++++--------- 3 files changed, 20 insertions(+), 9 deletions(-) diff --git a/sys/x86/include/clock.h b/sys/x86/include/clock.h index d492196bac85..83c8351ed31c 100644 --- a/sys/x86/include/clock.h +++ b/sys/x86/include/clock.h @@ -28,6 +28,7 @@ void i8254_init(void); void i8254_delay(int); void clock_init(void); void lapic_calibrate(void); +void tsc_calibrate(void); /* * Driver to clock driver interface. diff --git a/sys/x86/isa/clock.c b/sys/x86/isa/clock.c index 2eb1c343f4db..513174626892 100644 --- a/sys/x86/isa/clock.c +++ b/sys/x86/isa/clock.c @@ -413,6 +413,7 @@ cpu_initclocks(void) td = curthread; + tsc_calibrate(); lapic_calibrate_timer(); cpu_initclocks_bsp(); CPU_FOREACH(i) { @@ -428,6 +429,7 @@ cpu_initclocks(void) sched_unbind(td); thread_unlock(td); #else + tsc_calibrate(); lapic_calibrate_timer(); cpu_initclocks_bsp(); #endif diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c index 53e2c7dcfe42..6debe3ecca6d 100644 --- a/sys/x86/x86/tsc.c +++ b/sys/x86/x86/tsc.c @@ -693,23 +693,27 @@ tsc_update_freq(uint64_t new_freq) /* * Perform late calibration of the TSC frequency once ACPI-based timecounters - * are available. + * are available. At this point timehands are not set up, so we read the + * highest-quality timecounter directly rather than using (s)binuptime(). */ -static void -tsc_calib(void *arg __unused) +void +tsc_calibrate(void) { - sbintime_t t_start, t_end; + struct timecounter *tc; uint64_t freq_khz, tsc_start, tsc_end; + u_int t_start, t_end; register_t flags; int cpu; if (tsc_disabled) return; + tc = atomic_load_ptr(&timecounter); + flags = intr_disable(); cpu = curcpu; tsc_start = rdtsc_ordered(); - t_start = sbinuptime(); + t_start = tc->tc_get_timecount(tc) & tc->tc_counter_mask; intr_restore(flags); DELAY(1000000); @@ -719,19 +723,23 @@ tsc_calib(void *arg __unused) flags = intr_disable(); tsc_end = rdtsc_ordered(); - t_end = sbinuptime(); + t_end = tc->tc_get_timecount(tc) & tc->tc_counter_mask; intr_restore(flags); sched_unbind(curthread); thread_unlock(curthread); - freq_khz = (SBT_1S / 1024) * (tsc_end - tsc_start) / (t_end - t_start); + if (t_end <= t_start) { + /* Assume that the counter has wrapped around at most once. */ + t_end += (uint64_t)tc->tc_counter_mask + 1; + } + + freq_khz = tc->tc_frequency * (tsc_end - tsc_start) / (t_end - t_start); - tsc_update_freq(freq_khz * 1024); + tsc_update_freq(freq_khz); tc_init(&tsc_timecounter); set_cputicker(rdtsc, tsc_freq, !tsc_is_invariant); } -SYSINIT(tsc_calib, SI_SUB_CLOCKS + 1, SI_ORDER_ANY, tsc_calib, NULL); void resume_TSC(void)