From nobody Mon Dec 06 16:21:50 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BA16618B08D5; Mon, 6 Dec 2021 16:21:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J77wt4f56z4tB7; Mon, 6 Dec 2021 16:21:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 553EC6CD6; Mon, 6 Dec 2021 16:21:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1B6GLoOu070891; Mon, 6 Dec 2021 16:21:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1B6GLoTx070890; Mon, 6 Dec 2021 16:21:50 GMT (envelope-from git) Date: Mon, 6 Dec 2021 16:21:50 GMT Message-Id: <202112061621.1B6GLoTx070890@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: 553af8f1ec71 - main - x86: Perform late TSC calibration before LAPIC timer calibration List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 553af8f1ec71d397b5b4fd5876622b9269936e63 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1638807711; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4UisTjvrpjo+ALuVWlg5bJna85FYCiFq4zitCALp2/M=; b=MZTZ4nOeLeaM6zCYQGG7+xQyj0bANWxR3bCmY3fYurPcUqL7EVHyXHySgVH5u2ChaFwhRj AkltTrDXHu0lEs6j6VULuLDbxwhfcrJ4XDzzadAWNcntgjb29kJOERXrEthnG6/APgveVT 2A8Ap6fXHTKAAKnX4tIIjHt4Mp7d3+CPQbViqBgwOXnw0pv1gFVLZFa+rvIJk4Xl/MZpmi +qMckPpaNEzUy474rLlLjhfF2O5RQp6oWz/5PafPvhvlcFt9UPGm7ydQOgIuucTu+IQweq 4Fu6UtWBrSeLudrd6u7pLV5N+Nr+BwHqcHQTUuiSq0oRYFFz0Bhxn3p5Z6PY5g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1638807711; a=rsa-sha256; cv=none; b=nTOByIWueHiRFPSxYVWmpLacqINRv/0yh2XoUr93SOXatMeST/Up7QVHfkTKpgaa6ogwEG uHAU4EYQ4LNxJAZmVDeGfII5VjW0qSf/6UZKoF6cxtgE9F/ONOS8tIbWr8xsSVYE1fNGPo OgSXtnrfb2IwWTuk0IWSqV3D+0bRySs5pu1McCbeLCiVZ5Xkc8IMOtknwVmFG+N2nmFFbT 6fSsP5KfAWV91oU0zaWIgqmIHl2bm3nRCAG1SNAzeg56cUpK5FnzVeVZQnbqn49692mfbN joj5qm5AyJa52lvSz0Iz+x0rSWQTv0vI0K4O83mdd1A1xpxzvlNY/R94ZiJRMQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=553af8f1ec71d397b5b4fd5876622b9269936e63 commit 553af8f1ec71d397b5b4fd5876622b9269936e63 Author: Mark Johnston AuthorDate: 2021-12-06 15:42:19 +0000 Commit: Mark Johnston CommitDate: 2021-12-06 15:42:19 +0000 x86: Perform late TSC calibration before LAPIC timer calibration This ensures that LAPIC calibration is done using the correct tsc_freq value, i.e., the one associated with the TSC timecounter. It does mean though that TSC calibration cannot use sbinuptime() to read the reference timecounter, as timehands are not yet set up. Reviewed by: kib, jhb MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D33209 --- sys/x86/include/clock.h | 1 + sys/x86/isa/clock.c | 2 ++ sys/x86/x86/tsc.c | 26 +++++++++++++++++--------- 3 files changed, 20 insertions(+), 9 deletions(-) diff --git a/sys/x86/include/clock.h b/sys/x86/include/clock.h index d492196bac85..83c8351ed31c 100644 --- a/sys/x86/include/clock.h +++ b/sys/x86/include/clock.h @@ -28,6 +28,7 @@ void i8254_init(void); void i8254_delay(int); void clock_init(void); void lapic_calibrate(void); +void tsc_calibrate(void); /* * Driver to clock driver interface. diff --git a/sys/x86/isa/clock.c b/sys/x86/isa/clock.c index 2eb1c343f4db..513174626892 100644 --- a/sys/x86/isa/clock.c +++ b/sys/x86/isa/clock.c @@ -413,6 +413,7 @@ cpu_initclocks(void) td = curthread; + tsc_calibrate(); lapic_calibrate_timer(); cpu_initclocks_bsp(); CPU_FOREACH(i) { @@ -428,6 +429,7 @@ cpu_initclocks(void) sched_unbind(td); thread_unlock(td); #else + tsc_calibrate(); lapic_calibrate_timer(); cpu_initclocks_bsp(); #endif diff --git a/sys/x86/x86/tsc.c b/sys/x86/x86/tsc.c index bba5e352b031..693e9a9f2c75 100644 --- a/sys/x86/x86/tsc.c +++ b/sys/x86/x86/tsc.c @@ -695,23 +695,27 @@ tsc_update_freq(uint64_t new_freq) /* * Perform late calibration of the TSC frequency once ACPI-based timecounters - * are available. + * are available. At this point timehands are not set up, so we read the + * highest-quality timecounter directly rather than using (s)binuptime(). */ -static void -tsc_calib(void *arg __unused) +void +tsc_calibrate(void) { - sbintime_t t_start, t_end; + struct timecounter *tc; uint64_t freq_khz, tsc_start, tsc_end; + u_int t_start, t_end; register_t flags; int cpu; if (tsc_disabled) return; + tc = atomic_load_ptr(&timecounter); + flags = intr_disable(); cpu = curcpu; tsc_start = rdtsc_ordered(); - t_start = sbinuptime(); + t_start = tc->tc_get_timecount(tc) & tc->tc_counter_mask; intr_restore(flags); DELAY(1000000); @@ -721,19 +725,23 @@ tsc_calib(void *arg __unused) flags = intr_disable(); tsc_end = rdtsc_ordered(); - t_end = sbinuptime(); + t_end = tc->tc_get_timecount(tc) & tc->tc_counter_mask; intr_restore(flags); sched_unbind(curthread); thread_unlock(curthread); - freq_khz = (SBT_1S / 1024) * (tsc_end - tsc_start) / (t_end - t_start); + if (t_end <= t_start) { + /* Assume that the counter has wrapped around at most once. */ + t_end += (uint64_t)tc->tc_counter_mask + 1; + } + + freq_khz = tc->tc_frequency * (tsc_end - tsc_start) / (t_end - t_start); - tsc_update_freq(freq_khz * 1024); + tsc_update_freq(freq_khz); tc_init(&tsc_timecounter); set_cputicker(rdtsc, tsc_freq, !tsc_is_invariant); } -SYSINIT(tsc_calib, SI_SUB_CLOCKS + 1, SI_ORDER_ANY, tsc_calib, NULL); void resume_TSC(void)