git: 94df301ac27e - stable/13 - Wait longer for a previous IPI to be sent
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Wed, 02 Mar 2022 21:57:15 UTC
The branch stable/13 has been updated by vangyzen: URL: https://cgit.FreeBSD.org/src/commit/?id=94df301ac27e7df785d1d581a36dc14d456436c2 commit 94df301ac27e7df785d1d581a36dc14d456436c2 Author: Eric van Gyzen <vangyzen@FreeBSD.org> AuthorDate: 2021-04-26 15:01:17 +0000 Commit: Eric van Gyzen <vangyzen@FreeBSD.org> CommitDate: 2022-03-02 21:56:30 +0000 Wait longer for a previous IPI to be sent When sending an IPI, if a previous IPI is still pending delivery, native_lapic_ipi_vectored() waits for the previous IPI to be sent. We've seen a few inexplicable panics with the current timeout of 50 ms. Increase the timeout to 1 second and make it tunable. No hardware specification mentions a timeout in this case; I checked the Intel SDM, Intel MP spec, and Intel x2APIC spec. Linux and illumos wait forever. In Linux, see __default_send_IPI_shortcut() in arch/x86/kernel/apic/ipi.c. In illumos, see apic_send_ipi() in usr/src/uts/i86pc/io/pcplusmp/apic_common.c. However, misbehaving hardware could hang the system if we wait forever. Reviewed by: mav kib MFC after: 1 week Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D29942 (cherry picked from commit 2f32a971b7f936170b4d61318238e6cf89f198b5) --- sys/x86/x86/local_apic.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c index 59b5cb600725..bbe3a63157d4 100644 --- a/sys/x86/x86/local_apic.c +++ b/sys/x86/x86/local_apic.c @@ -204,6 +204,7 @@ static u_long lapic_timer_divisor, count_freq; static struct eventtimer lapic_et; #ifdef SMP static uint64_t lapic_ipi_wait_mult; +static int __read_mostly lapic_ds_idle_timeout = 1000000; #endif unsigned int max_apic_id; @@ -214,6 +215,11 @@ SYSCTL_INT(_hw_apic, OID_AUTO, eoi_suppression, CTLFLAG_RD, &lapic_eoi_suppression, 0, ""); SYSCTL_INT(_hw_apic, OID_AUTO, timer_tsc_deadline, CTLFLAG_RD, &lapic_timer_tsc_deadline, 0, ""); +#ifdef SMP +SYSCTL_INT(_hw_apic, OID_AUTO, ds_idle_timeout, CTLFLAG_RWTUN, + &lapic_ds_idle_timeout, 0, + "timeout (in us) for APIC Delivery Status to become Idle (xAPIC only)"); +#endif static void lapic_calibrate_initcount(struct lapic *la); @@ -2088,7 +2094,6 @@ native_lapic_ipi_raw(register_t icrlo, u_int dest) } } -#define BEFORE_SPIN 50000 #ifdef DETECT_DEADLOCK #define AFTER_SPIN 50 #endif @@ -2135,7 +2140,7 @@ native_lapic_ipi_vectored(u_int vector, int dest) icrlo |= APIC_DESTMODE_PHY | APIC_TRIGMOD_EDGE | APIC_LEVEL_ASSERT; /* Wait for an earlier IPI to finish. */ - if (!lapic_ipi_wait(BEFORE_SPIN)) { + if (!lapic_ipi_wait(lapic_ds_idle_timeout)) { if (KERNEL_PANICKED()) return; else