git: 248f0cabca75 - main - make maximum interrupt number tunable on ARM, ARM64, MIPS, and RISC-V

[Andriy Gapon]

On 2021-01-19 21:42, Michal Meloun wrote:
> 
> 
> On 19.01.2021 19:41, John Baldwin wrote:
>> On 1/19/21 8:43 AM, Jessica Clarke wrote:
>>> On 19 Jan 2021, at 00:36, Oleksandr Tymoshenko <gonzo at FreeBSD.org>
>>> wrote:
>>>> @@ -142,21 +143,15 @@ static bool irq_assign_cpu = false;
>>>> #endif
>>>> #endif
>>>>
>>>> -/*
>>>> - * - 2 counters for each I/O interrupt.
>>>> - * - MAXCPU counters for each IPI counters for SMP.
>>>> - */
>>>> -#ifdef SMP
>>>> -#define INTRCNT_COUNT   (NIRQ * 2 + INTR_IPI_COUNT * MAXCPU)
>>>> -#else
>>>> -#define INTRCNT_COUNT   (NIRQ * 2)
>>>> -#endif
>>>> +int intr_nirq = NIRQ;
>>>> +SYSCTL_UINT(_machdep, OID_AUTO, nirq, CTLFLAG_RDTUN, &intr_nirq, 0,
>>>> +    "Number of IRQs");
>>>
>>> Unsigned integer, given the SYSCTL_UINT?
>>>
>>> What's stopping us from dynamically resizing rather than forcing the
>>> user to use a tunable (which also means that, in practice, the limit
>>> will remain unchanged, because you want GENERIC kernels to work out of
>>> the box)?
>>
>> For x86 (and probably the same here), the biggest headache is that there
>> is a dynamically allocated array indexed by IRQ value at interrupt
>> invocation time.  Currently it uses no locks since the value of the
>> pointer is set during boot time and then never changes.  Making it
>> dynamic would require allowing that pointer to change and dealing with
>> what that entails in a very hot path.  Generally speaking, the majority
>> of interrupts used in a given system are allocated during boot and not
>> by hotplug devices added post-boot, so there is also not a lot of utility
>> in having this run-time tunable vs a boot-time tunable.
>>
> To be exact, the main headaches are  intrcnt and intrnames arrays and
> their relationship to the sysctl interface. These fields are used
> without locking, they do not have the ability to mark interrupt at given
> index as unused. This is the real reason why we can't unload a driver
> that acts as an interrupt controller (e.g. USB parallel port with the
> possibility of interrupts on each pin).
> Everything else in subr_intr.c should be prepared for dynamic
> allocation/delete of interrupt sources.

x86 does not use subr_intr / intrng though.

I've been playing with an idea of using two-level interrupt "arrays" for
things like interrupt counts and descriptiopns.
That is, having fixed size chunks and a dynamically growing array of
pointers to the chunks with a restriction that once a chunk is allocated
it is never deallocated.  So a pointer to an entry stored in such a
chunk would remain stable.

I also tried to use vmem(9) to keep track of allocated interrupts
(interrupt sources) so that they could be dynamically created and
removed.  I think that it is a feasible idea, but I got lost in
implementation details.


-- 
Andriy Gapon