Question on "Map the entire KVA range into the SLB. We must not fault there" vs. the modern VM_MAX_KERNEL_ADDRESS

[Justin Hibbits]

On Mon, 18 Feb 2019 18:58:43 -0800
Mark Millard <marklmi at yahoo.com> wrote:

> It takes me a bit to provide context for the question . . .
> 
> I'll first note:
> 
> #define VM_MAX_SAFE_KERNEL_ADDRESS      VM_MAX_KERNEL_ADDRESS
> 
> moea64_mid_bootstrap has:
> 
>         virtual_avail = VM_MIN_KERNEL_ADDRESS;
>         virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS;
> 
>         /*
>          * Map the entire KVA range into the SLB. We must not fault
> there. */     
>         #ifdef __powerpc64__
>         for (va = virtual_avail; va < virtual_end; va +=
> SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0);
>         #endif
> 
> but the prefaulting is based on:
> 
> #define
> PCPU_MD_AIM64_FIELDS                                            \
> struct slb      slb[64];                                        \
> 
> where the slb is based on the upper 36 bits of the passed-in va values
> (effective address value in powerpc terms).
> 
> The address range now being covered is:
> 
> #define VM_MIN_KERNEL_ADDRESS           0xe000000000000000
> #define VM_MAX_KERNEL_ADDRESS           0xe0000007ffffffff
> 
> So for the upper 36 bits:
> 
> 0xe000_0000_0
> 0xe000_0007_f
> 
> and the range 0x00 through 0x7f has 128 possibilities.
> 
> So more than the slb can track. (Also: slb[USER_SLB_SLOT]
> is avoided so really 63 of the 64 are available.)
> 
> Later (larger) va values are the replacing earlier
> (smaller) va values via slb_insert_kernel usage that
> in turn involves random replacements via (n_slbs
> being 64 in the G5 context):
> 
>         i = mftb() % n_slbs;
>         if (i == USER_SLB_SLOT)
>                         i = (i+1) % n_slbs;
> 
> So apparently only the last 63 from:
> 
>         for (va = virtual_avail; va < virtual_end; va +=
> SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0);
> 
> are known to still be present in the slb.
> 
> 
> 
> So the question . . .
> 
> How can:
> 
> "Map the entire KVA range into the SLB. We must not fault there"
> 
> be true? Is it really a requirement? If it is, then something seems
> to be wrong as far as handling the modern VM_MAX_KERNEL_ADDRESS value
> goes.
> 
> 
> The old VM_MAX_KERNEL_ADDRESS value would have 0x00 through 0x1c
> and so would fit this particular constraint. The old value
> 0xe0000001c7ffffff looks like something was being avoided
> (why not 0xe0000001ffffffff ?). But I've not found what or why.

If this were cause for concern I'd likely see it on my POWER9, which
actually only has 32 SLB entries, 4 of which are pinned, the remaining
28 being FIFO.  Much less than 64 on the G5.

- Justin