r344744 ppc64 still needs kern.smp.disabled=1

[Mark Millard]

On 2019-Mar-4, at 10:40, Dennis Clarke <dclarke at blastwave.org> wrote:

> Merely a note that r344744 hangs at starting CPU3 message and then the
> fans roar as usual. Of course usefdt=1 is being used here also.
> 
> If there is a svn co that I can do from some other hackary location :-)
> then I would be happy to build and test however current head doesn't
> allow for all cores to come online.
> 

Justin H. has said he will work on the issue. My hack
to head -r344018 is:

Index: /usr/src/sys/powerpc/aim/slb.c
===================================================================
--- /usr/src/sys/powerpc/aim/slb.c      (revision 344018)
+++ /usr/src/sys/powerpc/aim/slb.c      (working copy)
@@ -464,6 +464,28 @@
        critical_exit();
 }
 
+void hack_into_slb_if_needed(void* vap); // HACK!!!
+void hack_into_slb_if_needed(void* vap) // HACK!!!
+{ // HACK!!!
+       struct slb *cache= PCPU_GET(aim.slb);
+       vm_offset_t va=    (vm_offset_t)vap;
+       uint64_t    slbv=  kernel_va_to_slbv(va);
+       uint64_t    esid=  va>>ADDR_SR_SHFT;
+       uint64_t    slbe=  (esid<<SLBE_ESID_SHIFT) | SLBE_VALID;
+       int i;
+
+       for (i = 0; i < n_slbs; i++) {
+               if (i == USER_SLB_SLOT)
+                       continue;
+               if (cache[i].slbe == (slbe | i))
+                       break;
+       }
+
+       if (i==n_slbs)
+               slb_insert_kernel(slbe,slbv);
+} // HACK!!!
+
+
 void
 slb_insert_user(pmap_t pm, struct slb *slb)
 {
Index: /usr/src/sys/powerpc/aim/mp_cpudep.c
===================================================================
--- /usr/src/sys/powerpc/aim/mp_cpudep.c        (revision 344018)
+++ /usr/src/sys/powerpc/aim/mp_cpudep.c        (working copy)
@@ -107,6 +107,8 @@
        powerpc_sync();
 }
 
+extern void hack_into_slb_if_needed(void* vap); // HACK!!!
+
 uintptr_t
 cpudep_ap_bootstrap(void)
 {
@@ -116,12 +118,17 @@
        mtmsr(msr);
 
        pcpup->pc_curthread = pcpup->pc_idlethread;
+
 #ifdef __powerpc64__
        __asm __volatile("mr 13,%0" :: "r"(pcpup->pc_curthread));
 #else
        __asm __volatile("mr 2,%0" :: "r"(pcpup->pc_curthread));
 #endif
+
        pcpup->pc_curpcb = pcpup->pc_curthread->td_pcb;
+
+       hack_into_slb_if_needed(pcpup->pc_curpcb); // HACK!!!
+
        sp = pcpup->pc_curpcb->pcb_sp;
 
        return (sp);

But it is still possible for a replacement of the slb entry
after slb_insert_kernel(slbe,slbv) but before
pcpup->pc_curpcb->pcb_sp . Still, the hack makes booting
far more reliable than the original code.

I no longer revert the VM_MAX_KERNEL_ADDRESS value and instead
use the above.



I also have a hack for the stuck-sleeping problems seen for
buf*daeomon* threads, pmac_thermal, and the like. I've been
able to run a more than 24 hour test with no sleep hang-ups,
where the test included an over 7 hour buildworld buildkernel.
This was with a debug version of the kernel. I've not tested
non-debug yet for the stuck-sleeping issue. I've also observed
no problems across a fair number of reboots. But, like the
racy slb replacement issue, what I've done may not guarantee
to never fail: a constant might have to be increased, for
example. (Several somewhat smaller constant values were
observed to be not sufficient when tested.)

There is a significant effort going on by a few folks that
are looking at improving the official code that is related
to the stuck-sleeping issue, for all architectures.
Hopefully, multi-socket PowerMac G5's will not need any hacks
once they are done.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)