Failed attempt to boot a (non-debug) head -r339076 on an old PowerMac G5 "Quad Core" (built via devel/powerpc64-gcc): Waking up CPU 1

[Andreas Tobler]

On 13.10.18 21:18, Nathan Whitehorn wrote:
> 
> 
> On 10/9/18 2:07 PM, Andreas Tobler wrote:
>> On 09.10.18 22:40, Andreas Tobler wrote:
>>> On 09.10.18 22:35, Mark Millard via freebsd-ppc wrote:
>>>> [Reverting head -r334498 in my head -r339076 context was enough to get
>>>> the G5 so-called "Quad Core" to boot just fine as a variant of
>>>> -r339076 .]
>>>>
>>>> On 2018-Oct-9, at 12:54 PM, Mark Millard <marklmi at yahoo.com> wrote:
>>>>
>>>>> On 2018-Oct-9, at 8:20 AM, Mark Millard <marklmi at yahoo.com> wrote:
>>>>>
>>>>>> [The stable/head mix seems to be a wrong idea: 11.2 gets past
>>>>>> the SMP: messages just fine on the so-called G5 "Quad Core".]
>>>>>>
>>>>>> On 2018-Oct-8, at 5:14 PM, Mark Millard <marklmi at yahoo.com> wrote:
>>>>>>
>>>>>>> On 2018-Oct-8, at 1:27 PM, Justin Hibbits <chmeeedalf at
>>>>>>> gmail.com> wrote:
>>>>>>>
>>>>>>>>> . . .
>>>>>>>>
>>>>>>>> It would be helpful to know the last known-good SVN revision,
>>>>>>>> both for
>>>>>>>> Head and 11.x, as well as the oldest failing one.  Since my G5
>>>>>>>> bit the
>>>>>>>> dust, I can't check locally.
>>>>>>>
>>>>>> . . .
>>>>>
>>>>> There are examples of head's kernels that sometimes
>>>>> fail to get to the "SMP:" messages and sometimes work
>>>>> for getting there (and beyond). So:
>>>>>
>>>>> My reporting any example failure is a solid indicator
>>>>> of the "does not reach "SMP:" problem in that build.
>>>>> (All tries reached the waking message on at least cpu
>>>>> 1.)
>>>>>
>>>>> My reporting "worked" for a revision might be a
>>>>> misclassification. (This makes for a messier
>>>>> "binary-like search".)
>>>>>
>>>>> That said, the summary of the later detail is:
>>>>>
>>>>> head -r334494 kernel worked
>>>>> head -r334528 kernel failed
>>>>>
>>>>> (There is nothing between those for:
>>>>>
>>>>> https://artifact.ci.freebsd.org/snapshot/head/r*/powerpc/powerpc64/kernel.txz
>>>>>
>>>>>
>>>>> so getting a smaller range requires builds.
>>>>> I've not attempted that.)
>>>>>
>>>>> The only machine-dependent powerpc64 change between
>>>>> those 2 that I see is:
>>>>>
>>>>> Author: jhibbits
>>>>> Date: Fri Jun  1 21:37:20 2018
>>>>> New Revision: 334498
>>>>> URL:
>>>>> https://svnweb.freebsd.org/changeset/base/334498
>>>>>
>>>>>
>>>>> Log:
>>>>>     Increase powerpc64 KVA from ~7.25GB to 32GB
>>>>> . . .
>>>>>
>>>>> . . .
>>>>
>>>> In my -r339076 build context I reverted -r334498, did a
>>>> buildkernel, installed it, and rebooted into -r339076.
>>>>
>>>> The result booted just fine.
>>>>
>>>> It does appear that, for head, -r334498 makes the difference
>>>> for some reason.
>>>
>>> Unfortunately I have to confirm your findings.
>>
>> Mark, how much physical ram do you have? Can you adjust the
>> VM_MAX_KERNEL_ADDRESS just below the amount of RAM you have and see if
>> -CURRENT boots? Here it does, I have 14GB and I adjusted
>> VM_MAX_KERNEL_ADDRESS to 12GB. It is just a trial to find out what is
>> happening.
>>
>> Thanks,
>> Andreas
> 
> My guess is that the combination of large RAM (so big DMAP) and large
> KVA is causing SLB overflows: a combination of these above 16 GB will
> make this rate non-zero. These are fine while in the kernel or userspace
> -- they get handled transparently -- but could be fatal during a call to
> Open Firmware, which, on Apple hardware, operates with the MMU on since
> OF doesn't have the appropriate trap handlers to handle DSE/ISE. This
> would also explain why the issue only affects Apple hardware. There
> would be a few ways to fix it:
> 1. Distill the OF tree to an FDT before starting the kernel (usefdt=1
> option in the loader). There are still issues with this on some Apple
> hardware.

What are the issues and how can we/I help to solve them?
On my DP 7,2 and my Quad 11,2 I run into theses issues I guess.
On the DP I hang before Timecounter "timebase"
On the Quad I ran into the issue mentioned on irc. The patch is applied 
correctly. The backtrace shows 'bus_try_attach_pending'

Thanks,
Andreas

> 2. Right before calling into OF, and after any possibility of  prefault
> all the possible SLB entries corresponding to any 32-bit address (all
> the ones OF could possibly use) to eliminate the possibility of
> in-firmware SLB faults.
> 3. Leave the kernel SLB fault handlers (+ associated metadata at low
> addresses) in place during OF calls.
> -Nathan
> 
> _______________________________________________
> freebsd-ppc at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ppc
> To unsubscribe, send any mail to "freebsd-ppc-unsubscribe at freebsd.org"
>