[Bug 205458] 11.0-CURRENT/10-STABLE powerpc64: a PowerMac G5 specific sys/powerpc/ofw/ofw_machdep.c change for reliable PowerMac G5 booting (with lots of RAM)

[Mark Millard]

On 2016-Sep-18, at 4:46 AM, Jukka A. Ukkonen <jau789 at gmail.com> wrote:

> On 09/17/16 00:41, Mark Millard wrote:
>> Jukka A. Ukkonen jau789 at gmail.com wrote on Fri Sep 16 15:36:05 UTC 2016 :
>> 
>>> it seems my PowerMac G5 is perfectly happy with this...
>>> 
>>>        __asm __volatile("mfsprg0 %0\n\t"
>>>                         : "=&r"(ofw_sprg0_save)
>>>                         :
>>>                         );
>> 
>> for the content in ofw_sprg_prepare(void).
>> 
>> Interestingly ofw_sprg0_save's use to save and restore requires that the same processor be in use in the restore side, ofw_sprg_restore(void): SPRG0 content is processor specific.
>> 
>> If the sprg0 save-restore is even required is not obvious to me: being required would imply that that something else is adjusting it between the save and restore code executions.
>> 
>> The following is just in the form of another simplest-local-changes experiment/analysis pass, not a "how it should be coded for general FreeBSD use" form for __powerpc64__ or at least for for PowerMac/iMac/Xserve G5's specifically. .  .
>> 
>> Given what is reported above by Jukka it would appear that the above __asm lines possibly could be removed/disabled if ofw_sprg_restore(void) also had its __asm line removed/disabled: i.e.,
>> 
>>        __asm __volatile("mtsprg0 %0" :: "r"(ofw_sprg0_save));
>> 
>> would no log be in use to access ofw_sprg0_save.
>> 
>> Effectively overall for the intended __powerpc64__ PowerMac(/iMac?/Xserver?) experimental context ofw_sprg_prepare and ofw_sprg_restore are then no-ops.
>> 
>> In effect ofw_sprg0_save and ofmsr[1] to ofmsr[4] would not be in significant use. But. . .
>> 
>> OF_initial_setup(void *fdt_ptr, void *junk, int (*openfirm)(void *))
>> 
>> still would have __asm references to ofmsr[1] through ofmsr[4] where they are initialized.
>> 
>> 
>> You might want to try deleting/disabling the __asm statements in ofw_sprg_prepare(void) and in ofw_sprg_restore(void) at the same time to see if it makes any difference. If it does make a difference that would be interesting and important. Otherwise it helps identify some more (PowerMac/. . . context specific) unnecessary code.
> 
> 
> You asked it and I tested it. I must admit, though, that
> I had a kind of an Arthur Dent moment during the test.
> Remember the red button on the wall with the sign on
> the side telling: "Do not press this button!", and
> then Arthur being himself happily presses the button
> anyway just to see the warning sign to be illuminated.
> I was not quite so lucky.
> It seems touching the current sprg0 treatment is a
> guaranteed method to panic the system right at the
> beginning of autoconfigure.
> 
> --jau

Thanks for doing the experiment. In my view the result was interesting and good to know for sure.

The results suggest that Apple's Open Firmware changes sprg0 --at least sometimes-- so, then, after the Open Firmware call FreeBSD needs to re-establish its own sprg0 value from before the Open Firmware call.

An implication would be that it is possibly a matter of luck that the FreeBSD SLB fault handler is sometimes (only?) invoked while the FreeBSD sprg0 value is still in place when variations on my hack are in place. If it is only sometimes instead of always that the SLB handler has the FreeBSD sprg0 value to use, then register and memory trashing likely would still be possible, just less frequent (fewer types of contexts).

So it seems that the use of the mode:

usefdt=1

(currently "from the loader prompt") in order to avoid any calls into Open Firmware may be required to get full reliability. (See below.)

After checking in https://svnweb.freebsd.org/changeset/base/302214 back on 2016-Jun-26 Nathan had written:

> One thing it would be great to have some testing on after this change is 
> the FDT layer in loader. If you set usefdt=1 from the loader prompt, 
> loader will distill the OF device tree into an FDT and then stop Open 
> Firmware completely before transferring control to FreeBSD. This should 
> avoid any possible problems accessing Open Firmware from the kernel, as 
> well as making boot a little faster.

But I'd already temporarily lost access to the PowerMacs before he sent that note.

I intend to experiment with this once I have access again. (But I can not beat 11.0-RELEASE's schedule.) You have already suffered enough.

Your results suggest that I do such testing with the configuration that failed above.



Normally I'd only have subjected myself to the sprg0 testing activity that I asked you for --but I do not have access to the PowerMac G5s yet. Hopefully your information is useful to Nathan. But it may be too late for 11.0-RELEASE to get a matching patch from Nathan up front.

Sorry the testing lead to needing to recover from the bad build.

===
Mark Millard
markmi at dsl-only.net