My PowerMac G5's no longer crash at boot: PowerMac G5 specific ofwcall changes with justifying evidence [Darwin 32-bit ABI violation]

[Mark Millard]

I started to look into the powerpc/GENERIC (32 bit) PowerMac requirements and what they imply for ofwcall since I intend on adding the %r1 and %r3 validation code.

But I discovered that the ofwcall64.S code is not really 32-bit Darwin PowerPC ABI compliant.

I am using Mac OS X Internals, A Systems Approach, by Amit Singh for documentation of the Apple PowerPC 32 bit Darwin ABI. Technically it documents just the Darwin/Mac OS X 32 bit PowerPC ABI. I'm not aware of anything that explicitly says that openfirmware on PowerMac's is an exact match. But I'd expect a match. (I also looked at The PowerPC Compiler Writer's Guide, although it is not Apple specific.)

The below is based on Internals book's material starting from the page labeled 231 and using the diagram on the page labeled 232 and 235 mostly.

The page labeled 233 says: "A 32-bit Darwin ABI stack frame is 16-byte aligned."


I've choose to present a simple/non-optimized translation of what the 32-bit Darwin PowerPC ABI would imply, translated into simple code that conceptually might go in ofwcall. But Commented Code indicating the ABI points.

I must also note that the material is based strictly on the 32-bit Darwin PowerPC ABI involved and what it allows/requires --and not on any extra knowledge of what openfirmware really does that may make some points irrelevant.

And ofwcall's fake Darwin ABI frame is presented rather fully in its details because openfirmware (the to-be-called routine) can use some of that space.

[Side note: it appears that any exceptions/interrupts that might use the same stack must avoid a 224 (220?) byte "red zone" area above %r1 because some leaf  functions are allowed to use it without forming a activation frame (well, 220 bytes but with 16-byte alignment pad making it 224). AIX would have that be 220 --as far as I can tell because it does not have the same 16-byte alignment rule.]

        .data
        .align  16 /* 32-bit Darwin PowerPC ABI requires 16-byte (quad-word) stack frame alignment */.
ofwstk:
        .space  OFWSTKSZ /* Should keep to 16-byte multiples */
        .align  4 /* back to normal */
...
        mr      %r18,%r1
        lis     %r1,(ofwstk+OFWSTKSZ-112)@ha 
        addi    %r1,%r1,(ofwstk+OFWSTKSZ-112)@l /* Needs to produce a 16-byte (quad-word) aligned %r1 value */
                                                /* with room for the following... */

        /* These are not from the 32-bit Darwin PowerPC ABI: They are after the fake ofwcall ABI frame. */
        /* %r1+80 is also 16-byte aligned. */
        std     %r18,80+0(%r1)  /* Save FreeBSD stack pointer */
        std     %r2,80+8(%r1)   /* Save FreeBSD TOC */
        std     %r14,80+16(%r1) /* Save FreeBSD MSR */
        /* Bytes at/after %r1+80+24 not used. The above are not from the 32-bit Darwin ABI. */

        /* The 32-bit Darwin PowerPC ABI ofwcall stack structure follows: */

        addi    %r19,%r1,80
        stw     %r19,0(%r1)     /* backchain (non-negative offsets from there are junk here
                                 * relative to the 32-bit Darwin PowerPC ABI but negative 
                                 * offsets from there can be okay if small enough to be 
                                 * in ofwstk from %r1 to ofwstk+OFWSTKSZ-112+79 inclusive)
                                 */
        li      %r19,0
        stw     %r19,4(%r1)     /* space for openfirmware (the to-be-called routine) to store CR */
        stw     %r19,8(%r1)     /* space for openfirmware (the to-be-called routine) to store LR */
        stw     %r19,12(%r1)    /* Darwin linkage reserved space */
        stw     %r19,16(%r1)    /* more Darwin linkage reserved space */
        stw     %r19,20(%r1)    /* space for TOC (despite 32-bit Darwin ABI not using TOC: it has the space) */
        /* End of 24 byte linkage area */
        /* Empty local stack area. If non-empty: 16-bit aligned? Prefix padding shown in book that produces such alignment. */
        /* Parameter area required because a function (openfirmware) is called. Must be double word aligned. */
        stw     %r19,24(%r1)    /* Parameter area with the only argument's slot: space reserved despite register argument usage */
        stw     %r19,28(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,32(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,36(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,40(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,44(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,48(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        stw     %r19,52(%r1)    /* Parameter area is at least 8 words and the space is reserved despite register argument usage */
        /* NOTE: openfirmware (the to-be-called routine) is allowed to use the above parameter area! 
         *       This is true of the 32-bit Darwin PowerPC ABI despite register argument passing!
         *       Typical use is to temporarily free up a register.
         */
        /* End of 32 byte parameter area */
        stw     %r19,56(%r1)    /* padding to have 16 byte stack alignment */
        stw     %r19,60(%r1)    /* padding to have 16 byte stack alignment */
        /* End of 8-byte sized padding to cause 16-byte alignment */
        /* GPR save area: 8-byte prefix padding to force 16-byte-multiple for size(?) is shown in figures. */
        stw     %r19,64(%r1)    /* padding to have 16 byte size for GPR area */
        stw     %r19,68(%r1)    /* padding to have 16 byte size for GRP area */
        stw     %r19,72(%r1)    /* %r30 always stored for GPR save/restore (fake value here) */
        stw     %r19,76(%r1)    /* %r31 always stored for GPR save/restore (fake value here) */
        /* End of 16-byte GRP save area */
        /* Empty FPR save area but must be double word aligned */
        /* +80 is again 16-byte aligned. The FreeBSD extras are not part of the 32-bit Darwin ABI. */

Note: %r30 is used as the frame pointer register. %r31 as the PIC-offset table register.

Now compare that to the existing sort of code (but with the %r14, %r18, and %r19 usage to match the above):

        .data
        .align  4
ofwstk:
        .space  OFWSTKSZ
...
        mr      %r18,%r1
        lis     %r1,(ofwstk+OFWSTKSZ-32)@ha
        addi    %r1,%r1,(ofwstk+OFWSTKSZ-32)@l
        std     %r18,8(%r1)     /* Save FreeBSD stack pointer */
        std     %r2,16(%r1)     /* Save FreeBSD TOC */
        std     %r14,24(%r1)    /* Save FreeBSD MSR */
        li      %r19,0
        stw     %r19,4(%r1)
        stw     %r19,0(%r1)

So what I get from the comparison is that:

std     %r18,8(%r1) (FreeBSD stack pointer)
is using space that openfirmware is allowed to put the LR in and also some reserved space.

std     %r2,16(%r1) (FreeBSD TOC)
is using reserved space and the TOC space from the Darwin ABI.

std     %r14,24(%r1) (FreeBSD MSR)
is using the parameter space that openfirmware is allowed to put to other uses.

These risk openfirmware replacing part of the FreeBSD stack pointer and the FreeBSD MSR.

Also:

        stw     %r19,4(%r1)
        stw     %r19,0(%r1)

only covers the backchain and space where openfirmware could put CR values. It does not span where openfirmware could put LR values or the parmeter area that openfirmware is allowed to use.


As an example of the parameter area criteria and the size of the stack frame from the book: the pages labeled 234-235 has f3() that calls g(); (no arguments). For this it says...

"f3 calls a function that takes no arguments. Nevertheless, this introduces a parameter area on f3's stack. A parameter area is at least 8 words (32 bytes) in size. f3's stack is 80 bytes."

Of course openfirmware does take a 32-pointer as an argument but the 8 words covers that and the stack frame size does not change compared to the book's f3 example.

And, of course, the three 64-bit values stored in ofwstk below all this are in addition to the above.

Quoting various statements about the "Parameter area" (for function f1 calling f2):

0) "The Parameter area must be large enough to hold the largest parameter list of all the functions that f1 calls."

(Material about passing arguments in registers. Then...) 

1) "However f1 must reserve space for all arguments of f2 in any case --even if it is able to pass all arguments in registers. f2 is free to use f1's parameter area for storing arguments if it wants to free up the corresponding registers for other use. Thus, in a subroutine call, the caller sets up a parameter area in its own stack portion, and the callee can access the caller's parameter area for loading or storing arguments."


===
Mark Millard
markmi at dsl-only.net