What does the FreeBSD/i386 ABI say about stack alignment?

[Warner Losh]

On 01/14/2011 09:42, Warner Losh wrote:
> On 01/14/2011 01:28, Kostik Belousov wrote:
>> On Thu, Jan 13, 2011 at 04:34:22PM -0700, Warner Losh wrote:
>>> On 01/13/2011 13:28, Kostik Belousov wrote:
>>>> On Thu, Jan 13, 2011 at 12:19:00PM -0500, Ryan Stone wrote:
>>>>> I've been trying to get an application compiled with gcc 4.5.1 
>>>>> running
>>>>> on FreeBSD 8.1, but it's been crashing during startup with a SIGBUS.
>>>>> It turns out that the problem is that gcc is issuing SSE
>>>>> instructions(in my case, a movdqa) that assume that the stack will be
>>>>> aligned to a 16-byte boundary.  It seems that Linux/i386 guarantees
>>>>> this, and I worry that gcc has extended this assumption to all i386
>>>>> architectures.  I'm assuming that FreeBSD doesn't make any such
>>>>> promises based on the fact that I'm getting crashes.
>>>>>
>>>>> There does seem to be a flag (-mstackrealign) that you can set to
>>>>> force gcc to align the stack to what it wants, but that pessimizes 
>>>>> the
>>>>> generated code a bit.  Some googling would seem to indicate that
>>>>> -mpreferred-stack-boundary won't always handle this problem 
>>>>> correctly.
>>>>>
>>>>> Any ideas?  My inclination, at least for our local source tree 
>>>>> here at
>>>>> $WORK, would be to accommodate gcc and guarantee the stack alignment
>>>>> that it wants rather than pessimize our application.  It seems we 
>>>>> have
>>>>> an old local patch/hack in our FreeBSD 6.1 tree(apparently based on
>>>>> this:
>>>>> http://www.freebsd.org/cgi/getmsg.cgi?fetch=438552+0+/usr/local/www/db/text/2000/freebsd-current/20000507.freebsd-current). 
>>>>>
>>>>>   I believe that this patch is the reason why we haven't seen the
>>>>> problem when running on 6.1, but the patch doesn't seem to work
>>>>> anymore on 8.1.
>>>> Look at lib/csu/i386-elf/crt1_s.S, we align stack on startup.
>>>> My understanding is that the requirement is (%esp&   0xf) == 0 just 
>>>> before
>>>> the call to the function. And we are off by 4 (this is my fault).
>>>>
>>>> Please give this a try.
>>>>
>>>> diff --git a/lib/csu/i386-elf/crt1_s.S b/lib/csu/i386-elf/crt1_s.S
>>>> index d7ed0a2..17ac0e3 100644
>>>> --- a/lib/csu/i386-elf/crt1_s.S
>>>> +++ b/lib/csu/i386-elf/crt1_s.S
>>>> @@ -42,6 +42,7 @@ _start:
>>>>       .cfi_def_cfa_register %ebp
>>>>       andl    $0xfffffff0,%esp # align stack
>>>>       leal    8(%ebp),%eax
>>>> +    subl    $4,%esp
>>>>       pushl    %eax        # argv
>>>>       pushl    4(%ebp)        # argc
>>>>       pushl    %edx        # rtld cleanup
>>> I'm seeing weird core dumps for ssh and friends on i386 on stable/8 
>>> from
>>> a few days ago.  Could that be related?
>> Few days ago ? It was in the tree for probably one year.
>> I very much doubt it, but cannot say anything until you show the 
>> backtrace.
>>
>> Our in-tree gcc masks this by typically doing stack realignment on the
>> entry into the main().
>
> I tend to think you are right...  The backtrace doesn't have an 
> aligned instruction to worry about.  I'll rebuild to make sure it is 
> all sane...

Finally got a chance to test the rebuild.  Once I rebuilt, everything 
worked fine.  I rebuild exactly the same sources I started with, so this 
wasn't my sshd dumping core problem.  I believe I narrowed the problem 
down to a badly installed libc over NFS...

Warner