misc/177624: Swapcontext can get compiled incorrectly

[Bruce Evans]

On Thu, 4 Apr 2013, Brian Demsky wrote:

>> Description:
> Here is the code for swap context:
>
> int
> swapcontext(ucontext_t *oucp, const ucontext_t *ucp)
> {
>      int ret;
>
>      if ((oucp == NULL) || (ucp == NULL)) {
>              errno = EINVAL;
>              return (-1);
>      }
>      oucp->uc_flags &= ~UCF_SWAPPED;
>      ret = getcontext(oucp);
>      if ((ret == 0) && !(oucp->uc_flags & UCF_SWAPPED)) {
>              oucp->uc_flags |= UCF_SWAPPED;
>              ret = setcontext(ucp);
>      }
>      return (ret);
> }

> On the OS X port of libc in Mac OSX 10.7.5, this gets compiled as:

> ...
> 0x00007fff901e870b <swapcontext+89>:    pop    %rbx
> 0x00007fff901e870c <swapcontext+90>:    pop    %r14
> 0x00007fff901e870e <swapcontext+92>:    jmpq   0x7fff90262855 <setcontext>
>
> The problem is that rbx is callee saved by compiled version of swapcontext and then reused before getcontext is called.  Getcontext then stores the wrong value for rbx and setcontext later restores the wrong value for rbx.  If the caller had any value in rbx, it has been trashed at this point.

Later you wrote:

> The analysis is a little wrong about the problem.  Ultimately, the tail call to set context trashes the copies of bx and r14 on the stack….

The bug seems to be in setcontext().  It must preserve the callee-saved
registers, not restore them.  This would happen automatically if more
were written in C.  But setcontext() can't be written entirely in C,
since it must save all callee-saved registers including ones not used
and therefore not normally saved by any C function that it might be in,
and possibly also including callee-saved registers for nonstandard or
non-C ABIs.  In FreeBSD, it is apparently always a syscall.

In FreeBSD, this bug doesn't occur on at least amd64 or i386 because the
C version of swapcontext() has never been used on these arches.
swapcontext() is a syscall too.

If setcontext() is a syscall, then it has a minor problem even knowing
what the ABI's callee-saved registers are.  At least the FreeBSD amd64
version doesn't know anything about this.  It uses much the same code
as for asynchronous signal handling, so it just restores all registers,
including scratch ones that don't need to be preserved.  It even
restores the return register to the trap frame, although it can't
return this to userland.  This can probably be fixed a library wrapper.

In FreeBSD on amd64, getcontext(), setcontext() and swapcontext() are
all syscalls, but their documenation is misplaced in a section 3 man
page.  swapcontext() is misplaced together with makecontext(), which
actually is library function.  Oops, not quite.  getcontext() is
actually a small wrapper around an undocumented setcontext() syscall
(this is needed to adjust the instruction pointer register).  Only
makecontext() is in C, and not a wrapper.  I don't count the wrappers
that just make a syscall as library functions, since the corresponding
syscalls can be made easily without using the C library and this should
be documented.

Bruce