FreeBSD x86 vs x86-64 Questions about Security

[Peter Wemm]

On Friday 11 February 2005 07:45 am, Coleman Kane wrote:
> I believe there is still a large overlap of opcodes in both
> architectures. For instance, NOP is still 0x90. In addition I believe
> the default operand size (i.e.: no REX prefix) in PM64 is 32-bit,
> just like PM32 (protected mode 64 vs. 32 bit). This would lead me to
> believe that many stack-smashing exploits may still apply.
[..]
> So if OP is a CALL, or JMP or similar, you could use the same code,
> provided the destination address is the same. It will execute. Now if
> it succeeds, that is another matter which I have not looked into yet.
> I think the IA32 uses INT 0x80, while amd64 uses SYSCALL to handle
> syscalls.

I've been thinking about this.  If you have a 64 bit binary that has its 
stack clobbered with i386 shellcode and somehow get it to run, the 
syscall differences won't *necessarily* kill it.  If you had COMPAT_IA32 
compiled in, the shellcode could execute the syscall handler.

However..  the int 0x80 syscall code can only pass 32 bit args to the 
syscall handlers.  In other words, it can't express the address of 
things like "/bin/sh" if it is located on the stack.  Remember, the 64 
bit stack is very high in address space, so the 32 bit address 
truncated versions would give either an EFAULT or garbage when the 
kernel attempts to copyin the paths etc.

This doesn't meant that it is impossible though.  I can still imagine 
ways to exploit things that I won't go into much detail about.  But it 
would certainly add some resistance to i386 script-kiddy attacks.

Turning off COMPAT_IA32 will stop any possibility if shared i386+amd64 
shellcode from being able to make a syscall.  But there are other 
relatively easy ways to do a syscall still, especially in a dynamically 
linked binary.

Adding a check in the i386 syscall wrapper to make sure the syscall 
arrived from a 32 bit binary wouldn't solve any problems.  It would be 
simpler for the exploiters to just provide two syscall paths in common 
shellcode and runtime select them, or (much much easier) just include 
64 bit shellcode in the package.

Damn, we must get around to finishing PG_NX support and getting the 
signal trampoline off the stack so that we can set no-execute on it.   
(Again, not that it eliminates all possible attacks, but because it 
raises the bar a little)
-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5