two proposed linuxulator fixes + ptrace

[Konstantin Belousov]

On Thu, Jun 14, 2018 at 11:31:20AM +0300, Yanko Yankulov wrote:
>  Hi all,
> 
> I stumbled on two problems trying to run a proprietary java/native Linux
> app on FreeBSD. I managed to get it working with two small changes to the
> Linux compatibility code. Also in order to pinpoint the issues I got the
> ptrace system working for Linux apps, enabling truss & gdb.
> 
> This is all on recent CURRENT.
> 
> The patches work for me, but I am absolutely sure there is better way to do
> this, especially for the ptrace part.
> 
> The first issue was 100% CPU usage on start with no progress. Turned out
> that the app was expecting TracerPid field in its /proc/pid/status, so just
> adding  "sbuf_printf(sb, "TracerPid:\t%d\n",     0);" in
> linprocfs_doprocstatus solved it.
> 
> The second was random segfaults in the Java code. Traced it to mangled r10
> in the sigreturn path. Just preserving the r10 when returning from
> sigreturn resolves it.
> 
> --- a/sys/amd64/linux/linux_sysvec.c
> +++ b/sys/amd64/linux/linux_sysvec.c
> @@ -228,8 +228,9 @@ linux_set_syscall_retval(struct thread *td, int error)
>          * the syscall.  So, do not clobber %rdx and %r10.
>          */
>         td->td_retval[1] = frame->tf_rdx;
> -       frame->tf_r10 = frame->tf_rcx;
> -
> +       if( td->td_sa.code != LINUX_SYS_linux_rt_sigreturn )
> +               frame->tf_r10 = frame->tf_rcx;
> +
>         cpu_set_syscall_retval(td, error);
> 
>          /* Restore all registers. */
> 
> 
> So this two fixes solved my issues and was able to run/use the problematic
> application.
> 
> The ptrace code is lot more messy, and I am really not happy with it, but I
> didn't have the time to figure out a cleaner  solution. Attaching it
> though, as it at least might help someone to devise a better fix.

Good work.

For the ptrace patch, I suggest you to put it on https://reviews.freebsd.org
and set at least me (kib), jhb and dchagin as reviewers.  I will handle it.

For the patch 1, TracePid, can you explain what is the meaning of the
pid reported ?

For the patch 3, %r10 preservation for linux_rt_sigreturn, shouldn't the
same handling applied to non-rt signal return ? And in fact, shouldn't
it be done based on the return code instead of the syscall number ? Look
at the amd64/amd64/vm_machdep.c:cpu_set_syscall_retval(), where I think
EJUSTRETURN case is used by linux sigreturns.