Re: libc/libsys split coming soon

On Sat, Feb 03, 2024 at 10:15:09AM +0100, Mateusz Guzik wrote:
> On 2/2/24, Brooks Davis <brooks@freebsd.org> wrote:
> > TL;DR: The implementation of system calls is moving to a seperate
> > library (libsys).  No changes are required to existing software (except
> > to ensure that libsys is present when building custom disk images).
> >
> > Code: https://github.com/freebsd/freebsd-src/pull/908
> >
> > After nearly a decade of intermittent work, I'm about to land a series
> > of patches which moves system calls, vdso support, and libc's parsing of
> > the ELF auxiliary argument vector into a separate library (libsys).  I
> > plan to do this early next week (February 5th).
> >
> > This change serves three primary purposes:
> >   1. It's easier to completely replace system call implementations for
> >      tracing or compartmentalization purposes.
> >   2. It simplifies the implementation of restrictions on system calls such
> >      as those implemented by OpenBSD's msyscall(2)
> >      (https://man.openbsd.org/msyscall.2).
> >   3. It allows language runtimes to link with libsys for system call
> >      implementations without requiring libc.
> >
> > libsys is an auxiliary filter for libc.  This means that for any symbol
> > defined by both, the libsys version takes precedence at runtime.  For
> > system call implementations, libc contains empty stubs.  For others it
> > contains copies of the functions (this could be further refined at a
> > later date).  The statically linked libc contains the full
> > implementations so linking libsys is not required.
> >
> 
> Do I read it correctly that everything dynamically linked will also be
> linked to libsys, as in executing such a binary will now require
> loading one extra .so?
> 
> Binary startup is very slow, for example execve of a hello world
> binary in a Linux-based chroot on FreeBSD is faster by a factor of 2
> compared to a native one. As such perf-wise this looks like a step in
> the wrong direction.
> 
> Is there a problem making it so that libc ends up unchanged, but all
> the bits are available separately in libsys if one does not want libc?

Doing so fails to accomplish goal 1 for people who want to do it system
wide and does nothing for goal 2.  That being said, it would be pretty
easy to make this something we can toggle if we find it's an issue in
practice.

That being said, it seems like we should be spending time on rtld
improvements instead of optimizing for its current (seemingly worse
than necessary) performance.

-- Brooks