Re: The Case for Rust (in the base system)

On Sat, Jan 20, 2024, 7:20 PM Alan Somers <asomers@freebsd.org> wrote:

> On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI <junchoon@dec.sakura.ne.jp>
> wrote:
> >
> > On Sat, 20 Jan 2024 15:31:23 -0700
> > Warner Losh <imp@bsdimp.com> wrote:
> >
> > > On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
> wigneddoom@yandex.ru>
> > > wrote:
> > >
> > > > What about external dependencies?
> > > >
> > > >
> https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
> > > >
> https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
> > > >
> > > > Is there any plan for which crates we should take into the base
> system?
> > > >
> > > > We have had C++ in base for many years, but I don’t see any good
> libraries
> > > > for CLI, logging, JSON, etc.
> > > >
> > > >
> > > >
> https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
> > > >
> > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V,
> > > > Power? Should we rewrite devd?
> > > >
> > > > I think we need to start by providing official repositories (e.g
> > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
> > > > for different languages that include stable bindings to the system
> API:
> > > > - sysctl
> > > > - libgeom
> > > > - libifconfig
> > > > - netgraph
> > > > - jail
> > > > - etc.
> > > >
> > > > So that it’s not just some anonymous on crates.io that represents
> these
> > > > bindings, but our community.
> > > > Officially, with support for a stable ABI for releases, security
> patches,
> > > > etc.
> > > >
> > > > After this, it will be possible to think about which components to
> include
> > > > in the base system.
> > > >
> > > > I would be glad to see a more modern language than C in the
> database, but
> > > > I’m afraid that it will be like with C++,
> > > > that we will get a couple of daemons and utilities and that’s all.
> > > >
> > >
> > > These are all good questions that need good answers, though
> necessarily to
> > > get started.
> > >
> > > But the other question that occured to me after my last posting was
> "What
> > > about build integration?"
> > > How much of the rust automation do we take in vs how much do we drive
> from
> > > a future bsd.rust.mk.
> > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
> need
> > > one for what we traditionally
> > > think of as libraries (which may or may not map 1:1 onto crates: we
> could
> > > have c callable libraries
> > > written in rust in the future, for example) and one for binaries.
> > > Initially, though, if we go with the
> > > 'make rust tests possible' then we'd likely need the appropriate
> packages
> > > installed for whatever
> > > dependencies we'd have in the tests. This would give us a taste for
> what
> > > we'd need to do for
> > > base, I'd think. Once we had that notion, I can easily see there
> needing to
> > > be some sort of
> > > rust bindings for ATF/kyua as one of the first libraries / crates that
> > > would test that aspect of
> > > the build system. That all would be up to the people writing the tests
> in
> > > rust, I'd imagine.
> > >
> > > While I could jot out the basics of this integration (so one could
> just add
> > > the rust
> > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever and
> then
> > > it would build
> > > if rust is enabled, and would emit a warning it was skipped because
> rust
> > > was disabled).
> > > We'd find out if this is workable or not and iterate from there. But
> that
> > > would also require
> > > active participation from the rust advocates to make it a reality: I
> can
> > > put together the
> > > build infrastructure for the disabled case, but likely can't on my own
> do
> > > the rust enabled
> > > case. I'd be happy to work with someone to do that, but I'm not going
> to be
> > > able to do
> > > that myself: my need for rust is slight, my knowledge of rust is weak,
> etc.
> > > Working with
> > > someone (or ideally several someones), though it could become reality.
> So
> > > please contact
> > > me if you'd like to work on this.
> > >
> > > Warner
> >
> > One way to go could be moving programs rewritten with rust to ports.
> > There are some programs (not in rust, though) moved to ports, like rcs.
>
> I've already done this with a few, though I didn't delete the C
> versions from base.
> usr.bin/gstat => sysutils/gstat-rs
> tools/regression/fsx => devel/fsx
>

So
% size `which gstat-rs` `which gstat`
     text     data   bss       dec        hex   filename
  2094442   176472   568   2271482   0x22a8fa   /usr/local/sbin/gstat-rs
    19350     1180    41     20571     0x505b   /usr/sbin/gstat
% file `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
FreeBSD-style, stripped
/usr/sbin/gstat:          ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs:
libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
/usr/sbin/gstat:
libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)

So that looks scary, like rust is 100x larger binaries...  But at runtime
it's about the same:
USER    PID   %CPU %MEM   VSZ   RSS TT  STAT STARTED         TIME COMMAND
imp   14735    0.0  0.0 14140  4828  0  S+   20:38        0:00.04 gstat
imp   14766    1.3  0.0 15772  6256  0  S+   20:39        0:00.02 gstat-rs

So the runtime size is at least in the same ballpark (still larger, but not
crazy larger). More CPU too,
but that's just a polling artifact I think (other times gstat had some, and
gstat-rs didn't).

Why is the rust binary so much larger? Are the rust runtime and
dependencies statically linked?

Warner

>
> > Currently, it would not be so realistic, but once we completely switch
> > to pkgbase, IIUC, programs in base can sanely depemd on ports programs,
> > excluding kernel and fundamental libraries.
> >
> > As non-rust consumers of graphics/librsvg2-rust can sanely link with
> > it, I assume kmods in ports written in rust can kldload'ed sanely.
> > This could be a good starting point.
> >
> > And would be not all, but test for rust libraries could be implemented
> > with C/C++ or any other language suitable, if the rust libraries can
> > sanely linked with test codes.
>
> Yes, if the Rust library implements a C interface, which most don't.
>
> >
> > Am I wrong?
>