Re: The Case for Rust (in the base system)

On Sat, Jan 20, 2024 at 08:43:53PM -0700, Warner Losh wrote:
> On Sat, Jan 20, 2024, 7:20 PM Alan Somers <asomers@freebsd.org> wrote:
> 
> > On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI <junchoon@dec.sakurane.jp>
> > wrote:
> > >
> > > On Sat, 20 Jan 2024 15:31:23 -0700
> > > Warner Losh <imp@bsdimp.com> wrote:
> > >
> > > > On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
> > wigneddoom@yandex.ru>
> > > > wrote:
> > > >
> > > > > What about external dependencies?
> > > > >
> > > > >
> > https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
> > > > >
> > https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
> > > > >
> > > > > Is there any plan for which crates we should take into the base
> > system?
> > > > >
> > > > > We have had C++ in base for many years, but I don’t see any good
> > libraries
> > > > > for CLI, logging, JSON, etc.
> > > > >
> > > > >
> > > > >
> > https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
> > > > >
> > > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V,
> > > > > Power? Should we rewrite devd?
> > > > >
> > > > > I think we need to start by providing official repositories (e.g
> > > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
> > > > > for different languages that include stable bindings to the system
> > API:
> > > > > - sysctl
> > > > > - libgeom
> > > > > - libifconfig
> > > > > - netgraph
> > > > > - jail
> > > > > - etc.
> > > > >
> > > > > So that it’s not just some anonymous on crates.io that represents
> > these
> > > > > bindings, but our community.
> > > > > Officially, with support for a stable ABI for releases, security
> > patches,
> > > > > etc.
> > > > >
> > > > > After this, it will be possible to think about which components to
> > include
> > > > > in the base system.
> > > > >
> > > > > I would be glad to see a more modern language than C in the
> > database, but
> > > > > I’m afraid that it will be like with C++,
> > > > > that we will get a couple of daemons and utilities and that’s all.
> > > > >
> > > >
> > > > These are all good questions that need good answers, though
> > necessarily to
> > > > get started.
> > > >
> > > > But the other question that occured to me after my last posting was
> > "What
> > > > about build integration?"
> > > > How much of the rust automation do we take in vs how much do we drive
> > from
> > > > a future bsd.rust.mk.
> > > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
> > need
> > > > one for what we traditionally
> > > > think of as libraries (which may or may not map 1:1 onto crates: we
> > could
> > > > have c callable libraries
> > > > written in rust in the future, for example) and one for binaries.
> > > > Initially, though, if we go with the
> > > > 'make rust tests possible' then we'd likely need the appropriate
> > packages
> > > > installed for whatever
> > > > dependencies we'd have in the tests. This would give us a taste for
> > what
> > > > we'd need to do for
> > > > base, I'd think. Once we had that notion, I can easily see there
> > needing to
> > > > be some sort of
> > > > rust bindings for ATF/kyua as one of the first libraries / crates that
> > > > would test that aspect of
> > > > the build system. That all would be up to the people writing the tests
> > in
> > > > rust, I'd imagine.
> > > >
> > > > While I could jot out the basics of this integration (so one could
> > just add
> > > > the rust
> > > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever and
> > then
> > > > it would build
> > > > if rust is enabled, and would emit a warning it was skipped because
> > rust
> > > > was disabled).
> > > > We'd find out if this is workable or not and iterate from there. But
> > that
> > > > would also require
> > > > active participation from the rust advocates to make it a reality: I
> > can
> > > > put together the
> > > > build infrastructure for the disabled case, but likely can't on my own
> > do
> > > > the rust enabled
> > > > case. I'd be happy to work with someone to do that, but I'm not going
> > to be
> > > > able to do
> > > > that myself: my need for rust is slight, my knowledge of rust is weak,
> > etc.
> > > > Working with
> > > > someone (or ideally several someones), though it could become reality.
> > So
> > > > please contact
> > > > me if you'd like to work on this.
> > > >
> > > > Warner
> > >
> > > One way to go could be moving programs rewritten with rust to ports.
> > > There are some programs (not in rust, though) moved to ports, like rcs.
> >
> > I've already done this with a few, though I didn't delete the C
> > versions from base.
> > usr.bin/gstat => sysutils/gstat-rs
> > tools/regression/fsx => devel/fsx
> >
> 
> So
> % size `which gstat-rs` `which gstat`
>      text     data   bss       dec        hex   filename
>   2094442   176472   568   2271482   0x22a8fa   /usr/local/sbin/gstat-rs
>     19350     1180    41     20571     0x505b   /usr/sbin/gstat
> % file `which gstat-rs` `which gstat`
> /usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64,
> version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
> FreeBSD-style, stripped
> /usr/sbin/gstat:          ELF 64-bit LSB pie executable, ARM aarch64,
> version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
> for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
> 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
> /usr/local/sbin/gstat-rs:
> libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
> libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
> libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
> libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
> libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
> /usr/sbin/gstat:
> libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
> libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
> libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
> libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
> libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
> libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
> libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
> libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
> libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
> libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)
> 
> So that looks scary, like rust is 100x larger binaries...  But at runtime
> it's about the same:
> USER    PID   %CPU %MEM   VSZ   RSS TT  STAT STARTED         TIME COMMAND
> imp   14735    0.0  0.0 14140  4828  0  S+   20:38        0:00.04 gstat
> imp   14766    1.3  0.0 15772  6256  0  S+   20:39        0:00.02 gstat-rs
> 
> So the runtime size is at least in the same ballpark (still larger, but not
> crazy larger). More CPU too,
> but that's just a polling artifact I think (other times gstat had some, and
> gstat-rs didn't).
> 
> Why is the rust binary so much larger? Are the rust runtime and
> dependencies statically linked?

Simple answer is yes.

More complete answer is that nobody, practically speaking, writes a program
in Rust + standard library.  It is always several dozens of crates that are
often indeed useful, and which bring up their own dependencies.  All of that
stuff is statically linked into the binary, remember that Rust does not
have stable ABI.

For some parts of the language, like async, you must use huge external
dependencies, importing which into the base system is not practical
(I do not mean a compiler, which seems to be already agreed not to).

Also Rust-generated code seems to be fatter than corresponding C, not least
because value-passing is much more common in Rust.  Looking at disassembly
the dominating theme are memory moves.