From nobody Sun Jan 21 03:43:53 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4THfNN11P6z589Xh for ; Sun, 21 Jan 2024 03:44:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4THfNM5lYVz4QyW for ; Sun, 21 Jan 2024 03:44:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x531.google.com with SMTP id 4fb4d7f45d1cf-55c02619a93so456554a12.3 for ; Sat, 20 Jan 2024 19:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1705808642; x=1706413442; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=H92rVvrBreax8DLr7XnDPBoa76FyvHsjmYtWGvmSnwI=; b=n8NHHW7xQmFgUNG3dswd+j9EMjLuHGCaIAt+QDu6sMmQNpwEzWYvS042Y88K0rq//d v+kzkMV5s4VSZ+YD0kXJgczGkxIeRzGYEm2aQ8ycGxcnZUgisX1lESdri+uvRGFUIyv/ NoenIo3UYa92kW5ez+1QqB71UMRmM/kz4oGnAW5vuHECb8qxnJuj+RZwHGJkDNEmPFEc 6c2e7TZjFa5eH9PhRiH7N0HJz69zubl3Oc6HwA0URpLFY/xZoWMmBOh538O3j1Lwr27q tiBZEVmkSb3S5vdlV6gIIURCIyXsjrn6ixPpffSOqnVY0USmieKBVtazMpv/0RZbYuda N9Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705808642; x=1706413442; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H92rVvrBreax8DLr7XnDPBoa76FyvHsjmYtWGvmSnwI=; b=sNoif+05rwZDl45ghjIrCOZ8N5Cr08cpSJvLDbkcpcl/csugVTlB9RzpgoXSHkov3m 8AF+ZN19Jwesx9gzwMLFsfS2xSyCVtrxR1Hkhy9JtO8M4eqPwKUow2bAaCI2f2fY6P+3 AHwTTdpNWeVbbdmfI/sfFFUdO359wqyok7wcbO9nEmsmsZtG//rc2CoLFROf1SFw0IVR aSEX3Kseki2z8oMiP2n59r3/4qahfkepykA4/BgSNJ8NSBQquuEccExT8PFBUFN1r8Dh cIjcfvLLaQuEf+uzQKkf0W+FnOlNfryeq1Rd1u9Vmsp/P6ULxXN7FSWZgkYpoukMRP+j oWEA== X-Gm-Message-State: AOJu0Ywq7N3di4kzKK8WCfanA7u8mUj/iyyyTy2lC4n5GIEyy1wu/OJl pggJCH8qjn7BRpEorqhH8+IEbb+gJ6uXJzH7Pu0dgrJpjKNj9MWogIcl76KMMTu9WUKUVM7xSIG My7U+0xnw1C7NKifCM+tOvdl3VUJgR3EO2CoHvg== X-Google-Smtp-Source: AGHT+IGY9LA4+Cq9UdnSPfNO9ZEJzEvYJ3d+EfnkMzckkZ2KKSUwqzaw8LVBnr+fmdThEJBATCuF3caVxh2nT7kRs5Y= X-Received: by 2002:a05:6402:35cb:b0:559:d75f:4902 with SMTP id z11-20020a05640235cb00b00559d75f4902mr1315912edc.74.1705808641958; Sat, 20 Jan 2024 19:44:01 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <1673801705774097@mail.yandex.ru> <20240121110611.af567b0ac3a8fd8593ffcb7f@dec.sakura.ne.jp> In-Reply-To: From: Warner Losh Date: Sat, 20 Jan 2024 20:43:53 -0700 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Alan Somers Cc: Tomoaki AOKI , Aleksandr Fedorov , FreeBSD Hackers , Scott Long , "Goran Meki??" Content-Type: multipart/alternative; boundary="00000000000028b509060f6c86b6" X-Rspamd-Queue-Id: 4THfNM5lYVz4QyW X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --00000000000028b509060f6c86b6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Jan 20, 2024, 7:20=E2=80=AFPM Alan Somers wro= te: > On Sat, Jan 20, 2024 at 7:06=E2=80=AFPM Tomoaki AOKI > wrote: > > > > On Sat, 20 Jan 2024 15:31:23 -0700 > > Warner Losh wrote: > > > > > On Sat, Jan 20, 2024 at 11:45=E2=80=AFAM Aleksandr Fedorov < > wigneddoom@yandex.ru> > > > wrote: > > > > > > > What about external dependencies? > > > > > > > > > https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L1= 9 > > > > > https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20 > > > > > > > > Is there any plan for which crates we should take into the base > system? > > > > > > > > We have had C++ in base for many years, but I don=E2=80=99t see any= good > libraries > > > > for CLI, logging, JSON, etc. > > > > > > > > > > > > > https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-to= ols > > > > > > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V= , > > > > Power? Should we rewrite devd? > > > > > > > > I think we need to start by providing official repositories (e.g > > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git) > > > > for different languages that include stable bindings to the system > API: > > > > - sysctl > > > > - libgeom > > > > - libifconfig > > > > - netgraph > > > > - jail > > > > - etc. > > > > > > > > So that it=E2=80=99s not just some anonymous on crates.io that repr= esents > these > > > > bindings, but our community. > > > > Officially, with support for a stable ABI for releases, security > patches, > > > > etc. > > > > > > > > After this, it will be possible to think about which components to > include > > > > in the base system. > > > > > > > > I would be glad to see a more modern language than C in the > database, but > > > > I=E2=80=99m afraid that it will be like with C++, > > > > that we will get a couple of daemons and utilities and that=E2=80= =99s all. > > > > > > > > > > These are all good questions that need good answers, though > necessarily to > > > get started. > > > > > > But the other question that occured to me after my last posting was > "What > > > about build integration?" > > > How much of the rust automation do we take in vs how much do we drive > from > > > a future bsd.rust.mk. > > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely > need > > > one for what we traditionally > > > think of as libraries (which may or may not map 1:1 onto crates: we > could > > > have c callable libraries > > > written in rust in the future, for example) and one for binaries. > > > Initially, though, if we go with the > > > 'make rust tests possible' then we'd likely need the appropriate > packages > > > installed for whatever > > > dependencies we'd have in the tests. This would give us a taste for > what > > > we'd need to do for > > > base, I'd think. Once we had that notion, I can easily see there > needing to > > > be some sort of > > > rust bindings for ATF/kyua as one of the first libraries / crates tha= t > > > would test that aspect of > > > the build system. That all would be up to the people writing the test= s > in > > > rust, I'd imagine. > > > > > > While I could jot out the basics of this integration (so one could > just add > > > the rust > > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever and > then > > > it would build > > > if rust is enabled, and would emit a warning it was skipped because > rust > > > was disabled). > > > We'd find out if this is workable or not and iterate from there. But > that > > > would also require > > > active participation from the rust advocates to make it a reality: I > can > > > put together the > > > build infrastructure for the disabled case, but likely can't on my ow= n > do > > > the rust enabled > > > case. I'd be happy to work with someone to do that, but I'm not going > to be > > > able to do > > > that myself: my need for rust is slight, my knowledge of rust is weak= , > etc. > > > Working with > > > someone (or ideally several someones), though it could become reality= . > So > > > please contact > > > me if you'd like to work on this. > > > > > > Warner > > > > One way to go could be moving programs rewritten with rust to ports. > > There are some programs (not in rust, though) moved to ports, like rcs. > > I've already done this with a few, though I didn't delete the C > versions from base. > usr.bin/gstat =3D> sysutils/gstat-rs > tools/regression/fsx =3D> devel/fsx > So % size `which gstat-rs` `which gstat` text data bss dec hex filename 2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs 19350 1180 41 20571 0x505b /usr/sbin/gstat % file `which gstat-rs` `which gstat` /usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, stripped /usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style, stripped 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat` /usr/local/sbin/gstat-rs: libgeom.so.5 =3D> /lib/libgeom.so.5 (0x60fd38647000) libthr.so.3 =3D> /lib/libthr.so.3 (0x60fd38b57000) libgcc_s.so.1 =3D> /lib/libgcc_s.so.1 (0x60fd39af1000) libc.so.7 =3D> /lib/libc.so.7 (0x60fd3be6f000) libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x60fd3a009000) libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x60fd3a55e000) /usr/sbin/gstat: libdevstat.so.7 =3D> /lib/libdevstat.so.7 (0x448867cd000) libgeom.so.5 =3D> /lib/libgeom.so.5 (0x4488710b000) libedit.so.8 =3D> /lib/libedit.so.8 (0x44887f8d000) libtinfow.so.9 =3D> /lib/libtinfow.so.9 (0x44888aab000) libncursesw.so.9 =3D> /lib/libncursesw.so.9 (0x44889c60000) libc.so.7 =3D> /lib/libc.so.7 (0x4488aaf4000) libkvm.so.7 =3D> /lib/libkvm.so.7 (0x44888f77000) libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x4488ba02000) libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x4488c68d000) libelf.so.2 =3D> /lib/libelf.so.2 (0x4488ca45000) So that looks scary, like rust is 100x larger binaries... But at runtime it's about the same: USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-rs So the runtime size is at least in the same ballpark (still larger, but not crazy larger). More CPU too, but that's just a polling artifact I think (other times gstat had some, and gstat-rs didn't). Why is the rust binary so much larger? Are the rust runtime and dependencies statically linked? Warner > > > Currently, it would not be so realistic, but once we completely switch > > to pkgbase, IIUC, programs in base can sanely depemd on ports programs, > > excluding kernel and fundamental libraries. > > > > As non-rust consumers of graphics/librsvg2-rust can sanely link with > > it, I assume kmods in ports written in rust can kldload'ed sanely. > > This could be a good starting point. > > > > And would be not all, but test for rust libraries could be implemented > > with C/C++ or any other language suitable, if the rust libraries can > > sanely linked with test codes. > > Yes, if the Rust library implements a C interface, which most don't. > > > > > Am I wrong? > --00000000000028b509060f6c86b6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, Jan 20, 2024, 7:20=E2=80=AFPM= Alan Somers <a= somers@freebsd.org> wrote:
O= n Sat, Jan 20, 2024 at 7:06=E2=80=AFPM Tomoaki AOKI <junchoon@dec= .sakura.ne.jp> wrote:
>
> On Sat, 20 Jan 2024 15:31:23 -0700
> Warner Losh <imp@bsdimp.com> wrote:
>
> > On Sat, Jan 20, 2024 at 11:45=E2=80=AFAM Aleksandr Fedorov <= wigneddoom@yandex.ru>
> > wrote:
> >
> > > What about external dependencies?
> > >
> > > = https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19<= /a>
> > >
https= ://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
> > >
> > > Is there any plan for which crates we should take into the b= ase system?
> > >
> > > We have had C++ in base for many years, but I don=E2=80=99t = see any good libraries
> > > for CLI, logging, JSON, etc.
> > >
> > >
> > > https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-to= ols
> > >
> > > Where is the support for Freebsd as a primary platform? ARM,= RISC-V,
> > > Power? Should we rewrite devd?
> > >
> > > I think we need to start by providing official repositories = (e.g
> > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
> > > for different languages that include stable bindings to the = system API:
> > > - sysctl
> > > - libgeom
> > > - libifconfig
> > > - netgraph
> > > - jail
> > > - etc.
> > >
> > > So that it=E2=80=99s not just some anonymous on crates.io<= /a> that represents these
> > > bindings, but our community.
> > > Officially, with support for a stable ABI for releases, secu= rity patches,
> > > etc.
> > >
> > > After this, it will be possible to think about which compone= nts to include
> > > in the base system.
> > >
> > > I would be glad to see a more modern language than C in the = database, but
> > > I=E2=80=99m afraid that it will be like with C++,
> > > that we will get a couple of daemons and utilities and that= =E2=80=99s all.
> > >
> >
> > These are all good questions that need good answers, though neces= sarily to
> > get started.
> >
> > But the other question that occured to me after my last posting w= as "What
> > about build integration?"
> > How much of the rust automation do we take in vs how much do we d= rive from
> > a future
bsd.rust.mk.
> > I can sketch out bsd.rust.mk (to pick an arbitrary name, = we'd likely need
> > one for what we traditionally
> > think of as libraries (which may or may not map 1:1 onto crates: = we could
> > have c callable libraries
> > written in rust in the future, for example) and one for binaries.=
> > Initially, though, if we go with the
> > 'make rust tests possible' then we'd likely need the = appropriate packages
> > installed for whatever
> > dependencies we'd have in the tests. This would give us a tas= te for what
> > we'd need to do for
> > base, I'd think. Once we had that notion, I can easily see th= ere needing to
> > be some sort of
> > rust bindings for ATF/kyua as one of the first libraries / crates= that
> > would test that aspect of
> > the build system. That all would be up to the people writing the = tests in
> > rust, I'd imagine.
> >
> > While I could jot out the basics of this integration (so one coul= d just add
> > the rust
> > tools to a subdir or subdirs, include the bsd.rust.mk or = whatever and then
> > it would build
> > if rust is enabled, and would emit a warning it was skipped becau= se rust
> > was disabled).
> > We'd find out if this is workable or not and iterate from the= re. But that
> > would also require
> > active participation from the rust advocates to make it a reality= : I can
> > put together the
> > build infrastructure for the disabled case, but likely can't = on my own do
> > the rust enabled
> > case. I'd be happy to work with someone to do that, but I'= ;m not going to be
> > able to do
> > that myself: my need for rust is slight, my knowledge of rust is = weak, etc.
> > Working with
> > someone (or ideally several someones), though it could become rea= lity. So
> > please contact
> > me if you'd like to work on this.
> >
> > Warner
>
> One way to go could be moving programs rewritten with rust to ports. > There are some programs (not in rust, though) moved to ports, like rcs= .

I've already done this with a few, though I didn't delete the C
versions from base.
usr.bin/gstat =3D> sysutils/gstat-rs
tools/regression/fsx =3D> devel/fsx

So
% size `which gstat-rs` `which gstat`=
=C2=A0 =C2=A0 =C2=A0text =C2=A0 =C2=A0 data =C2=A0 bss =C2=A0 =C2=A0 = =C2=A0 dec =C2=A0 =C2=A0 =C2=A0 =C2=A0hex =C2=A0 filename
=C2=A0 2094442= =C2=A0 176472 =C2=A0 568 =C2=A0 2271482 =C2=A0 0x22a8fa =C2=A0 /usr/local/= sbin/gstat-rs
=C2=A0 =C2=A0 19350 =C2=A0 =C2=A0 1180 =C2=A0 =C2=A041 =C2= =A0 =C2=A0 20571 =C2=A0 =C2=A0 0x505b =C2=A0 /usr/sbin/gstat
= % file `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs: ELF 64-b= it LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked= , interpreter /libexec/ld-elf.so.1, FreeBSD-style, stripped
/usr/sbin/gs= tat: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ELF 64-bit LSB pie executable, ARM a= arch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-el= f.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
8:36pm brazo= s:[3826]> ldd `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs= :
libgeom.so.5 =3D> /lib/libgeom.so.5 (0x60fd38647000)
libthr.so= .3 =3D> /lib/libthr.so.3 (0x60fd38b57000)
libgcc_s.so.1 =3D> /lib= /libgcc_s.so.1 (0x60fd39af1000)
libc.so.7 =3D> /lib/libc.so.7 (0x60f= d3be6f000)
libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x60fd3a009000)<= br> libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x60fd3a55e000)
/usr/sbin/gs= tat:
libdevstat.so.7 =3D> /lib/libdevstat.so.7 (0x448867cd000)
l= ibgeom.so.5 =3D> /lib/libgeom.so.5 (0x4488710b000)
libedit.so.8 =3D&= gt; /lib/libedit.so.8 (0x44887f8d000)
libtinfow.so.9 =3D> /lib/libti= nfow.so.9 (0x44888aab000)
libncursesw.so.9 =3D> /lib/libncursesw.so.= 9 (0x44889c60000)
libc.so.7 =3D> /lib/libc.so.7 (0x4488aaf4000)
= libkvm.so.7 =3D> /lib/libkvm.so.7 (0x44888f77000)
libbsdxml.so.4 =3D= > /lib/libbsdxml.so.4 (0x4488ba02000)
libsbuf.so.6 =3D> /lib/libs= buf.so.6 (0x4488c68d000)
libelf.so.2 =3D> /lib/libelf.so.2 (0x4488ca= 45000)

So that looks scary, like rust is 100x larg= er binaries...=C2=A0 But at runtime it's about the same:
USER= =C2=A0 =C2=A0PID =C2=A0 %CPU %MEM =C2=A0 VSZ =C2=A0 RSS TT =C2=A0STAT STAR= TED =C2=A0 =C2=A0 =C2=A0 =C2=A0 TIME COMMAND
imp =C2=A0 14735 =C2= =A0 =C2=A00.0 =C2=A00.0 14140 =C2=A04828 =C2=A00 =C2=A0S+ =C2=A0 20:38 =C2= =A0 =C2=A0 =C2=A0 =C2=A00:00.04 gstat
imp =C2=A0 14766 =C2=A0 =C2= =A01.3 =C2=A00.0 15772 =C2=A06256 =C2=A00 =C2=A0S+ =C2=A0 20:39 =C2=A0 =C2= =A0 =C2=A0 =C2=A00:00.02 gstat-rs

So the runtime s= ize is at least in the same ballpark (still larger, but not crazy larger). = More CPU too,
but that's just a polling artifact I think (oth= er times gstat had some, and gstat-rs didn't).

=
Why is the rust binary so much larger? Are the rust runtime and depend= encies statically linked?

Warner

>
> Currently, it would not be so realistic, but once we completely switch=
> to pkgbase, IIUC, programs in base can sanely depemd on ports programs= ,
> excluding kernel and fundamental libraries.
>
> As non-rust consumers of graphics/librsvg2-rust can sanely link with > it, I assume kmods in ports written in rust can kldload'ed sanely.=
> This could be a good starting point.
>
> And would be not all, but test for rust libraries could be implemented=
> with C/C++ or any other language suitable, if the rust libraries can > sanely linked with test codes.

Yes, if the Rust library implements a C interface, which most don't.
>
> Am I wrong?
--00000000000028b509060f6c86b6--