From nobody Mon Aug 05 22:19:23 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wd9pd2qPDz5Sp3x for ; Mon, 05 Aug 2024 22:19:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wd9pd0xcrz47lt for ; Mon, 5 Aug 2024 22:19:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-7a0e8b76813so7470187a12.3 for ; Mon, 05 Aug 2024 15:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1722896375; x=1723501175; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=rF/QFYE5FhGm48Anfg0xUIsZIBfwBJc9p88a6Lrl1eI=; b=vAaNCMtRoBHCUGbjZfBavAHZE+IYVi5zEp76OAJqK+NmefVzFFnTN2qP0nb+FKu4VU jwZOEODxTRxFDY9Z+6EPSXbvgpzaKJsPbFNqy8lY2pJ+1sPpBDaKTyMrTCOEQRcqxNPq cf6zIV8nphZH+uRgcmktVigJ/Nj+qsxz98JOpL+RLwJ1uCbxsv7hBsTpYf7H4qbZTSy1 0hR7XUktJN9myCcNygQZBjPJotl0L9efu5L2uk/k9a6hZvsLOhhQMGpi8atCnbaTA7rs VKw9nughmJFYLbrcPy633/HUAwuHgOrKX/4oi9fJAvuyEaDcVJeF71ogP1pzuQAHZj31 kzbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722896375; x=1723501175; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rF/QFYE5FhGm48Anfg0xUIsZIBfwBJc9p88a6Lrl1eI=; b=l4U8Rl6wden943KteLIVkOb3qufJ19/Z1JLmIioiy/s1e6M5P42XjcQjp1eqUD+Spk owNKq9AMQZM/8Vb56eh7IIvPoZLEfkzmA66Qjwtx7/oOEkbudDcpW5JeQOWycyJUoMuF TzQseTel+moKg6DuNyBh3F9JZzFFtjzBvHxLPXUmDlyC1aR5CumhCjhTozfLf3jFMVIS nz0VNT9Nv6kC0C4Q9k8JDaH9ti4xXoFbuUxebGWe6SLyj6wQVshQ0GFujo3nl7tOBi1h 4K3P6cA0GaHobgqqhH/CisUnIKUQN6NVh9mt07BulyuPfvIJlSzlRSVjVv+GHE7b9v+X g4UA== X-Forwarded-Encrypted: i=1; AJvYcCXchAIk9NeUGl2QXg/qzJPdiH4K45t4Cvsaim6Tn+lIp89xqzVHMmZ135cguW10r2f3kr1bMxeCc8uxv8e8Ml+rNPTD6k75tW9EPQ0= X-Gm-Message-State: AOJu0YxM/K4fapRisJ4wBog4rfM5AbLHDut6NmYe6PvZRU4cJ8RrR6eh L4hvc1qW20jYqFzCOdwkT3Wokf4iiqwyKLs41difwV+yyNsNVoJxTJVfX48sckh7r6q2Bx8EHSy iC7jsi30zyGJgxkTGiCDsWlvosFpBrdccqlBa7g== X-Google-Smtp-Source: AGHT+IEcmQAW/vD4HLrAMfQOBgmL8reCTVQmDRfd1YH5SKHrJH6Dq7OGYviNI/EBrqRfBaPE6MmoWnYkZ2zhVOvg9Iw= X-Received: by 2002:a17:90b:4a08:b0:2c9:9fcd:aa51 with SMTP id 98e67ed59e1d1-2cff9419a1fmr15073035a91.5.1722896374728; Mon, 05 Aug 2024 15:19:34 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <704D020F-78A4-4926-AE3C-41F7FD619A89@cschubert.com> <20240805210149.nrkHN3j3@steffen%sdaoden.eu> <202408052127.475LROnE067608@critter.freebsd.dk> <202408052206.475M6h8E067967@critter.freebsd.dk> In-Reply-To: <202408052206.475M6h8E067967@critter.freebsd.dk> From: Warner Losh Date: Mon, 5 Aug 2024 16:19:23 -0600 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Poul-Henning Kamp Cc: Steffen Nurpmeso , Cy Schubert , freebsd-hackers@freebsd.org, Bakul Shah Content-Type: multipart/alternative; boundary="00000000000066a6a0061ef712ce" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Wd9pd0xcrz47lt --00000000000066a6a0061ef712ce Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Aug 5, 2024 at 4:06=E2=80=AFPM Poul-Henning Kamp wrote: > -------- > Warner Losh writes: > > > > > Most user space tools could be written in lua. > > > > > > That /exact/ same argument was made for Perl :-) > > > > Lua's been in the base since I imported it for the boot loader, though.= .. > > Lua is is much more "language" than "eco-system", by design as I > understand it, so that is a different situation than Perl or Rust. > Indeed. The standard stuff is fairly small, and we're doing it all as a 'private' library, so ports will never see what we pull in. I personally do not subscribe to to the "let's rewrite all the 50 > year old source code to make it more safe" philosophy, but there > are valid arguments when the old code is horrible. > Plus we're writing new stuff only, and typically only where it makes a lot of sense (lots of string processing). Plus we're keeping the scripts as compatible with what little ecosystem there is so we can go back and forth between the ports lua and the base flua. > But there are some wrinkles. > > First: Anything setuid/setgid is off-limits. > > There are good reasons why we dont have setuid shell-scripts (any more!) > > I guess with most systems being effectively single-user these days, > that may not be as much a security focus as it was back in the 1990ies. > Yea. No plans there. > Second: Performance. > > I cannot remember the exact subset of bin programs somebody did in > Perl as proof of concept, but it slowed down buildworld a LOT to > fire up all of Perl to do trivial stuff like "echo", "chown" and > "mkdir". > > Lua may be cheaper than Perl, but it will still be measurable. > Yea. I'm guessing you wouldn't notice, but why do that. There's no benefit and only a myriad of ways to introduce new bugs or non-posix conformance where we were conformant before. I'm definitely in the "why are we rewriting stuff in rust" because it doesn't move the ball forward, really. At best it's a great leap sideways, maybe with marginally better actual safety. At worst, it's a great leap int= o a morass of almost compatible that causes great grief in the gaps, or worse, has new security problems the old one didn't. So rewriting for the sake of rewriting seems like a giant waste of resources. Rewriting strategically to fix areas that have had safey issues may be different, but cp.rst isn't going to be any better, than cp.c in most aspects because cp.c has had 50 years to be debugged. And 50 years makes up for a lot of danger in the language.... So there may be things that we get some advantage out of by doing a rewrite in rust, but I'm in the 'case by case basis' camp there: those cases where the cost / benefit ratio is favorable should be considered. But they can't be considered entirely in a vacuum because there's a non-zero cost to rust in the base, even as an external toolchain. Having said all that, I'd love to see us be able to make better of rust and new rust programs where it makes sense. That's why I've been encouraging people to give it a go to show us the money. To show up that we can integrate it (even if it is just a few lines in Makefile.inc1 that builds everything, optionally, as part of buildworld). That shows us we can keep the dependency hell under control, that we can update things sanely (more ecosystem here, not language). How much work is it to track the latest versions, how do we deal with that as the number of new rust programs grow, how do we deal with ABI stability, etc. And to show us if there's an actual advantage to all of that over what we can do in ports, or what we might do with pkgbase somehow. I'm unsure of the outcome of all this, but I think it would be wrong to shout it down completely. To do all that, people need room to experiment and show what's what. Warner --00000000000066a6a0061ef712ce Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Aug 5, 2024 at 4:06=E2=80=AFP= M Poul-Henning Kamp <phk@phk.freeb= sd.dk> wrote:
--------
Warner Losh writes:

> > > Most user space tools could be written in lua.
> >
> > That /exact/ same argument was made for Perl :-)
>
> Lua's been in the base since I imported it for the boot loader, th= ough...

Lua is is much more "language" than "eco-system", by de= sign as I
understand it, so that is a different situation than Perl or Rust.

Indeed. The standard stuff is fairly small, and= we're doing it all as
a 'private' library, so ports = will=C2=A0never see what we pull in.

I personally do not subscribe to to the &q= uot;let's rewrite all the 50
year old source code to make it more safe" philosophy, but there
are valid arguments when the old code is horrible.
Plus we're writing new stuff only, and typically only where= it makes
a lot of sense (lots of string processing). Plus we'= ;re keeping the scripts
as compatible with what little ecosystem = there is so we can go back
and forth between the ports lua and th= e base flua.
=C2=A0
But there are some wrinkles.

First: Anything setuid/setgid is off-limits.

There are good reasons why we dont have setuid shell-scripts (any more!)
I guess with most systems being effectively single-user these days,
that may not be as much a security focus as it was back in the 1990ies.
=

Yea. No plans there.
=C2=A0
Second: Performance.

I cannot remember the exact subset of bin programs somebody did in
Perl as proof of concept, but it slowed down buildworld a LOT to
fire up all of Perl to do trivial stuff like "echo", "chown&= quot; and
"mkdir".

Lua may be cheaper than Perl, but it will still be measurable.

Yea. I'm guessing you wouldn't notice, but = why do that. There's
no benefit and only a myriad of ways to = introduce new bugs
or non-posix conformance where we were conform= ant before.

I'm definitely in the "why ar= e we rewriting stuff in rust" because
it doesn't move th= e ball forward, really. At best it's a great leap sideways,
m= aybe with marginally better actual safety. At worst, it's a great leap = into
a morass of almost compatible that causes great grief in the= gaps, or
worse, has new security problems the old one didn't= . So rewriting
for the sake of rewriting seems like a giant waste= of resources.
Rewriting strategically to fix areas that have had= safey=C2=A0issues
may be different, but cp.rst isn't going t= o be any better, than
cp.c in most aspects because cp.c has had 5= 0 years to be
debugged.=C2=A0 And 50 years makes up for a lot of = danger
in the language.... So there may be things that we get som= e
advantage out of by doing a rewrite in rust, but I'm in the=
'case by case basis' camp there: those cases where the c= ost / benefit
ratio is favorable should be considered. But they c= an't be
considered entirely in a vacuum because there's a= non-zero
cost to rust in the base, even as an external toolchain= .

Having said all that, I'd love to see us be = able to make better of rust and
new rust programs where it makes = sense. That's why I've been
encouraging people to give it= a go to show us the money. To
show up that we can integrate it (= even if it is just a few lines in
Makefile.inc1 that builds every= thing, optionally, as part of
buildworld). That shows us we can k= eep the dependency hell
under control, that we can update things = sanely (more ecosystem
here, not language). How much work is it t= o track the latest versions,
how do we deal with that as the numb= er of new rust programs grow,
how do we deal with ABI stability, = etc. And to show us if there's an
actual advantage to all of = that over what we can do in ports, or what
we might do with pkgba= se somehow. I'm unsure of the outcome of all
this, but I thin= k it would be wrong to shout it down completely. To do
all that, = people need room to experiment and show what's what.

Warner

--00000000000066a6a0061ef712ce--