From nobody Thu Aug 29 21:16:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WvvGg3kyXz5MRXy for ; Thu, 29 Aug 2024 21:16:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WvvGg1Fqcz4Rkm for ; Thu, 29 Aug 2024 21:16:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2d3c0d587e4so854276a91.2 for ; Thu, 29 Aug 2024 14:16:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1724966186; x=1725570986; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=4VzIHlDHfYal3QgR5sqnQmPT8r2wo+19ZMiFV0WU3Hc=; b=HvjSfrWzz5SIZd1w6vGlq9Lq9qBaeKukspkzU3zXiorPRNPYJwhTjuBB9xoILm3VBv SR6vRJWys2nDFKroL5DpmXsDvqRrZMvlJNm+lnbuzDgpfM9MMZoZ6xTp5KosI336vbjS iTEYxZDsVeKUBcfGv/K3KQ00IwuvnXAsASXfIzWtHH4XXwmjKLFZldAGtDrg+Q8ILghI 0zhADOcP+D7sPmmthE8QH6iOR1bkXpL3zFvFn2++Ef/Nw/nqGbh7gOhAwwmXOSDUrE2A 9tRfiSAjMemEkIZRAXgiqUhsN+3idowlNivEUm1JS+RpBT84RYbdbMfzpMQlRLoOADhA c9Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724966186; x=1725570986; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4VzIHlDHfYal3QgR5sqnQmPT8r2wo+19ZMiFV0WU3Hc=; b=F9+VI0dXhVNHqoV8JHRCEQPDVE4O91j5/nVVhuTVh2kmZv8NuevGWwXpDVHg1hV06i d1SakLqKuyliOZheoC8tCf4aUJxwAx9QYpxXuYjpH95iuQvOMED290sAyeSW8e76qTyX HMsgdTb/8sFbFMFAtofqUdGYcO2mZuFpZ1DkaSi1zd0vlYgDiU7+9zs4AVN/3OPuwyju XpZyaHmDuWEHnEl1Fse4ybstmwMdi7wWxvQb950WWMnbSvRvc36usYYKCpfvoBaqFLZq Aw/I+5Ms39zfIT1rVQydYQ0nWdjPPVIW1uXpNY+mFa8drJDTNE7Cg5gIItEQr1S0AT8h tcrw== X-Forwarded-Encrypted: i=1; AJvYcCVhFX3n78ZzIemOidZim3hb4Q3tPmgPYZwxe+1dZtKUPA7BlvwcymtqchXPqUlW739HN6+eZ6pIE/RGpN04WWg=@freebsd.org X-Gm-Message-State: AOJu0YzQNrE2JJreJiUlHNEefusQjyOPsW34qFpzux5EjbpgIE1Z8p45 aPWeXLhzebdJ2AOTesThNkwENNge7vYlI6avV5WC5dWXCmExImK6/JI3TXgvm77r1tkHQ1RT5+5 KK1tLP9ozCIUOhhAno8cu2V6I6tAqIvn03LK1OnnSjYf4jMmA X-Google-Smtp-Source: AGHT+IFvbBuO7M+/HH8keJsY915e2Sx2yRE51OUWStLZmt/5wFNzN7lzMydRAf5PKeOhfr3Axg/UefTSoyr7TAoPLnI= X-Received: by 2002:a17:90a:d812:b0:2d3:c4d3:de19 with SMTP id 98e67ed59e1d1-2d855d45b7amr4452391a91.0.1724966185566; Thu, 29 Aug 2024 14:16:25 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Thu, 29 Aug 2024 15:16:05 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Alan Somers Cc: Shawn Webb , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000bdad180620d8fcb2" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WvvGg1Fqcz4Rkm --000000000000bdad180620d8fcb2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Aug 29, 2024, 2:55=E2=80=AFPM Alan Somers wro= te: > On Thu, Aug 29, 2024 at 2:21=E2=80=AFPM Shawn Webb > wrote: > > > > Hey Alan et al, > > > > I apologize for the silence on my end. It has been a busier two months > > than anticipated. In that time, I've been entertaining some thoughts > > on Rust in base support. ${LIFE} is starting to calm down again, and I > > do believe I'll be able to start the research in time in September. I > > will be splitting my free time between this and studying for my OSCP > > cert. > > > > So, to those thoughts, in list form (in no particular order): > > > > 1. Use of Rust compiler toolchain support will be for userland > > components in an opt-in fashion. Meaning, all userland components > > written in Rust will be optional. > > 2. It does not make sense to perform a vendor import of the Rust > > compiler toolchain and standard libraries. All Rust code in the src > > tree must be built from an external toolchain. > > 3. I believe the notion of an external toolchain could be abstracted > > such that we can support any optional userland component written in > > a language supported by that external toolchain. This would imply > > that other alternative languages could be supported with minimal > > work (Zig, TypeScript, Python, Java, etc.) > > 4. We could provide auto-detection mechanisms for determining which > > external toolchains are available, their language support, etc. The > > initial proof-of-concept would likely be limited to Rust to save on > > time and complexity. > > 5. As the work matures, and perhaps as a requisite for eventual > > inclusion, we could land support for more than Rust. This might be > > a step too far, but hey, it's one of the thoughts I had. > > 6. So all of this wrapped up means that: > > A. This is NOT a call to rewrite everything in Rust. This work will > > only permit NEW, OPTIONAL components to be written. > > B. Other languages/toolchains/ecosystems could be supported, not > > just Rust. > > C. Initial focus is on userland components. Rust in the kernel is > > out of scope for this initial proof-of-concept. > > D. I would like to see Rust in the kernel. That would be a good > > next area of focus once userland support reaches some level of > > maturity. > > > > My first goal will be to get a better understanding of > > src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also > > study your work, Alan. I really appreciate the time you have taken. I > > might reach out to you and Warner directly for further questions. > > Relying on an external toolchain and allowing for future external > toolchains other than Rust sounds like a really good plan. > It's the only way it could work. Importing rust with its current level of maturity and lag is logistically impossible or nearly so.rust doesn't have a long enough support timeline to work well with our stable branches. Llvm is tricky enough... Conceivably we could even ditch our Lua import and use the same > mechanism for that (but please, save Lua discussion for a separate > flame war ;) ). That can't possibly work since we build it into the loader with changes that are unavoidable. There's no flame war, it's just impossibile. The situation is entirely different. But the lua import is the easiest of all the vendor imports I do. Warner I anticipate that the next big decisions will be > "what components are so important that they musn't require an external > toolchain?" and "how much cargo crate bloat is too much?". But we can > cross those bridges when we come to them. I look forward to seeing > your work, and please don't hesitate to ask for help. > Managing these things in a FreeBSD context will be challenging and i look forward to the results we get. Warner -Alan > > --000000000000bdad180620d8fcb2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Aug 29, 2024, 2:55=E2=80=AFPM Alan Somers <= asomers@freebsd.org> wrote:
On Thu, Aug 29, 2024 at 2:21=E2=80= =AFPM Shawn Webb <shawn.webb@hardenedbsd.org> wrote: >
> Hey Alan et al,
>
> I apologize for the silence on my end. It has been a busier two months=
> than anticipated. In that time, I've been entertaining some though= ts
> on Rust in base support. ${LIFE} is starting to calm down again, and I=
> do believe I'll be able to start the research in time in September= . I
> will be splitting my free time between this and studying for my OSCP > cert.
>
> So, to those thoughts, in list form (in no particular order):
>
> 1. Use of Rust compiler toolchain support will be for userland
>=C2=A0 =C2=A0 components in an opt-in fashion. Meaning, all userland co= mponents
>=C2=A0 =C2=A0 written in Rust will be optional.
> 2. It does not make sense to perform a vendor import of the Rust
>=C2=A0 =C2=A0 compiler toolchain and standard libraries. All Rust code = in the src
>=C2=A0 =C2=A0 tree must be built from an external toolchain.
> 3. I believe the notion of an external toolchain could be abstracted >=C2=A0 =C2=A0 such that we can support any optional userland component = written in
>=C2=A0 =C2=A0 a language supported by that external toolchain. This wou= ld imply
>=C2=A0 =C2=A0 that other alternative languages could be supported with = minimal
>=C2=A0 =C2=A0 work (Zig, TypeScript, Python, Java, etc.)
> 4. We could provide auto-detection mechanisms for determining which >=C2=A0 =C2=A0 external toolchains are available, their language support= , etc. The
>=C2=A0 =C2=A0 initial proof-of-concept would likely be limited to Rust = to save on
>=C2=A0 =C2=A0 time and complexity.
> 5. As the work matures, and perhaps as a requisite for eventual
>=C2=A0 =C2=A0 inclusion, we could land support for more than Rust. This= might be
>=C2=A0 =C2=A0 a step too far, but hey, it's one of the thoughts I h= ad.
> 6. So all of this wrapped up means that:
>=C2=A0 =C2=A0 A. This is NOT a call to rewrite everything in Rust. This= work will
>=C2=A0 =C2=A0 =C2=A0 =C2=A0only permit NEW, OPTIONAL components to be w= ritten.
>=C2=A0 =C2=A0 B. Other languages/toolchains/ecosystems could be support= ed, not
>=C2=A0 =C2=A0 =C2=A0 =C2=A0just Rust.
>=C2=A0 =C2=A0 C. Initial focus is on userland components. Rust in the k= ernel is
>=C2=A0 =C2=A0 =C2=A0 =C2=A0out of scope for this initial proof-of-conce= pt.
>=C2=A0 =C2=A0 D. I would like to see Rust in the kernel. That would be = a good
>=C2=A0 =C2=A0 =C2=A0 =C2=A0next area of focus once userland support rea= ches some level of
>=C2=A0 =C2=A0 =C2=A0 =C2=A0maturity.
>
> My first goal will be to get a better understanding of
> src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll = also
> study your work, Alan. I really appreciate the time you have taken. I<= br> > might reach out to you and Warner directly for further questions.

Relying on an external toolchain and allowing for future external
toolchains other than Rust sounds like a really good plan.
=

It's the only= way it could work. Importing rust with its current level of maturity and l= ag is logistically impossible or nearly so.rust doesn't have a long eno= ugh support timeline to work well with our stable branches. Llvm is tricky = enough...=C2=A0

Conceivably we could even ditch our Lua import and use the same
mechanism for that (but please, save Lua discussion for a separate
flame war ;) ).=C2=A0

<= div dir=3D"auto">That can't possibly work since we build it into the lo= ader with changes that are unavoidable. There's no flame war, it's = just impossibile. The situation is entirely different.=C2=A0

But the lua import is the easiest of = all the vendor imports I do.

Warner=C2=A0


I anticipate that the next big decisions will be
"what components are so important that they musn't require an exte= rnal
toolchain?" and "how much cargo crate bloat is too much?".= =C2=A0 But we can
cross those bridges when we come to them.=C2=A0 I look forward to seeing your work, and please don't hesitate to ask for help.
<= /div>

Managing these thi= ngs in a FreeBSD context will be challenging and i look forward to the resu= lts we get.

Warner=C2=A0=

-Alan

--000000000000bdad180620d8fcb2--