Re: A Demo of rust-in-base

On Thu, Aug 29, 2024, 2:55 PM Alan Somers <asomers@freebsd.org> wrote:

> On Thu, Aug 29, 2024 at 2:21 PM Shawn Webb <shawn.webb@hardenedbsd.org>
> wrote:
> >
> > Hey Alan et al,
> >
> > I apologize for the silence on my end. It has been a busier two months
> > than anticipated. In that time, I've been entertaining some thoughts
> > on Rust in base support. ${LIFE} is starting to calm down again, and I
> > do believe I'll be able to start the research in time in September. I
> > will be splitting my free time between this and studying for my OSCP
> > cert.
> >
> > So, to those thoughts, in list form (in no particular order):
> >
> > 1. Use of Rust compiler toolchain support will be for userland
> >    components in an opt-in fashion. Meaning, all userland components
> >    written in Rust will be optional.
> > 2. It does not make sense to perform a vendor import of the Rust
> >    compiler toolchain and standard libraries. All Rust code in the src
> >    tree must be built from an external toolchain.
> > 3. I believe the notion of an external toolchain could be abstracted
> >    such that we can support any optional userland component written in
> >    a language supported by that external toolchain. This would imply
> >    that other alternative languages could be supported with minimal
> >    work (Zig, TypeScript, Python, Java, etc.)
> > 4. We could provide auto-detection mechanisms for determining which
> >    external toolchains are available, their language support, etc. The
> >    initial proof-of-concept would likely be limited to Rust to save on
> >    time and complexity.
> > 5. As the work matures, and perhaps as a requisite for eventual
> >    inclusion, we could land support for more than Rust. This might be
> >    a step too far, but hey, it's one of the thoughts I had.
> > 6. So all of this wrapped up means that:
> >    A. This is NOT a call to rewrite everything in Rust. This work will
> >       only permit NEW, OPTIONAL components to be written.
> >    B. Other languages/toolchains/ecosystems could be supported, not
> >       just Rust.
> >    C. Initial focus is on userland components. Rust in the kernel is
> >       out of scope for this initial proof-of-concept.
> >    D. I would like to see Rust in the kernel. That would be a good
> >       next area of focus once userland support reaches some level of
> >       maturity.
> >
> > My first goal will be to get a better understanding of
> > src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
> > study your work, Alan. I really appreciate the time you have taken. I
> > might reach out to you and Warner directly for further questions.
>
> Relying on an external toolchain and allowing for future external
> toolchains other than Rust sounds like a really good plan.
>

It's the only way it could work. Importing rust with its current level of
maturity and lag is logistically impossible or nearly so.rust doesn't have
a long enough support timeline to work well with our stable branches. Llvm
is tricky enough...

Conceivably we could even ditch our Lua import and use the same
> mechanism for that (but please, save Lua discussion for a separate
> flame war ;) ).


That can't possibly work since we build it into the loader with changes
that are unavoidable. There's no flame war, it's just impossibile. The
situation is entirely different.

But the lua import is the easiest of all the vendor imports I do.

Warner


I anticipate that the next big decisions will be
> "what components are so important that they musn't require an external
> toolchain?" and "how much cargo crate bloat is too much?".  But we can
> cross those bridges when we come to them.  I look forward to seeing
> your work, and please don't hesitate to ask for help.
>

Managing these things in a FreeBSD context will be challenging and i look
forward to the results we get.

Warner

-Alan
>
>