Re: The Case for Rust (in the base system)

On Wed, Jul 31, 2024 at 8:37 AM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
>
> On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote:
> > In a recent thread on src-committers, we discussed the costs and
> > benefits of including Rust code in the FreeBSD base system.  To
> > summarize, the cost is that it would double our build times.  imp
> > suggested adding an additional step after buildworld for stuff that
> > requires an external toolchain.  That would ease the build time pain.
> > The benefit is that some tools would become easier to write, or even
> > become possible.  Here is a list of actual and potential Rust projects
> > that could benefit from being in-tree.  If anybody else has items to
> > add, I suggest moving this into the project wiki:
> >
> > Stuff that could only be written in Rust if it were in base
> > ===========================================================
> >
> > * ctl-exporter (I started this, but discovered that the CTL stats API is
> >   unstable, so it can't live in ports.  Instead, I had to do it in C).
> >   https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
> >
> > * fusefs tests.  Absolutely impossible to do in C.  I considered Rust, but went
> >   with C++ so they could live in base.  They are too closely coupled to
> >   fusefs(5) to live out-of-tree.
> >   https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
> >
> > * devd.  Currently C++, but imp suggested a rewrite.
> >   https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
> >
> > * zfsd.  Currently C++, but I've long pondered a rewrite.  Using Rust would
> >   make it more testable.
> >   https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
> >
> > * nscd.  Currently C, but confusing and with no test coverage.  I've
> >   contemplated a rewrite myself, but I don't want to do it in C.
> >   https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
> >
> > * The userland portion of the 802.11ac and Lightning stacks.  scottl suggested
> >   that these were good candidates for Rust.
> >
> > * freebsd-kpi-r14-0 .  https://crates.io/crates/freebsd-kpi-r14-0
> >
> > Stuff that can live in ports, but would be nicer in base
> > ========================================================
> >
> > * gstat-rs https://crates.io/crates/gstat
> >
> > * geom-exporter (I've started this, but haven't published it)
> >
> > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
> >
> > * virtiofsd-rs .  Nobody has yet tried to port it to FreeBSD.  But if the
> >   connection to bhyve(8) is too intimate, it might be hard to do in ports.
> >   https://gitlab.com/virtio-fs/virtiofsd
> >
> > * jail-exporter https://crates.io/crates/jail_exporter
> >
> > * Various jail managers have been attempted in Rust.  I think these are fine in
> >   ports, but others like Goran Mekic have opined that they should be moved to
> >   base instead.
> >
> > * musikid's pjdfstest rewrite.  I think it would be great to start using this
> >   to test the base system's file systems.  If the tests themselves lived in
> >   base, they would be easier to sync with file system development.
> >   https://github.com/musikid/pjdfstest
> >
> > * pf-rs.  I suspect that the API isn't very stable.
> >   https://crates.io/crates/pf-rs
> >
> > * benchpmc.  The pmc counter names changes between releases.
> >   https://crates.io/crates/benchpmc
> >
> > FreeBSD-related applications that are just fine in ports
> > =========================================================
> >
> > * fsx-rs.  Unlike pjdfstest, this only tests datapath APIs.  Those are usually
> >   more stable than control path APIs, so I think there's little to be gained by
> >   moving this into base. https://crates.io/crates/fsx
> >
> > * ztop.  It uses ZFS's kstats sysctl interface, which is pretty stable.
> >   https://crates.io/crates/ztop
> >
> > * iocage-provision  https://crates.io/crates/iocage-provision
> >
> > * rsblk https://crates.io/crates/rsblk
> >
> > * xfuse  https://github.com/KhaledEmaraDev/xfuse
> >
> > Other FreeBSD-related libraries in Rust
> > =======================================
> > Just see the list at https://crates.io/keywords/freebsd
> >
>
> One new data point: DARPA is looking to rewrite a significant amount
> of C code to Rust with their "Translating All C to Rust (TRACTOR)"
> project:
> https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view

Interesting.  And since you bring it up, I have two new data points myself:

* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing.  So I rewrote that part in
Rust, just to help my understanding.  Later, I rewrote the XML
parsing, too.  Then I rewrote the LUN creation and deletion, just to
see how hard it would be.  All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too.  Config file parsing
is more consistent, no memory leaks, etc.  Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.

* geom-exporter: I mentioned this project up above.  It's finished
now, and you can find it in ports at net-mgmt/geom-exporter .