Re: The Case for Rust (in the base system)
Date: Sat, 20 Jan 2024 18:37:25 UTC
On Sat, Jan 20, 2024, 9:51 AM Alan Somers <asomers@freebsd.org> wrote: > In a recent thread on src-committers, we discussed the costs and > benefits of including Rust code in the FreeBSD base system. To > summarize, the cost is that it would double our build times. imp > suggested adding an additional step after buildworld for stuff that > requires an external toolchain. That would ease the build time pain. > The benefit is that some tools would become easier to write, or even > become possible. Here is a list of actual and potential Rust projects > that could benefit from being in-tree. If anybody else has items to > add, I suggest moving this into the project wiki: > While a good list, it's just a list. Rust is interesting but without experience in the FreeBSD context we won't know if it materially improves things. The why of Rust is currently somewhere between hype and reality. Those that love it, love it. Those that don't, don't see how this fad is different from all the other fads. That's part of why it's hard to sell: the rust toolchain is difficult and different; it can do things automatically with crates that are insecure and unrepeatable; etc. With that, though comes a lot of benefits in safety of the language in more ways than just security. What is needed to convince people it's not just a fad is to bring some things into the tree that aren't critical to the tree, per se, but that can let people opt in to see just how bad the downside is in a buildworld context and just what the upsides are in practice. If it really is a good fit, it will sell itself... So this isn't about what could be rust: a lot can be. But what's the point of rewriting things that haven't had security bugs that rust prevents? The benefits of using rust have to be documented and demonstrated rather than just hyped, asserted and sold. That's why I suggested we start with better tests in rust. That's the petfect way to make sure we get the external tool chain working. We get the integration right. We fight with whatever pain there is and fix it as needed. We learn if it actually fits in and if we can keep up the infrastructure or if that's not something we can do. Once that works, we can rewrite other things as the need or desire arises. So no, we aren't going to rush headlong into fullscale rust replacement of all the things. However I'll note Linux hasn't done that either. They are taking a one step at a time approach. Step one is build integration with an external toolchain, imho. Until we have that, 0 Rust will be in the system. We already can do ports... but if you want it more central the first steps will be in making it possible to build rust binaries at all in a repeatable way. Warner Stuff that could only be written in Rust if it were in base > =========================================================== > > * ctl-exporter (I started this, but discovered that the CTL stats API is > unstable, so it can't live in ports. Instead, I had to do it in C). > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34 > > * fusefs tests. Absolutely impossible to do in C. I considered Rust, but > went > with C++ so they could live in base. They are too closely coupled to > fusefs(5) to live out-of-tree. > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > > * devd. Currently C++, but imp suggested a rewrite. > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would > make it more testable. > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd > > * nscd. Currently C, but confusing and with no test coverage. I've > contemplated a rewrite myself, but I don't want to do it in C. > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > * The userland portion of the 802.11ac and Lightning stacks. scottl > suggested > that these were good candidates for Rust. > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > Stuff that can live in ports, but would be nicer in base > ======================================================== > > * gstat-rs https://crates.io/crates/gstat > > * geom-exporter (I've started this, but haven't published it) > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the > connection to bhyve(8) is too intimate, it might be hard to do in ports. > https://gitlab.com/virtio-fs/virtiofsd > > * jail-exporter https://crates.io/crates/jail_exporter > > * Various jail managers have been attempted in Rust. I think these are > fine in > ports, but others like Goran Mekic have opined that they should be moved > to > base instead. > > * musikid's pjdfstest rewrite. I think it would be great to start using > this > to test the base system's file systems. If the tests themselves lived in > base, they would be easier to sync with file system development. > https://github.com/musikid/pjdfstest > > * pf-rs. I suspect that the API isn't very stable. > https://crates.io/crates/pf-rs > > * benchpmc. The pmc counter names changes between releases. > https://crates.io/crates/benchpmc > > FreeBSD-related applications that are just fine in ports > ========================================================= > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are > usually > more stable than control path APIs, so I think there's little to be > gained by > moving this into base. https://crates.io/crates/fsx > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty stable. > https://crates.io/crates/ztop > > * iocage-provision https://crates.io/crates/iocage-provision > > * rsblk https://crates.io/crates/rsblk > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > Other FreeBSD-related libraries in Rust > ======================================= > Just see the list at https://crates.io/keywords/freebsd >