Re: The Case for Rust (in the base system)

On Sat, Jan 20, 2024 at 09:14:59PM -0700, Alan Somers wrote:
> On Sat, Jan 20, 2024, 8:44 PM Warner Losh <imp@bsdimp.com> wrote:
> 
> >
> >
> > On Sat, Jan 20, 2024, 7:20 PM Alan Somers <asomers@freebsd.org> wrote:
> >
> >> On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI <junchoon@dec.sakura.ne.jp>
> >> wrote:
> >> >
> >> > On Sat, 20 Jan 2024 15:31:23 -0700
> >> > Warner Losh <imp@bsdimp.com> wrote:
> >> >
> >> > > On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
> >> wigneddoom@yandex.ru>
> >> > > wrote:
> >> > >
> >> > > > What about external dependencies?
> >> > > >
> >> > > >
> >> https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
> >> > > >
> >> https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
> >> > > >
> >> > > > Is there any plan for which crates we should take into the base
> >> system?
> >> > > >
> >> > > > We have had C++ in base for many years, but I don’t see any good
> >> libraries
> >> > > > for CLI, logging, JSON, etc.
> >> > > >
> >> > > >
> >> > > >
> >> https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
> >> > > >
> >> > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V,
> >> > > > Power? Should we rewrite devd?
> >> > > >
> >> > > > I think we need to start by providing official repositories (e.g
> >> > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
> >> > > > for different languages that include stable bindings to the system
> >> API:
> >> > > > - sysctl
> >> > > > - libgeom
> >> > > > - libifconfig
> >> > > > - netgraph
> >> > > > - jail
> >> > > > - etc.
> >> > > >
> >> > > > So that it’s not just some anonymous on crates.io that represents
> >> these
> >> > > > bindings, but our community.
> >> > > > Officially, with support for a stable ABI for releases, security
> >> patches,
> >> > > > etc.
> >> > > >
> >> > > > After this, it will be possible to think about which components to
> >> include
> >> > > > in the base system.
> >> > > >
> >> > > > I would be glad to see a more modern language than C in the
> >> database, but
> >> > > > I’m afraid that it will be like with C++,
> >> > > > that we will get a couple of daemons and utilities and that’s all.
> >> > > >
> >> > >
> >> > > These are all good questions that need good answers, though
> >> necessarily to
> >> > > get started.
> >> > >
> >> > > But the other question that occured to me after my last posting was
> >> "What
> >> > > about build integration?"
> >> > > How much of the rust automation do we take in vs how much do we drive
> >> from
> >> > > a future bsd.rust.mk.
> >> > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
> >> need
> >> > > one for what we traditionally
> >> > > think of as libraries (which may or may not map 1:1 onto crates: we
> >> could
> >> > > have c callable libraries
> >> > > written in rust in the future, for example) and one for binaries.
> >> > > Initially, though, if we go with the
> >> > > 'make rust tests possible' then we'd likely need the appropriate
> >> packages
> >> > > installed for whatever
> >> > > dependencies we'd have in the tests. This would give us a taste for
> >> what
> >> > > we'd need to do for
> >> > > base, I'd think. Once we had that notion, I can easily see there
> >> needing to
> >> > > be some sort of
> >> > > rust bindings for ATF/kyua as one of the first libraries / crates that
> >> > > would test that aspect of
> >> > > the build system. That all would be up to the people writing the
> >> tests in
> >> > > rust, I'd imagine.
> >> > >
> >> > > While I could jot out the basics of this integration (so one could
> >> just add
> >> > > the rust
> >> > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever
> >> and then
> >> > > it would build
> >> > > if rust is enabled, and would emit a warning it was skipped because
> >> rust
> >> > > was disabled).
> >> > > We'd find out if this is workable or not and iterate from there. But
> >> that
> >> > > would also require
> >> > > active participation from the rust advocates to make it a reality: I
> >> can
> >> > > put together the
> >> > > build infrastructure for the disabled case, but likely can't on my
> >> own do
> >> > > the rust enabled
> >> > > case. I'd be happy to work with someone to do that, but I'm not going
> >> to be
> >> > > able to do
> >> > > that myself: my need for rust is slight, my knowledge of rust is
> >> weak, etc.
> >> > > Working with
> >> > > someone (or ideally several someones), though it could become
> >> reality. So
> >> > > please contact
> >> > > me if you'd like to work on this.
> >> > >
> >> > > Warner
> >> >
> >> > One way to go could be moving programs rewritten with rust to ports.
> >> > There are some programs (not in rust, though) moved to ports, like rcs.
> >>
> >> I've already done this with a few, though I didn't delete the C
> >> versions from base.
> >> usr.bin/gstat => sysutils/gstat-rs
> >> tools/regression/fsx => devel/fsx
> >>
> >
> > So
> > % size `which gstat-rs` `which gstat`
> >      text     data   bss       dec        hex   filename
> >   2094442   176472   568   2271482   0x22a8fa   /usr/local/sbin/gstat-rs
> >     19350     1180    41     20571     0x505b   /usr/sbin/gstat
> > % file `which gstat-rs` `which gstat`
> > /usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64,
> > version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
> > FreeBSD-style, stripped
> > /usr/sbin/gstat:          ELF 64-bit LSB pie executable, ARM aarch64,
> > version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
> > for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
> > 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
> > /usr/local/sbin/gstat-rs:
> > libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
> > libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
> > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
> > libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
> > libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
> > libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
> > /usr/sbin/gstat:
> > libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
> > libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
> > libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
> > libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
> > libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
> > libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
> > libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
> > libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
> > libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
> > libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)
> >
> > So that looks scary, like rust is 100x larger binaries...  But at runtime
> > it's about the same:
> > USER    PID   %CPU %MEM   VSZ   RSS TT  STAT STARTED         TIME COMMAND
> > imp   14735    0.0  0.0 14140  4828  0  S+   20:38        0:00.04 gstat
> > imp   14766    1.3  0.0 15772  6256  0  S+   20:39        0:00.02 gstat-rs
> >
> > So the runtime size is at least in the same ballpark (still larger, but
> > not crazy larger). More CPU too,
> > but that's just a polling artifact I think (other times gstat had some,
> > and gstat-rs didn't).
> >
> > Why is the rust binary so much larger? Are the rust runtime and
> > dependencies statically linked?
> >
> 
> Yes, that's a large part of it. Rust libraries are usually statically
> linked (though they don't have to be). For example, in the output above,
> notice that gstat-rs does not link to ncurses. That's because the
> equivalent library is statically linked in instead.
By 'equivalent' do you mean crossterm crate?
In other words, gstat-rs does not use [n]curses, am I right?

Looking at the list of supported terminals, it sounds as if crossterm
just hardcodes xterm control sequences.


> Also, rust programs by
> default include goodies like stack unwinding on panic, which takes extra
> code too. But that can be turned off if you really want to save space.
> -Alan
> 
> >