Re: It's not Rust, it's FreeBSD (and LLVM)

Am 2024-09-03 17:32, schrieb Poul-Henning Kamp:
> What is FreeBSD ?
> -----------------
> 
> Forget Rust for the moment, I promise I will come back to it.
> 
> FreeBSD as a project was created almost entirely as protest against
> the incompetent "UNIX-industry" as it existed around 1990.
> 
> One of the stupidities we reacted against, was the unbundling of
> the C-compiler:  If you bought a UNIX system, the C-compiler would
> cost you extra, even through everybody knew that the vendor got it
> as part of their source license from AT&T.
> 
> So from the very start, FreeBSD decided to deliver "A complete UNIX
> system with full source" and the only concession was that, hard
> disks costing what they did, you could choose to not install the
> manual pages and the source code on systems which did not need them.

We are waaaay past that. The world is not only ftp and smtp servers 
anymore.

> The source tree became our citadel: "FreeBSD is src".  If something
> was not in src, it was not FreeBSD.

We are way past that too, FreeBSD is src+ports+docs(+community).

[...]
> But a FreeBSD system recompiling itself from source is even rarer.

In your world. And in the world of some other people. But there are a 
lot of worlds where this is not true. I have systems which are updated 
from src, and use only packages which are build locally.

> And when it does, LLVM, source code we import verbatim from an
> entirely different project, and which no sane person would call
> "Related to FreeBSD", takes up more than three quarters of the
> compile time.
> 
> That is objectively absurd.
> 
> The only reason we do that, is because we stil have that outdated
> "FreeBSD is src" emotional hangup.

I don't think so. I'm sure everyone agrees that FreeBSD is not only src. 
You already told it yourself. It's src+ports+pkg + other add-ons which 
all live in the FreeBSD ecosystem.

> We need to find a contemporary and useful answer to "What is FreeBSD?"
> 
> The only answer I can think of
> ------------------------------
> 
> "FreeBSD is ports (some of those ports contain the kernel and 
> userland)"

I propose "FreeBSD is packages" (and at the same time it isn't, but all 
of the before mentioned stuff). How those packages are generated doesn't 
matter, this is an implementation detail. "make buildworld" can create 
packages. "make installworld" can install those packages. "make 
distribpkg" can copy the basepackages to a directory structure similar 
to what poudriere does which would allow to use "make whatever" in src 
and poudriere to generate a package set for your site, your product, or 
only yourself if you want to. It also would allow to continue using 
finger memory, and have those which which to do so update the stuff via 
the old way instead of using pkg (while having it installed via pkg in 
the background).

> As part of the migration, we yank LLVM out of the src.

Together with the possibility to use an external toolchain, this is 
possible. I would welcome it to have a "freebsd-15-llvm-YYYYMMDDnn" port 
which can be build on all supported releases and contains a compiler 
which is able to compile FreeBSD-15 instead of llvm in src. Ideally this 
tollchain port doesn't contain much local patches, so that I could grab 
the source of it on OSX or Linux and use it to build FreeBSD (well... I 
wouldn't, so far I've build FreeBSD only on FreeBSD, but we have 
committers which use this, so we should keep that in mind when we create 
such a port).

This external toolchain support should be generic (and I cross fingers 
that Shawn has the time to come up with something sensible / 
extensible). In that case the discussion about which language to use 
doesn't matter anymore. It would allow up the possibility to have the 
non-optional stuff managed in src, and optional stuff managed in ports. 
It would allow to play around with different languages and to provide 
replacements in different languages in parallel. This means that any 
alternative is able to prove itself. Without the bikesheeding of 
theoretical stuff, but by presenting evidence.

> LLVM does not belong in src by any sane criteria, and any microscopic
> benefits of "tight integration" can be delivered with a 
> "toolchain-llvm"
> (meta-)port.
> 
> Our minimal install images will contain:
> 
> 	The installer
> 	The kernel package(s)
> 	The userland package(s)

An implementation detail which only matters once we have the possibility 
to chose from packages...

> "pkg upgrade" also upgrade kernel and userland packages - Welcome to
> the century of the fruitbat.

I still think "make installworld" should be able to update my system 
(and I should not care if this is done by using pkg or not, e.g. by 
pointing pkg to the buildworld-generated packages repo in OBJDIR).

A more generic out of the box view than what PHK started follows now, 
not directed at PHK directly but to all of us:

I do not think this is all what is needed. This proposal is surely a 
step into the (IMO) right direction, but we should also discuss where we 
stand with what we have already and where we should go with that.

flua is there, but there is no integration of what we have (recently 
there started a discussion to add some freebsd modules to make freebsd 
basesystem stuff accessible from flua, which seems to go into a (IMO) 
good direction).

Similar for C++. We have some C++ code in the tree, but we have no guide 
for it. Not only in terms of extensions to our style-guide, but also in 
terms of what we want to see in terms of good practices. C++ is complex, 
had a lot of changes since I first seen it in 1999. IMO we should use 
the most recent C++ standard which our oldest supported release is able 
to compile (to be reviewed every time a release is EoL resp. with every 
major update of the external toolchain for the oldest release we 
support). And we should provide some guidelines of what we want to see 
in our C++ code (as a very basic example: use smart pointers instead of 
raw pointers, or X instead of Y). Some of this may be common knowledge, 
or best practices, but IMO we should have a document of what we want to 
see. We don't have that for C either, but I think we should have that 
for C too. We want to attract new people, and something like that would 
make it more easy for new people to join. Similar for our kernel 
interfaces. We have several of them, but we have no document which 
explains which kind of interface to use for which kind of stuff. And we 
have multiple ways to do something in the kernel, some of them old stuff 
replaced by new stuff in some places but not all places, and a document 
which talks about those parts (e.g. new code should use X instead of Y, 
or if you touch it, please also do a migration to Y at the same time).

I bring this up here, because this is part of our code debt (and there 
is surely other stuff which should be included too). And every 
discussion about C vs C++ vs language of the day is implicitly a 
discussion about code debt. All the arguments about we don't need more 
than C are valid. At the same time all the arguments about the need for 
<insert your favorite language> are valid too. In the end it's about 
code debt, about cognitive load, code quality assurance and so on, not 
about the language itself. While we gain experience, we all don't get 
younger, may have not slept well last night, may have a hangover, 
medical issues, have a broken heart, or be distracted. Anything which 
makes it more easy to produce a good result is important, and as a 
project we don't have the luxury to be able to say "we always did it 
like this". We are proud to provide possibilities, but not policies on 
our OS. We should be open minded enough because of that to provide the 
possibility to let people use any language, not only in userland, but 
also the kernel. If someone comes up with a subset of C++ to be used in 
the kernel, why not? If this needs a bit of support in src, why not? The 
same for rust or any other language. We need to draw a sane line 
somewhere, sure, but we should not blindly say no to it. Someone wants 
to prove that language X is a good addition to the kernel and it needs 
2k lines (an arbitrary number I just picked out of thin air, don't nail 
me down on this number) of changes to make it possible which can not be 
done as a module and does not come with a too big cost, why not (mark it 
experimental, make it an option, ...)? In the end any touring complete 
language with enough low level possibilities is suitable to write an OS, 
and if the language of day is suitable or not, can be seen by letting it 
proof itself in this regard. If it isn't suitable, no important new 
stuff will be produced and it will vanish. If it is suitable to an 
extend never imagined, it will take over the world and old stuff is 
replaced. Time will tell, but nothing bad will happen if we allow the 
possibility to give it a try and make it even easy to give it a try.

Bye,
Alexander.

-- 
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org    netchild@FreeBSD.org  : PGP 0x8F31830F9F2772BF