Re: It's not Rust, it's FreeBSD (and LLVM)

On Tuesday, September 10th, 2024 at 2:47 AM, David Chisnall <theraven@FreeBSD.org> wrote:

> On 9 Sep 2024, at 23:12, Rick Macklem <rick.macklem@gmail.com> wrote:
>
>> As for build times, I already find "make buildworld" takes way too long
>> for me, so adding another ginormous compiler won't make much difference.
>
> As with so many other things, this would be addressable if someone had the time to do it. A single developer at Microsoft can edit the code and build their own custom Windows VM image on a fairly low-end laptop, in spite of the fact that most laptops don’t have enough disk space for a full git clone of the Windows source tree. This is possible because of a combination of two things:
>
> - git-vfs lets you have a local view of a remote git repo and fetches files on demand as they’re accessed. Don’t run `grep` over your source tree!
> - A build system that has cloud caching, so if you have changed one file you’ll rebuild everything that depends on that file but download the artefacts for everything else.
>
> It’s a bit sad that building Windows typically takes a developer less time than building FreeBSD. The engineering effort involved in the second part is huge though. It would probably need the Foundation to pay someone to work on it for a year (I think this would be well worth the investment, but that’s another discussion).

I have written a build system that I used to want FreeBSD to adopt someday, and it will have remote build caching. However, after looking at how FreeBSD does the build, I'm not sure how much build caching will help. But I am a new user, not an expert, so I could be wrong.

What I found is that the src tree is compiled into /usr/obj. I presume object files from previous compiles allow for incremental rebuilds. I think we would have to measure how much incremental compiles help the build times, but I suspect that build caching won't help simply because if a file changes, it and its downstream dependents will be built anyway, build caching or not.

The one place it might help is a full rebuild. I'm not optimistic that full rebuilds happen that often. Even if they do, would it be worth touching the network during a build?

Now as for the engineering effort, yes, it would require a lot, but there's only two hard part: making the build system hermetic (although that can easily be done nonportably using jails) and making the build [reproducible](https://reproducible-builds.org/). The rest of it is just tedious busywork, at least if the build system is designed correctly.

One of those tedious jobs is to essentially reimplement all of the build system (talking about the Makefiles, not bmake) once the new build system (bmake replacement) is ready. And then, even when that's done, there will be a day when either both sets of build scripts are supported, or everyone on -CURRENT just switches overnight.

In other words, replacing the build system would be disruptive. Even though I have one that I'd like FreeBSD to use, I actually decided against continuing that effort.

> I’m not sure if bmake can be extended enough to support this

It probably can, but I think there are easier paths.

> (Buck2 looks to be the most promising of the new build systems designed around this model but it hasn’t yet had a stable release and comes with big ‘do not use this yet’ warnings).

Yes, Buck2 is the most promising, but it's written in Rust, so if Buck2 were adopted, Rust would have to become part of src or an external toolchain.

That's one of the hardest things about build systems: they are a dependency for everything, but they have dependencies too.

> This would also need infrastructure to make sure each commit to head or a stable branch was built in a trusted environment and pushed to the caches (and that random developers’ builds did not).

Completely agree. But if FreeBSD has CI/CD, then that could be what pushes. If not, then adding the build caches would be a good excuse to add it.

However, the reason Buck2 works so well is because it does take builds from random developers and pushes them to the build caches. It can do this because the "random" developers are not truly random, but Facebook employees. And it builds those random developers' stuff remotely in sandboxes.

If FreeBSD's build caches won't work like that, I'm not sure how useful they would be because developers who would pull from others for testing and review would all have to build the same files. But if FreeBSD devs could push to the build cache, or even better, have the server build their changes for everyone else, I think that would be the most effective time savings for devs. Just building HEAD or -STABLE would not save much, IMO.

Gavin Howard