Re: Building a Linuxulator userland from source

Amazing work. Thanks Felix!


On 8/17/23 23:23, Felix Palmen wrote:
> Hi all,
> 
> for the last two weeks, I've been working on a spike in ports which now
> reached a state where I want to show it to and discuss it with fellow
> ports hackers.
> 
> First, a link to my feature branch (warning, will be rebased every now
> and then):
> <https://github.com/Zirias/zfbsd-ports/commits/linux>
> 
> The goal is to create a replacement for the now antiquated linux-c7
> userland. While the classic approach would be to find another Linux
> distribution that's not too much of a moving target and start
> "repackaging" that, I want to try something different: Build the
> required packages from source.
> 
> ** Why
> 
> It will be quite some work to do this, I'm not really sure about it yet
> (and how it would compare to the repackaging approach), so feasibility
> is yet to be decided. But I hope to get at least these two advantages:
> 
> - Provide the newest GNU libs (glibc, libstdc++, ...) built against
>   exactly the Linux version emulated by the FreeBSD version this will
>   run on. This should make it possible to run a lot more Linux binaries
>   without relying on e.g. Linux jails.
> - When binaries don't work for missing Linux libraries, make it somewhat
>   easy to add them, maybe based on already existing FreeBSD ports.
> 
> ** State
> 
> I just reached a state where I can build a working Linux-native GNU
> toolchain (binutils, glibc, gcc) for C and C++ on aarch64, amd64 and
> i386. From here on, it should be simpler, there are already two ports in
> my branch (archivers/linux-bzip2 and archivers/linux-xz) using that
> native toolchain for building.
> 
> ** How
> 
> The native toolchain is built by a cross toolchain, the packages for
> this cross-toolchain are prefixed "lxcross-". For building this cross
> toolchain, bootstrapping versions of binutils and gcc are needed to
> build the initial glibc, these versions are suffixed "-bootstrap".
> 
> lxcross ports set PREFIX to ${LXCROSSBASE}, which defaults to
> ${LOCALBASE}/linux-cross. lxcross-*-bootstrap ports set PREFIX to
> ${LXBOOTSTRAP}, this one defaults to ${LXCROSSBASE}/bootstrap.
> 
> ** Open issues
> 
> This is an unordered list off my head, so most likely incomplete.
> 
> - Some trickery with PREFIX is currently needed. The ports framework
>   expects PREFIX to be used as is by the upstream build system. This
>   won't hold for building Linux packages, PREFIX must be /compat/linux
>   for that, but passed to the upstream build system in DESTDIR.
> - LIB_DEPENDS don't work, which could probably be solved in the
>   framework. Right now, I'm using a hacky workaround to define
>   LINLIB_DEPENDS and add it to both RUN_ and BUILD_DEPENDS.
> - A lot of smaller things that *should* be provided by the framework,
>   some of them probably by USES=linux, are currently copy&pasted to
>   every port needing them. I wanted to keep it simple while first trying
>   to get it to work, so the framework isn't touched yet at all.
> - Some stage-qa checks get confused, some (e.g. checking that everything
>   is stripped) don't work.
> - In my tests, "poudriere testport" failed at least on i386, because it
>   mounts linprocfs on /compat/linux/proc and then tries to remove
>   /compat/linux (remove pre-existing PREFIX). To test the ports, I had
>   to slightly modify the testport script for now.
> - For the Linux headers, there should be a metaport picking the Linux
>   version based on ${OSVERSION}. This doesn't exist yet, Linux 4.4.x is
>   always used.
> - Building the final linux-gcc ports, I get weird error messages
>   directly to poudriere's terminal (they do NOT appear in the build
>   log!) like this:
>     ELF interpreter /usr/lib/ld-linux.so.2 not found, error 2
>   I have no idea where this comes from, so far I couldn't identify any
>   negative effect though.
> 
> Acknowledgement: I found quite some useful info for doing this in the
> "Linux from Scratch" book. Of course you can't just follow the book
> (very different scenario, it assumes building on Linux and not doing any
> staging/packaging), but it *does* have some helpful hints.
> 
> Cheers, Felix
>