Re: Building a Linuxulator userland from source

In message <xcztahm3vu3bjghjqqxuoy2xabyjmyfq22jw6mkaaaqo7wa36s@fdq7dlvpuhlk>
, F
elix Palmen writes:
> Hi all,
>
> for the last two weeks, I've been working on a spike in ports which now
> reached a state where I want to show it to and discuss it with fellow
> ports hackers.
>
> First, a link to my feature branch (warning, will be rebased every now
> and then):
> <https://github.com/Zirias/zfbsd-ports/commits/linux>
>
> The goal is to create a replacement for the now antiquated linux-c7
> userland. While the classic approach would be to find another Linux
> distribution that's not too much of a moving target and start
> "repackaging" that, I want to try something different: Build the
> required packages from source.
>
> ** Why
>
> It will be quite some work to do this, I'm not really sure about it yet
> (and how it would compare to the repackaging approach), so feasibility
> is yet to be decided. But I hope to get at least these two advantages:
>
> - Provide the newest GNU libs (glibc, libstdc++, ...) built against
>   exactly the Linux version emulated by the FreeBSD version this will
>   run on. This should make it possible to run a lot more Linux binaries
>   without relying on e.g. Linux jails.
> - When binaries don't work for missing Linux libraries, make it somewhat
>   easy to add them, maybe based on already existing FreeBSD ports.
>
> ** State
>
> I just reached a state where I can build a working Linux-native GNU
> toolchain (binutils, glibc, gcc) for C and C++ on aarch64, amd64 and
> i386. From here on, it should be simpler, there are already two ports in
> my branch (archivers/linux-bzip2 and archivers/linux-xz) using that
> native toolchain for building.
>
> ** How
>
> The native toolchain is built by a cross toolchain, the packages for
> this cross-toolchain are prefixed "lxcross-". For building this cross
> toolchain, bootstrapping versions of binutils and gcc are needed to
> build the initial glibc, these versions are suffixed "-bootstrap".
>
> lxcross ports set PREFIX to ${LXCROSSBASE}, which defaults to
> ${LOCALBASE}/linux-cross. lxcross-*-bootstrap ports set PREFIX to
> ${LXBOOTSTRAP}, this one defaults to ${LXCROSSBASE}/bootstrap.
>
> ** Open issues
>
> This is an unordered list off my head, so most likely incomplete.
>
> - Some trickery with PREFIX is currently needed. The ports framework
>   expects PREFIX to be used as is by the upstream build system. This
>   won't hold for building Linux packages, PREFIX must be /compat/linux
>   for that, but passed to the upstream build system in DESTDIR.
> - LIB_DEPENDS don't work, which could probably be solved in the
>   framework. Right now, I'm using a hacky workaround to define
>   LINLIB_DEPENDS and add it to both RUN_ and BUILD_DEPENDS.
> - A lot of smaller things that *should* be provided by the framework,
>   some of them probably by USES=3Dlinux, are currently copy&pasted to
>   every port needing them. I wanted to keep it simple while first trying
>   to get it to work, so the framework isn't touched yet at all.
> - Some stage-qa checks get confused, some (e.g. checking that everything
>   is stripped) don't work.
> - In my tests, "poudriere testport" failed at least on i386, because it
>   mounts linprocfs on /compat/linux/proc and then tries to remove
>   /compat/linux (remove pre-existing PREFIX). To test the ports, I had
>   to slightly modify the testport script for now.
> - For the Linux headers, there should be a metaport picking the Linux
>   version based on ${OSVERSION}. This doesn't exist yet, Linux 4.4.x is
>   always used.
> - Building the final linux-gcc ports, I get weird error messages
>   directly to poudriere's terminal (they do NOT appear in the build
>   log!) like this:
>     ELF interpreter /usr/lib/ld-linux.so.2 not found, error 2
>   I have no idea where this comes from, so far I couldn't identify any
>   negative effect though.
>
> Acknowledgement: I found quite some useful info for doing this in the
> "Linux from Scratch" book. Of course you can't just follow the book
> (very different scenario, it assumes building on Linux and not doing any
> staging/packaging), but it *does* have some helpful hints.
>
> Cheers, Felix

Basically this would become another Linux distro, albeit a virtual one that runs under our Linuxulator.

Avoiding discussion about packaging -- we can package this any way we wish -- how will this support software written for distro A, B, or C. For example, Red Hat software doesn't neccesarily run on SuSE or Ubuntu because shared library dependencies may be different.

Building our own "distro" to run under the Linuxulator may require a complete set of packages and end-user applications because existing Linux software may require a Fedora, Debian or Red Hat library. Wouldn't this negate the need for a Linuxulator because a person can build most Linux software to run on native FreeBSD.

I think a better path might be to support multiple Linux userlands in parallel. Thus a user could simply copy or install vendor software for a Red Hat in one environment and a SuSE vendor software in another.

If the intention is to duplicate what a distro or different distros do, this would seem like a lot of extra work for little or no signficant advantage.

If the decision is to pursue this, we should also continue to support various other distributions binary packages so that vendor software built to run on distro A can run under FreeBSD's Linuxulator. Linux cross distro dependencies is a minefield and adding a "virtual" distro IMO only adds to the problem. I have to deal with this at $JOB. End-users don't understand it because to non-technical Linux end users, like management, Linux is a homogeneous thing.


-- 
Cheers,
Cy Schubert <Cy.Schubert@cschubert.com>
FreeBSD UNIX:  <cy@FreeBSD.org>   Web:  https://FreeBSD.org
NTP:           <cy@nwtime.org>    Web:  https://nwtime.org

			e^(i*pi)+1=0