From nobody Wed Jan 12 14:32:12 2022 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B6D3A195DB8E for ; Wed, 12 Jan 2022 14:32:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JYqlJ38yQz4gsX; Wed, 12 Jan 2022 14:32:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4DA4E64D9; Wed, 12 Jan 2022 14:32:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20CEWCT2055673; Wed, 12 Jan 2022 14:32:12 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20CEWCEo055672; Wed, 12 Jan 2022 14:32:12 GMT (envelope-from git) Date: Wed, 12 Jan 2022 14:32:12 GMT Message-Id: <202201121432.20CEWCEo055672@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Edward Tomasz Napierala Subject: git: 66a3dbbdcc - main - handbook: Update the Linuxulator chapter List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: trasz X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 66a3dbbdcc968dc7095b492bc23a9b39731afd32 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1641997932; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RrecWKZ6jrjtFFAJHZzYfczEaEWYFc643wekrwPkjWI=; b=L14ZllihDia3gWSigtOc8+M8WXI9v05wQ5TqyKkNJ+uBXGH0WV20LMXCYAQwFJA90pgDMH n1w2a0zNN+cJPp+4rzwrrtjZJcR4kNFDDIKRIvVlopuO1VQfxiv/p84RS/doF19Ch83yXz rfpHgTGqgZvUjMH+jAls1sL7Kfi0QenhIi5QCaD/r2xWdZUMMGXlOUf3G2xaoz8zPKHNuz 1fnoq7YF+twJtsRd31FlpI3nUk80tEkWt9rYjpJfvd4zjaYW4V5TZhN1Q+m9p/B+61zGQS T4tc7Sz+0pQ9DQLZw3L+OOZvMtkz95ga/uK7AJ4YJixXxSK2CcbZL2n3DowTmw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1641997932; a=rsa-sha256; cv=none; b=IaPzqaArMuLg7rNgWrOORXVkAZK5MRPet9FgFvJXJGnOHT58B3xFFDrxSTuToiClnhRtPI ojhkcn4cYX6lW0FQHmSCX0xjwXc77vSSzA86aVdvtvd82n+Ik+Hi2TKROYmvjoFFbKwoh6 aaXpaM0MhcKCuSk2Z4ntyaBKoCPG/Fgr/5guHxp7iXPdsoy52aXbXyDsaLFQC7CBhZjicv HuRlHFzh2mEuKQXbHulSPyZbqn/MuvbBb/7UH6sqHC+t44czRE1iwr2iVZPOCfCv+0UKty 7rwATdfy8vuZiuj7wDJNl3YkQes2Gl2thBj/iCKmWfA1/MjyBzoAA9GbcpzHow== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by trasz: URL: https://cgit.FreeBSD.org/doc/commit/?id=66a3dbbdcc968dc7095b492bc23a9b39731afd32 commit 66a3dbbdcc968dc7095b492bc23a9b39731afd32 Author: Edward Tomasz Napierala AuthorDate: 2022-01-12 14:30:09 +0000 Commit: Edward Tomasz Napierala CommitDate: 2022-01-12 14:31:25 +0000 handbook: Update the Linuxulator chapter Reviewed By: carlavilla, debdrup, Pau Amma Sponsored By: EPSRC Differential Revision: https://reviews.freebsd.org/D33738 --- .../content/en/books/handbook/linuxemu/_index.adoc | 131 ++++++++++++--------- 1 file changed, 76 insertions(+), 55 deletions(-) diff --git a/documentation/content/en/books/handbook/linuxemu/_index.adoc b/documentation/content/en/books/handbook/linuxemu/_index.adoc index aec5db21b5..672a4d74dd 100644 --- a/documentation/content/en/books/handbook/linuxemu/_index.adoc +++ b/documentation/content/en/books/handbook/linuxemu/_index.adoc @@ -1,9 +1,9 @@ --- -title: Chapter 10. Linux® Binary Compatibility +title: Chapter 10. Linux Binary Compatibility part: Part II. Common Tasks prev: books/handbook/printing next: books/handbook/wine -description: FreeBSD provides binary compatibility with Linux®, allowing users to install and run most Linux® binaries on a FreeBSD system without having to first modify the binary +description: FreeBSD provides binary compatibility with Linux, allowing users to install and run most Linux binaries on a FreeBSD system without having to first modify the binary tags: ["linux", "linuxulator", "emulation", "binary", "compatibility"] showBookMenu: true weight: 13 @@ -51,16 +51,10 @@ endif::[] [[linuxemu-synopsis]] == Synopsis -FreeBSD provides binary compatibility with Linux(R), allowing users to install and run most Linux binaries on a FreeBSD system without having to first modify the binary. -It has even been reported that, in some situations, Linux binaries perform better on FreeBSD than they do on Linux. +FreeBSD provides optional binary compatibility with Linux(R), allowing users to install and run unmodified Linux binaries. +It is available for the i386, amd64, and arm64 architectures. -However, some Linux-specific operating system features are not supported under FreeBSD. -For example, Linux binaries will not work on FreeBSD if they overly use i386(TM) specific calls, such as enabling virtual 8086 mode. - -[NOTE] -==== -Support for 64-bit binary compatibility with Linux was added in FreeBSD 10.3. -==== +Some Linux-specific operating system features are not yet supported; this mostly happens with functionality specific to hardware or related to system management, such as cgroups or namespaces. After reading this chapter, you will know: @@ -76,76 +70,106 @@ Before reading this chapter, you should: [[linuxemu-lbc-install]] == Configuring Linux Binary Compatibility -By default, Linux libraries are not installed and Linux binary compatibility is not enabled. -Linux libraries can either be installed manually or from the FreeBSD Ports Collection. +By default, Linux binary compatibility is not enabled. +To enable it at boot time, add this line to [.filename]#/etc/rc.conf#: -Before attempting to build the port, load the Linux kernel module, otherwise the build will fail: +[.programlisting] +.... +linux_enable="YES" +.... +Once enabled, it can be started without rebooting by running: [source,shell] .... -# kldload linux +# service linux start .... -For 64-bit compatibility: +The [.filename]#/etc/rc.d/linux# script will load necessary kernel modules and mount filesystems expected by Linux applications under [.filename]#/compat/linux#. +This is enough for statically linked Linux binaries to work. +They can be started in the same way native FreeBSD binaries can; they behave almost exactly like native processes and can be traced and debugged the usual way. + +Linux binaries linked dynamically (which is the vast majority) also require Linux shared libraries to be installed - they can run on top of the FreeBSD kernel, but they cannot use FreeBSD libraries; this is similar to how 32-bit binaries cannot use native 64-bit libraries. +There are several ways of providing those libraries: one can copy them over from an existing Linux installation using the same architecture, install them from FreeBSD packages, or install using man:deboostrap[8] (from package:sysutils/debootstrap[]), and others. + +[[linuxemu-packages]] +== CentOS Base System from FreeBSD Packages + +The easiest way to install Linux libraries is to install package:emulators/linux_base-c7[] package or port, which places the CentOS 7-derived base system into [.filename]#/compat/linux#: [source,shell] .... -# kldload linux64 +# pkg install linux_base-c7 .... -To verify that the module is loaded: - +FreeBSD provides packages for some Linux binary applications. +For example, to install Sublime Text, run this command: [source,shell] .... -% kldstat - Id Refs Address Size Name - 1 2 0xc0100000 16bdb8 kernel - 7 1 0xc24db000 d000 linux.ko +# pkg install linux-sublime .... -The package:emulators/linux_base-c7[] package or port is the easiest way to install a base set of Linux libraries and binaries on a FreeBSD system. -To install the port: +[[linuxemu-debootstrap]] +== Debian / Ubuntu Base System with man:debootstrap[8] -[source,shell] +An alternative way of providing Linux shared libraries is by using package:sysutils/debootstrap[]. +This has the advantage of providing a full Debian or Ubuntu distribution. +To use it, follow the instructions at FreeBSD Wiki: https://wiki.freebsd.org/LinuxJails[FreeBSD Wiki - Linux Jails]. + +It is possible to debootstrap into [.filename]#/compat/linux#, but it is discouraged to avoid collisions with files installed from FreeBSD ports and packages. +Instead, derive the directory name from the distribution or version name, e.g., [.filename]#/compat/ubuntu#. +If the bootstrapped instance is intended to provide Linux shared libraries without having to explicitly use chroot or jails, one can point the kernel at it by updating the `compat.linux.emul_path` sysctl and adding a line like this to [.filename]#/etc/sysctl.conf#: + +[.programlisting] .... -# pkg install emulators/linux_base-c7 +compat.linux.emul_path="/compat/ubuntu" .... -For Linux compatibility to be enabled at boot time, add this line to [.filename]#/etc/rc.conf#: +Afterwards chroot(8) into the newly created directory and install software in a way typical for the Linux distribution you have debootstrapped, for example: -[.programlisting] +[source,shell] .... -linux_enable="YES" +# chroot /compat/ubuntu /bin/bash +root@hostname:/# apt update .... -On 64-bit machines, [.filename]#/etc/rc.d/abi# will automatically load the module for 64-bit emulation. +[[linuxemu-advanced]] +== Advanced Topics -Since the Linux binary compatibility layer has gained support for running both 32- and 64-bit Linux binaries (on 64-bit x86 hosts), it is no longer possible to link the emulation functionality statically into a custom kernel. +The Linux compatibility layer is a work in progress. +Consult https://wiki.freebsd.org/Linuxulator[FreeBSD Wiki - Linuxulator] for more information. -For some applications, [.filename]#/compat/linux/proc#, [.filename]#/compat/linux/sys#, and [.filename]#/compat/linux/dev/shm# may need to be mounted. -Add this line to [.filename]#/etc/fstab#: +A list of all Linux-related man:sysctl[8] knobs can be found in man:linux[4]. +Some applications require specific filesystems to be mounted. +This is normally handled by the [.filename]#/etc/rc.d/linux# script, but can be disabled by adding this line to [.filename]#/etc/rc.conf#: + +[.programlisting] .... -linprocfs /compat/linux/proc linprocfs rw 0 0 -linsysfs /compat/linux/sys linsysfs rw 0 0 -tmpfs /compat/linux/dev/shm tmpfs rw,mode=1777 0 0 +linux_mounts_enable="NO" .... -Then mount the filesystem accordingly: +Filesystems mounted by the rc script will not work for Linux processes inside chroots or jails; if needed, configure them in [.filename]#/etc/fstab#: +.... +devfs /compat/linux/dev devfs rw,late 0 0 +tmpfs /compat/linux/dev/shm tmpfs rw,late,size=1g,mode=1777 0 0 +fdescfs /compat/linux/dev/fd fdescfs rw,late,linrdlnk 0 0 +linprocfs /compat/linux/proc linprocfs rw,late 0 0 +linsysfs /compat/linux/sys linsysfs rw,late 0 0 +.... -[source,shell] ----- -# mount /compat/linux/sys -# mount /compat/linux/proc -# mount /compat/linux/dev/shm ----- +Since the Linux binary compatibility layer has gained support for running both 32- and 64-bit Linux binaries (on 64-bit x86 hosts), it is no longer possible to link the emulation functionality statically into a custom kernel. [[linuxemu-libs-manually]] === Installing Additional Libraries Manually +[NOTE] +==== +For base system subdirectories created with man:debootstrap[8], use the instructions above instead. +==== + If a Linux application complains about missing shared libraries after configuring Linux binary compatibility, determine which shared libraries the Linux binary needs and install them manually. -From a Linux system, `ldd` can be used to determine which shared libraries the application needs. +From a Linux system using the same CPU architecture, `ldd` can be used to determine which shared libraries the application needs. For example, to check which shared libraries `linuxdoom` needs, run this command from a Linux system that has Doom installed: [source,shell] @@ -170,8 +194,7 @@ This example will result in the following files on the FreeBSD system: /compat/linux/lib/libc.so.4 -> libc.so.4.6.29 .... -If a Linux shared library already exists with a matching major revision number to the first column of the `ldd` output, -it does not need to be copied to the file named in the last column, as the existing library should work. +If a Linux shared library already exists with a matching major revision number to the first column of the `ldd` output, it does not need to be copied to the file named in the last column, as the existing library should work. It is advisable to copy the shared library if it is a newer version, though. The old one can be removed, as long as the symbolic link points to the new one. @@ -202,10 +225,10 @@ However, it is safe to replace the existing [.filename]#libc.so# with the newer Generally, one will need to look for the shared libraries that Linux binaries depend on only the first few times that a Linux program is installed on FreeBSD. After a while, there will be a sufficient set of Linux shared libraries on the system to be able to run newly installed Linux binaries without any extra work. -=== Installing Linux ELF Binaries +=== Branding Linux ELF Binaries -ELF binaries sometimes require an extra step. -When an unbranded ELF binary is executed, it will generate an error message: +The FreeBSD kernel uses several methods to determine if the binary to be executed is a Linux one: it checks the brand in the ELF file header, looks for known ELF interpreter paths and checks ELF notes; finally, by default, unbranded ELF executables are assumed to be Linux anyway. +Should all those methods fail, an attempt to execute the binary might result in error message: [source,shell] .... @@ -221,8 +244,6 @@ To help the FreeBSD kernel distinguish between a FreeBSD ELF binary and a Linux % brandelf -t Linux my-linux-elf-binary .... -Since the GNU toolchain places the appropriate branding information into ELF binaries automatically, this step is usually not necessary. - === Installing a Linux RPM Based Application To install a Linux RPM-based application, first install the package:archivers/rpm4[] package or port. @@ -260,7 +281,7 @@ When [.filename]#/compat/linux/etc/host.conf# does not exist, Linux applications Remove `bind` if a name server is not configured using [.filename]#/etc/resolv.conf#. [[linuxemu-advanced]] -== Advanced Topics +=== Miscellaneous This section describes how Linux binary compatibility works and is based on an email written to {freebsd-chat} by Terry Lambert mailto:tlambert@primenet.com[tlambert@primenet.com] (Message ID: `<199906020108.SAA07001@usr09.primenet.com>`). @@ -296,7 +317,7 @@ When a system call is called by the Linux binary, the trap code dereferences the Linux mode dynamically _reroots_ lookups. This is, in effect, equivalent to `union` to file system mounts. -First, an attempt is made to lookup the file in [.filename]#/compat/linux/original-path#. +First, an attempt is made to look up the file in [.filename]#/compat/linux/original-path#. If that fails, the lookup is done in [.filename]#/original-path#. This makes sure that binaries that require other binaries can run. For example, the Linux toolchain can all run under Linux ABI support. @@ -308,5 +329,5 @@ The only difference is that FreeBSD binaries get the FreeBSD _glue_ functions, a The FreeBSD _glue_ functions are statically linked into the kernel, and the Linux _glue_ functions can be statically linked, or they can be accessed via a kernel module. Technically, this is not really emulation, it is an ABI implementation. -It is sometimes called "Linux emulation" because the implementation was done at a time when there was no other word to describe what was going on. +It is sometimes called "Linux emulation" because the implementation was done at a time when there was no other word to describe what was going on. Saying that FreeBSD ran Linux binaries was not true, since the code was not compiled in.