svn commit: r344413 - in stable/11: release/tools share/mk stand stand/common stand/efi stand/efi/loader stand/efi/loader_4th stand/efi/loader_lua stand/efi/loader_simp stand/i386 stand/i386/loader...

Kyle Evans kevans at FreeBSD.org
Thu Feb 21 03:18:18 UTC 2019 

Author: kevans
Date: Thu Feb 21 03:18:12 2019
New Revision: 344413
URL: https://svnweb.freebsd.org/changeset/base/344413

Log:
  MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927,
  r337989, r338052, r338064, r338407, r338418, r338474
  
  r337806:
  Create a loader for each interpreter for x86 BIOS and all EFI
  
  Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
  default. Create LOADER_DEFAULT_INTERP to specify the default
  interpreter when no other is specified. LOADER_INTERP is the current
  interpreter language building. Turn building of lua on by default to
  match 4th. simploader is a simplified loader build w/o any interpreter
  language (but with a simple loader).  This is the historic behavir you
  got with WITHOUT_FORTH. Make a hard link to the default loader. This
  has to be a hard link rather than the more desirable soft link because
  older zfsboot blocks don't support symlinks.
  
  r337868:
  stand: Use -Oz/-Os for all loader/stand builds.
  
  While we're not super size constrained, the x86 BIOS /boot/loader has
  to be less than about 520k-530k to be reliable. The LUA loader is at
  this size today. -Oz saves 15-20% on the size, keeping us safely small
  enough (comparable to where we were with the 4th loader). This will
  also help with sjg's work on bringing in bearssl, though we may again
  be looking for space in the LUA loader.
  
  Size table for clang 6.0.0:
  	default		-O1		-Os		-Oz
  4th	442368		417792		389120		376832
  lua	524288		479232		446464		430080
  
  r337914:
  Install links for loader.efi.
  
  r337927:
  Add ashldi3 and ashrdi3 to mips.
  
  Now that we're using -Os, mips needs these routines.
  
  r337989:
  Copy the boot loader from the new location for the co-existing
  loaders.
  
  r338052:
  libsa: Add lshrdi3.c for powerpc* and mips
  
  r338064:
  Turn back the clock just a little: make userboot.so always be 4th
  
  Turns out there was a hidden dependency we hasn't counted upon.  The
  host load /boot/userboot.so to boot the VMs it runs. This means that
  the change to lua meant suddently that nobody could run their older
  VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
  whatever.  Even more than for the /boot/loader* binaries, we need a
  good coexistance strategy for this. While that's being designed and
  implemented, drop back to always 4th for userboot.so. This will fail
  safe in all but the most extreme environments (but lua-only hacks
  to .lua files won't be processes in VMs until we fix it).
  
  r338407:
  lualoader: Print error messages from command failures at the prompt
  
  Previously lualoader would remain silent, rather than printing
  command_errmsg or noting that a command had failed or was not found.
  
  r338418:
  userboot: handle guest interpreter mismatches more intelligently
  
  The switch to lualoader creates a problem with userboot: the host is
  inclined to build userboot with Lua, but the host userboot's interpreter
  must match what's available on the guest. For almost all FreeBSD guests in
  the wild, Lua is not yet available and a Lua-based userboot will fail.
  
  This revision updates userboot protocol to version 5, which adds a
  swap_interpreter callback to request a different interpreter, and tries to
  determine the proper interpreter to be used based on how the guest
  /boot/loader is compiled. This is still a bit of a guess, but it's likely
  the best possible guess we can make in order to get it right. The
  interpreter is now embedded in the resulting executable, so we can open
  /boot/loader on the guest and hunt that down to derive the interpreter it
  was built with.
  
  Using -l with bhyveload will not allow an intepreter swap, even if the
  loader specified happens to be a userboot with the wrong interpreter. We'll
  simply complain about the mismatch and bail out.
  
  For legacy guests without the interpreter marker, we assume they're 4th.
  For new guests with the interpreter marker, we'll read it and swap over
  to the proper interpreter if it doesn't match what the userboot we're using
  was compiled with.
  
  Both flavors of userboot are installed by default, userboot_4th.so and
  userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
  was broken by userboot being forced to 4th.
  
  r338474:
  Be a little conservative about when to force size optimizations.
  
  Reports have come in that there's issue with powerpc and sparc64 since
  we've switched to using -Oz / -Os. We don't strictly need them for
  !x86, so be conservative about when we enable them.

Added:
  stable/11/stand/efi/loader_4th/
     - copied from r337806, head/stand/efi/loader_4th/
  stable/11/stand/efi/loader_lua/
     - copied from r337806, head/stand/efi/loader_lua/
  stable/11/stand/efi/loader_simp/
     - copied from r337806, head/stand/efi/loader_simp/
  stable/11/stand/i386/loader_4th/
     - copied from r337806, head/stand/i386/loader_4th/
  stable/11/stand/i386/loader_lua/
     - copied from r337806, head/stand/i386/loader_lua/
  stable/11/stand/i386/loader_simp/
     - copied from r337806, head/stand/i386/loader_simp/
  stable/11/stand/userboot/userboot_4th/
     - copied from r338418, head/stand/userboot/userboot_4th/
  stable/11/stand/userboot/userboot_lua/
     - copied from r338418, head/stand/userboot/userboot_lua/
Modified:
  stable/11/release/tools/arm.subr
  stable/11/share/mk/src.opts.mk
  stable/11/stand/common/bootstrap.h
  stable/11/stand/common/interp_forth.c
  stable/11/stand/common/interp_lua.c
  stable/11/stand/common/interp_simple.c
  stable/11/stand/defs.mk
  stable/11/stand/efi/Makefile
  stable/11/stand/efi/loader/Makefile
  stable/11/stand/i386/Makefile
  stable/11/stand/i386/loader/Makefile
  stable/11/stand/i386/pxeldr/Makefile
  stable/11/stand/libsa/Makefile
  stable/11/stand/loader.mk
  stable/11/stand/userboot/Makefile
  stable/11/stand/userboot/userboot.h
  stable/11/stand/userboot/userboot/Makefile
  stable/11/stand/userboot/userboot/main.c
  stable/11/stand/userboot/userboot/version
  stable/11/usr.sbin/bhyveload/bhyveload.c
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/release/tools/arm.subr
==============================================================================
--- stable/11/release/tools/arm.subr	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/release/tools/arm.subr	Thu Feb 21 03:18:12 2019	(r344413)
@@ -152,7 +152,7 @@ arm_install_boot() {
 	BOOTFILES="$(chroot ${CHROOTDIR} realpath ${BOOTFILES})"
 
 	chroot ${CHROOTDIR} mkdir -p ${FATMOUNT}/EFI/BOOT
-	chroot ${CHROOTDIR} cp -p ${BOOTFILES}/efi/loader/loader.efi \
+	chroot ${CHROOTDIR} cp -p ${BOOTFILES}/efi/loader_lua/loader_lua.efi \
 		${FATMOUNT}/EFI/BOOT/$(efi_boot_name ${EMBEDDED_TARGET})
 
 	chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot

Modified: stable/11/share/mk/src.opts.mk
==============================================================================
--- stable/11/share/mk/src.opts.mk	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/share/mk/src.opts.mk	Thu Feb 21 03:18:12 2019	(r344413)
@@ -126,6 +126,7 @@ __DEFAULT_YES_OPTIONS = \
     LIBPTHREAD \
     LIBTHR \
     LOADER_GELI \
+    LOADER_LUA \
     LOADER_OFW \
     LOADER_UBOOT \
     LOCALES \
@@ -196,7 +197,6 @@ __DEFAULT_NO_OPTIONS = \
     LINT \
     LOADER_FIREWIRE \
     LOADER_FORCE_LE \
-    LOADER_LUA \
     NAND \
     OFED_EXTRA \
     OPENLDAP \

Modified: stable/11/stand/common/bootstrap.h
==============================================================================
--- stable/11/stand/common/bootstrap.h	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/common/bootstrap.h	Thu Feb 21 03:18:12 2019	(r344413)
@@ -164,6 +164,14 @@ extern int			isapnp_readport;
 extern char bootprog_info[];
 
 /*
+ * Interpreter information
+ */
+extern const char bootprog_interp[];
+#define	INTERP_DEFINE(interpstr) \
+const char bootprog_interp[] = "$Interpreter:" interpstr
+
+
+/*
  * Preloaded file metadata header.
  *
  * Metadata are allocated on our heap, and copied into kernel space

Modified: stable/11/stand/common/interp_forth.c
==============================================================================
--- stable/11/stand/common/interp_forth.c	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/common/interp_forth.c	Thu Feb 21 03:18:12 2019	(r344413)
@@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$");
 #include "ficl.h"
 
 extern unsigned bootprog_rev;
+INTERP_DEFINE("4th");
 
 /* #define BFORTH_DEBUG */
 

Modified: stable/11/stand/common/interp_lua.c
==============================================================================
--- stable/11/stand/common/interp_lua.c	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/common/interp_lua.c	Thu Feb 21 03:18:12 2019	(r344413)
@@ -60,6 +60,7 @@ static struct interp_lua_softc lua_softc;
 #define	LDBG(...)
 #endif
 
+INTERP_DEFINE("lua");
 
 static void *
 interp_lua_realloc(void *ud __unused, void *ptr, size_t osize __unused, size_t nsize)
@@ -135,7 +136,7 @@ interp_run(const char *line)
 	char	**argv;
 	lua_State *luap;
 	struct interp_lua_softc	*softc = &lua_softc;
-	int status;
+	int status, ret;
 
 	luap = softc->luap;
 	LDBG("executing line...");
@@ -147,14 +148,16 @@ interp_run(const char *line)
 		 * run it through cli_execute. If that fails, then we'll try it
 		 * as a builtin.
 		 */
+		command_errmsg = NULL;
 		if (parse(&argc, &argv, line) == 0) {
 			lua_getglobal(luap, "cli_execute");
 			for (nargc = 0; nargc < argc; ++nargc) {
 				lua_pushstring(luap, argv[nargc]);
 			}
 			status = lua_pcall(luap, argc, 1, 0);
+			ret = lua_tointeger(luap, 1);
 			lua_pop(luap, 1);
-			if (status != 0) {
+			if (status != 0 || ret != 0) {
 				/*
 				 * Lua cli_execute will pass the function back
 				 * through loader.command, which is a proxy to
@@ -166,7 +169,10 @@ interp_run(const char *line)
 				status = interp_builtin_cmd(argc, argv);
 			}
 			if (status != 0) {
-				printf("Command failed\n");
+				if (command_errmsg != NULL)
+					printf("%s\n", command_errmsg);
+				else
+					printf("Command failed\n");
 				status = CMD_ERROR;
 			}
 			free(argv);

Modified: stable/11/stand/common/interp_simple.c
==============================================================================
--- stable/11/stand/common/interp_simple.c	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/common/interp_simple.c	Thu Feb 21 03:18:12 2019	(r344413)
@@ -35,6 +35,8 @@ __FBSDID("$FreeBSD$");
 #include <string.h>
 #include "bootstrap.h"
 
+INTERP_DEFINE("simp");
+
 void
 interp_init(void)
 {

Modified: stable/11/stand/defs.mk
==============================================================================
--- stable/11/stand/defs.mk	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/defs.mk	Thu Feb 21 03:18:12 2019	(r344413)
@@ -53,8 +53,15 @@ CFLAGS+=	-I${SASRC} -D_STANDALONE
 CFLAGS+=	-I${SYSDIR}
 # Spike the floating point interfaces
 CFLAGS+=	-Ddouble=jagged-little-pill -Dfloat=floaty-mcfloatface
+.if ${MACHINE_ARCH} == "i386" || ${MACHINE_ARCH} == "amd64"
+# Slim down the image. This saves about 15% in size with clang 6 on x86
+# Our most constrained /boot/loader env is BIOS booting on x86, where
+# our text + data + BTX have to fit into 640k below the ISA hole.
+# Experience has shown that problems arise between ~520k to ~530k.
+CFLAGS.clang+=	-Oz
+CFLAGS.gcc+=	-Os
+.endif
 
-
 # GELI Support, with backward compat hooks (mostly)
 .if defined(LOADER_NO_GELI_SUPPORT)
 MK_LOADER_GELI=no
@@ -145,6 +152,18 @@ CFLAGS+=	-G0 -fno-pic -mno-abicalls
 CFLAGS+=	-mlittle-endian
 .endif
 .endif
+
+#
+# Have a sensible default
+#
+.if ${MK_FORTH} == "yes"
+LOADER_DEFAULT_INTERP?=4th
+.elif ${MK_LOADER_LUA} == "yes"
+LOADER_DEFAULT_INTERP?=lua
+.else
+LOADER_DEFAULT_INTERP?=simp
+.endif
+LOADER_INTERP?=${LOADER_DEFAULT_INTERP}
 
 # Make sure we use the machine link we're about to create
 CFLAGS+=-I.

Modified: stable/11/stand/efi/Makefile
==============================================================================
--- stable/11/stand/efi/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/efi/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -9,7 +9,10 @@ NO_OBJ=t
 .if ${COMPILER_TYPE} != "gcc" || ${COMPILER_VERSION} >= 40500
 
 SUBDIR.${MK_FDT}+=	fdt
-SUBDIR.yes+=	libefi loader boot1
+SUBDIR.yes+=	libefi boot1
+SUBDIR.${MK_FORTH}+= loader_4th
+SUBDIR.${MK_LOADER_LUA}+= loader_lua
+SUBDIR.yes+=	loader_simp
 
 .endif # ${COMPILER_TYPE} != "gcc" || ${COMPILER_VERSION} >= 40500
 

Modified: stable/11/stand/efi/loader/Makefile
==============================================================================
--- stable/11/stand/efi/loader/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/efi/loader/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -8,7 +8,8 @@ LOADER_EXT2FS_SUPPORT?=	no
 
 .include <bsd.init.mk>
 
-PROG=		loader.sym
+LOADER?=	loader_${LOADER_INTERP}
+PROG=		${LOADER}.sym
 INTERNALPROG=
 WARNS?=		3
 
@@ -24,6 +25,7 @@ SRCS=	autoload.c \
 	smbios.c \
 	vers.c
 
+CFLAGS+=	-I${.CURDIR}/../loader
 .if ${MK_LOADER_ZFS} != "no"
 CFLAGS+=	-I${ZFSSRC}
 CFLAGS+=	-DEFI_ZFS_BOOT
@@ -41,10 +43,11 @@ CWARNFLAGS.self_reloc.c+=	-Wno-error=maybe-uninitializ
 # warnings for now.
 CWARNFLAGS.main.c+=	-Wno-format
 
-.PATH: ${.CURDIR}/arch/${MACHINE}
-# For smbios.c
+.PATH: ${.CURDIR}/../loader
+.PATH: ${.CURDIR}/../loader/arch/${MACHINE}
+# For smbios.c XXX need to abstract properly
 .PATH: ${BOOTSRC}/i386/libi386
-.include "${.CURDIR}/arch/${MACHINE}/Makefile.inc"
+.include "${.CURDIR}/../loader/arch/${MACHINE}/Makefile.inc"
 
 CFLAGS+=	-I${.CURDIR}
 CFLAGS+=	-I${.CURDIR}/arch/${MACHINE}
@@ -78,20 +81,25 @@ HAVE_BCACHE=    yes
 CFLAGS+=	-DEFI_STAGING_SIZE=${EFI_STAGING_SIZE}
 .endif
 
+NEWVERSWHAT=	"EFI loader" ${MACHINE}
+VERSION_FILE=	${.CURDIR}/../loader/version
+
 # Always add MI sources
 .include	"${BOOTSRC}/loader.mk"
 
-FILES+=	loader.efi
-FILESMODE_loader.efi=	${BINMODE}
+FILES+=	${LOADER}.efi
+FILESMODE_${LOADER}.efi=	${BINMODE}
 
-LDSCRIPT=	${.CURDIR}/arch/${MACHINE}/ldscript.${MACHINE}
+.if ${LOADER_INTERP} == ${LOADER_DEFAULT_INTERP}
+LINKS+=		${BINDIR}/${LOADER}.efi ${BINDIR}/loader.efi
+.endif
+
+LDSCRIPT=	${.CURDIR}/../loader/arch/${MACHINE}/ldscript.${MACHINE}
 LDFLAGS+=	-Wl,-T${LDSCRIPT},-Bsymbolic,-znotext -shared
 
 CLEANFILES+=	loader.efi
 
-NEWVERSWHAT=	"EFI loader" ${MACHINE}
-
-loader.efi: ${PROG}
+${LOADER}.efi: ${PROG}
 	if ${NM} ${.ALLSRC} | grep ' U '; then \
 		echo "Undefined symbols in ${.ALLSRC}"; \
 		exit 1; \

Modified: stable/11/stand/i386/Makefile
==============================================================================
--- stable/11/stand/i386/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/i386/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -9,7 +9,9 @@ SUBDIR.yes=	mbr pmbr boot0 boot0sio btx boot2 cdboot g
 
 SUBDIR.${MK_LOADER_FIREWIRE}+=	libfirewire
 
-SUBDIR.yes+=	loader
+SUBDIR.${MK_FORTH}+= loader_4th
+SUBDIR.${MK_LOADER_LUA}+= loader_lua
+SUBDIR.yes+=	loader_simp
 
 # special boot programs, 'self-extracting boot2+loader'
 SUBDIR.yes+=	pxeldr

Modified: stable/11/stand/i386/loader/Makefile
==============================================================================
--- stable/11/stand/i386/loader/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/i386/loader/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -14,7 +14,7 @@ LOADER_BZIP2_SUPPORT?=	yes
 
 .include <bsd.init.mk>
 
-LOADER?=	loader
+LOADER?=	loader_${LOADER_INTERP}
 PROG=		${LOADER}.sym
 INTERNALPROG=
 NEWVERSWHAT?=	"bootstrap loader" x86
@@ -64,12 +64,13 @@ ${LOADER}: ${LOADER}.bin ${BTXLDR} ${BTXKERN}
 ${LOADER}.bin: ${LOADER}.sym
 	strip -R .comment -R .note -o ${.TARGET} ${.ALLSRC}
 
-# XXX TODO: Fix this when coexistence comes in
-.if ${MK_LOADER_ZFS} == "yes" # && ${LOADER_INTERP} == ${LOADER_DEFAULT_INTERP}
+.if ${MK_LOADER_ZFS} == "yes" && ${LOADER_INTERP} == ${LOADER_DEFAULT_INTERP}
 LINKS+=		${BINDIR}/${LOADER} ${BINDIR}/zfsloader
 .endif
+.if ${LOADER_INTERP} == ${LOADER_DEFAULT_INTERP}
+LINKS+=		${BINDIR}/${LOADER} ${BINDIR}/loader
+.endif
 FILES+=	${LOADER}
-# XXX INSTALLFLAGS_loader= -b
 FILESMODE_${LOADER}= ${BINMODE} -b
 
 # XXX crt0.o needs to be first for pxeboot(8) to work

Modified: stable/11/stand/i386/pxeldr/Makefile
==============================================================================
--- stable/11/stand/i386/pxeldr/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/i386/pxeldr/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -24,7 +24,8 @@ CFLAGS+=-DALWAYS_SERIAL
 
 CFLAGS+=-I${BOOTSRC}/i386/common
 
-LOADERBIN= ${BOOTOBJ}/i386/loader/loader.bin
+L=${LOADER_DEFAULT_INTERP}
+LOADERBIN= ${BOOTOBJ}/i386/loader_${L}/loader_${L}.bin
 
 CLEANFILES+= ${BOOT}.tmp
 

Modified: stable/11/stand/libsa/Makefile
==============================================================================
--- stable/11/stand/libsa/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/libsa/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -73,8 +73,13 @@ SRCS+=	aeabi_memcmp.S aeabi_memcpy.S aeabi_memmove.S a
 
 .if ${MACHINE_CPUARCH} == "powerpc"
 .PATH: ${LIBC_SRC}/quad
-SRCS+=	ashldi3.c ashrdi3.c
+SRCS+=	ashldi3.c ashrdi3.c lshrdi3.c
 SRCS+=	syncicache.c
+.endif
+
+.if ${MACHINE_CPUARCH} == "mips"
+.PATH: ${LIBC_SRC}/quad
+SRCS+=	ashldi3.c ashrdi3.c lshrdi3.c
 .endif
 
 # uuid functions from libc

Modified: stable/11/stand/loader.mk
==============================================================================
--- stable/11/stand/loader.mk	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/loader.mk	Thu Feb 21 03:18:12 2019	(r344413)
@@ -59,19 +59,20 @@ SRCS+=	isapnp.c
 SRCS+=	pnp.c
 .endif
 
-# Forth interpreter
-.if ${MK_FORTH} != "no"
-SRCS+=	interp_forth.c
-.include "${BOOTSRC}/ficl.mk"
-LDR_INTERP=	${LIBFICL}
-LDR_INTERP32=	${LIBFICL32}
-.elif ${MK_LOADER_LUA} != "no"
+.if ${LOADER_INTERP} == "lua"
 SRCS+=	interp_lua.c
 .include "${BOOTSRC}/lua.mk"
 LDR_INTERP=	${LIBLUA}
 LDR_INTERP32=	${LIBLUA32}
-.else
+.elif ${LOADER_INTERP} == "4th"
+SRCS+=	interp_forth.c
+.include "${BOOTSRC}/ficl.mk"
+LDR_INTERP=	${LIBFICL}
+LDR_INTERP32=	${LIBFICL32}
+.elif ${LOADER_INTERP} == "simp"
 SRCS+=	interp_simple.c
+.else
+.error Unknown interpreter ${LOADER_INTERP}
 .endif
 
 .if defined(BOOT_PROMPT_123)

Modified: stable/11/stand/userboot/Makefile
==============================================================================
--- stable/11/stand/userboot/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/userboot/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -1,8 +1,10 @@
 # $FreeBSD$
 
-.include <bsd.own.mk>
+.include <bsd.init.mk>
 
-SUBDIR=		test userboot
+SUBDIR.yes=		test
+SUBDIR.${MK_FORTH}+= userboot_4th
+SUBDIR.${MK_LOADER_LUA}+= userboot_lua
 
 .include <bsd.subdir.mk>
 

Modified: stable/11/stand/userboot/userboot.h
==============================================================================
--- stable/11/stand/userboot/userboot.h	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/userboot/userboot.h	Thu Feb 21 03:18:12 2019	(r344413)
@@ -42,6 +42,13 @@
 #define	USERBOOT_VERSION_4      4
 
 /*
+ * Version 5 added a callback for indicating that the guest
+ * should be restarted with a different interpreter.  The callback
+ * structure is still backward compatible.
+ */
+#define	USERBOOT_VERSION_5      5
+
+/*
  * Exit codes from the loader
  */
 #define	USERBOOT_EXIT_QUIT      1
@@ -210,4 +217,9 @@ struct loader_callbacks {
 	int	(*vm_set_register)(void *arg, int vcpu, int reg, uint64_t val);
 	int	(*vm_set_desc)(void *arg, int vcpu, int reg, uint64_t base,
 	    u_int limit, u_int access);
+
+	/*
+	 * Version 5 addition.
+	 */
+	void	(*swap_interpreter)(void *arg, const char *interp);
 };

Modified: stable/11/stand/userboot/userboot/Makefile
==============================================================================
--- stable/11/stand/userboot/userboot/Makefile	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/userboot/userboot/Makefile	Thu Feb 21 03:18:12 2019	(r344413)
@@ -8,10 +8,11 @@ PIC=yes
 
 .include <bsd.init.mk>
 
-SHLIB_NAME=	userboot.so
+SHLIB_NAME=	userboot_${LOADER_INTERP}.so
 STRIP=
 LIBDIR=		/boot
 
+.PATH:		${.CURDIR}/../userboot
 SRCS=		autoload.c
 SRCS+=		bcache.c
 SRCS+=		biossmap.c
@@ -37,7 +38,12 @@ CWARNFLAGS.main.c += -Wno-implicit-function-declaratio
 
 LDFLAGS+=	-nostdlib -Wl,-Bsymbolic
 
-NEWVERSWHAT=	"User boot" ${MACHINE_CPUARCH}
+NEWVERSWHAT=	"User boot ${LOADER_INTERP}" ${MACHINE_CPUARCH}
+VERSION_FILE=	${.CURDIR}/../userboot/version
+
+.if ${LOADER_INTERP} == ${LOADER_DEFAULT_INTERP}
+LINKS+=		${BINDIR}/${SHLIB_NAME} ${BINDIR}/userboot.so
+.endif
 
 .if ${MK_LOADER_ZFS} != "no"
 CFLAGS+=	-DUSERBOOT_ZFS_SUPPORT

Modified: stable/11/stand/userboot/userboot/main.c
==============================================================================
--- stable/11/stand/userboot/userboot/main.c	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/userboot/userboot/main.c	Thu Feb 21 03:18:12 2019	(r344413)
@@ -47,6 +47,9 @@ static int userboot_zfs_found;
 /* Minimum version required */
 #define	USERBOOT_VERSION	USERBOOT_VERSION_3
 
+#define	LOADER_PATH		"/boot/loader"
+#define	INTERP_MARKER		"$Interpreter:"
+
 #define	MALLOCSZ		(64*1024*1024)
 
 struct loader_callbacks *callbacks;
@@ -57,6 +60,7 @@ static jmp_buf jb;
 struct arch_switch archsw;	/* MI/MD interface boundary */
 
 static void	extract_currdev(void);
+static void	check_interpreter(void);
 
 void
 delay(int usec)
@@ -73,6 +77,62 @@ exit(int v)
 	longjmp(jb, 1);
 }
 
+static void
+check_interpreter(void)
+{
+	struct stat st;
+	size_t marklen, rdsize;
+	const char *guest_interp, *my_interp;
+	char *buf;
+	int fd;
+
+	/*
+	 * If we can't stat(2) or open(2) LOADER_PATH, then we'll fail by
+	 * simply letting us roll on with whatever interpreter we were compiled
+	 * with.  This is likely not going to be an issue in reality.
+	 */
+	buf =  NULL;
+	if (stat(LOADER_PATH, &st) != 0)
+		return;
+	if ((fd = open(LOADER_PATH, O_RDONLY)) < 0)
+		return;
+
+	rdsize = st.st_size;
+	buf = malloc(rdsize);
+	if (buf == NULL)
+		goto out;
+	if (read(fd, buf, rdsize) < rdsize)
+		goto out;
+
+	marklen = strlen(INTERP_MARKER);
+	my_interp = bootprog_interp + marklen;
+
+	/*
+	 * Here we make the assumption that a loader binary without the
+	 * interpreter marker is a 4th one.  All loader binaries going forward
+	 * should have this properly specified, so our assumption should always
+	 * be a good one.
+	 */
+	if ((guest_interp = memmem(buf, rdsize, INTERP_MARKER,
+	    marklen)) != NULL)
+		guest_interp += marklen;
+	else
+		guest_interp = "4th";
+
+	/*
+	 * The guest interpreter may not have a version of loader that
+	 * specifies the interpreter installed.  If that's the case, we'll
+	 * assume it's legacy (4th) and request a swap to that if we're
+	 * a Lua-userboot.
+	 */
+	if (strcmp(my_interp, guest_interp) != 0)
+		CALLBACK(swap_interpreter, guest_interp);
+out:
+	free(buf);
+	close(fd);
+	return;
+}
+
 void
 loader_main(struct loader_callbacks *cb, void *arg, int version, int ndisks)
 {
@@ -137,6 +197,14 @@ loader_main(struct loader_callbacks *cb, void *arg, in
 			(devsw[i]->dv_init)();
 
 	extract_currdev();
+
+	/*
+	 * Checking the interpreter isn't worth the overhead unless we
+	 * actually have the swap_interpreter callback, so we actually version
+	 * check here rather than later on.
+	 */
+	if (version >= USERBOOT_VERSION_5)
+		check_interpreter();
 
 	if (setjmp(jb))
 		return;

Modified: stable/11/stand/userboot/userboot/version
==============================================================================
--- stable/11/stand/userboot/userboot/version	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/stand/userboot/userboot/version	Thu Feb 21 03:18:12 2019	(r344413)
@@ -1,4 +1,5 @@
 $FreeBSD$
 
+1.2:	Userboot with lua or forth
 1.1:	Initial userland boot
 

Modified: stable/11/usr.sbin/bhyveload/bhyveload.c
==============================================================================
--- stable/11/usr.sbin/bhyveload/bhyveload.c	Thu Feb 21 02:52:27 2019	(r344412)
+++ stable/11/usr.sbin/bhyveload/bhyveload.c	Thu Feb 21 03:18:12 2019	(r344413)
@@ -75,6 +75,7 @@ __FBSDID("$FreeBSD$");
 #include <getopt.h>
 #include <libgen.h>
 #include <limits.h>
+#include <setjmp.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -98,6 +99,13 @@ static int disk_fd[NDISKS];
 static int ndisks;
 static int consin_fd, consout_fd;
 
+static int need_reinit;
+
+static void *loader_hdl;
+static char *loader;
+static int explicit_loader;
+static jmp_buf jb;
+
 static char *vmname, *progname;
 static struct vmctx *ctx;
 
@@ -560,6 +568,30 @@ cb_vm_set_desc(void *arg, int vcpu, int reg, uint64_t 
 	return (vm_set_desc(ctx, vcpu, reg, base, limit, access));
 }
 
+static void
+cb_swap_interpreter(void *arg, const char *interp_req)
+{
+
+	/*
+	 * If the user specified a loader but we detected a mismatch, we should
+	 * not try to pivot to a different loader on them.
+	 */
+	free(loader);
+	if (explicit_loader == 1) {
+		perror("requested loader interpreter does not match guest userboot");
+		cb_exit(NULL, 1);
+	}
+	if (interp_req == NULL || *interp_req == '\0') {
+		perror("guest failed to request an interpreter");
+		cb_exit(NULL, 1);
+	}
+
+	if (asprintf(&loader, "/boot/userboot_%s.so", interp_req) == -1)
+		err(EX_OSERR, "malloc");
+	need_reinit = 1;
+	longjmp(jb, 1);
+}
+
 static struct loader_callbacks cb = {
 	.getc = cb_getc,
 	.putc = cb_putc,
@@ -593,6 +625,9 @@ static struct loader_callbacks cb = {
 	/* Version 4 additions */
 	.vm_set_register = cb_vm_set_register,
 	.vm_set_desc = cb_vm_set_desc,
+
+	/* Version 5 additions */
+	.swap_interpreter = cb_swap_interpreter,
 };
 
 static int
@@ -661,16 +696,12 @@ usage(void)
 int
 main(int argc, char** argv)
 {
-	char *loader;
-	void *h;
 	void (*func)(struct loader_callbacks *, void *, int, int);
 	uint64_t mem_size;
-	int opt, error, need_reinit, memflags;
+	int opt, error, memflags;
 
 	progname = basename(argv[0]);
 
-	loader = NULL;
-
 	memflags = 0;
 	mem_size = 256 * MB;
 
@@ -705,6 +736,7 @@ main(int argc, char** argv)
 			loader = strdup(optarg);
 			if (loader == NULL)
 				err(EX_OSERR, "malloc");
+			explicit_loader = 1;
 			break;
 
 		case 'm':
@@ -747,6 +779,13 @@ main(int argc, char** argv)
 		exit(1);
 	}
 
+	/*
+	 * setjmp in the case the guest wants to swap out interpreter,
+	 * cb_swap_interpreter will swap out loader as appropriate and set
+	 * need_reinit so that we end up in a clean state once again.
+	 */
+	setjmp(jb);
+
 	if (need_reinit) {
 		error = vm_reinit(ctx);
 		if (error) {
@@ -767,13 +806,15 @@ main(int argc, char** argv)
 		if (loader == NULL)
 			err(EX_OSERR, "malloc");
 	}
-	h = dlopen(loader, RTLD_LOCAL);
-	if (!h) {
+	if (loader_hdl != NULL)
+		dlclose(loader_hdl);
+	loader_hdl = dlopen(loader, RTLD_LOCAL);
+	if (!loader_hdl) {
 		printf("%s\n", dlerror());
 		free(loader);
 		return (1);
 	}
-	func = dlsym(h, "loader_main");
+	func = dlsym(loader_hdl, "loader_main");
 	if (!func) {
 		printf("%s\n", dlerror());
 		free(loader);
@@ -790,7 +831,7 @@ main(int argc, char** argv)
 	addenv("smbios.bios.vendor=BHYVE");
 	addenv("boot_serial=1");
 
-	func(&cb, NULL, USERBOOT_VERSION_4, ndisks);
+	func(&cb, NULL, USERBOOT_VERSION_5, ndisks);
 
 	free(loader);
 	return (0);

More information about the svn-src-all mailing list