From nobody Wed May 01 13:01:59 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VTxzW5GLfz5JLtf; Wed, 1 May 2024 13:01:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VTxzW2gH0z3yp4; Wed, 1 May 2024 13:01:59 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1714568519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CpvRYjvMmbVonrCsUqIoLwwwJ0NZx+N9vESIHKEq1Xw=; b=GNuiiObM5fT7ytqugPrlwT7f7VglI/Cf01MkvNTvJhxNBvFnpGypfjG/8+Ep/hRwZ7gwla tiPs6ecFglxcffVwXSFO8CPgrcDGqyQMhsJ89uTrT0AtyWxsIcbKhyvYu84OATniS6j3yu cyA5lMZ4ooOhn7laxD7E6xbNmQi7wuk763amLc7fWrOuR/B9OdeCfTBvVJ4SCecmzjL5Il FurAUNT2jd7w2OA3LxLe+U2Nq91fkECeg3vYQS8yHCwULlnQJ1hcCFH9vnsjOXvstRs1TG 19rS2lMXmWzLq6t2uVffKwdzUWyuxM7fLBagYkAzhkzHJ7ANSUbhJWOwoUTqyA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1714568519; a=rsa-sha256; cv=none; b=R/0RxsGiihVHgAFnVSFB6isB8SHsZFmbZPPc9J70KtM2LUa9J+LtjfUpc67r6pNDNgQ+G+ EvqpgGcOSQuB/xWN6A7cTIOSHGW9BSvM4GBwYPYtqdlTj7aVcds41AMn//KSjFJUhy95mv x3j5kAkQ9Cw9/cJaUYF0dx5rQyh2DsFKT77e7N/4lQfXaXPZdYxfoVlsEBl51EBMGjHX+H 6WSM6K6OaX1ZmrL66VogFmclNncapY47E8X1rE65g24MY8xm04S+qhW3oicGpv8mBt/6jP N1CXUAeFdHrCDas4CIKxnqkiLg1u//PxSyMpftmzAo8sDWVB9D7rrwLeyQd1PQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1714568519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CpvRYjvMmbVonrCsUqIoLwwwJ0NZx+N9vESIHKEq1Xw=; b=pGH2zi1wZlS2/o+mT0Bc7GAb83vEg6gCcpJ0uV+R2d/eWxrXbXkKx6PcmSHLaipDqpl6j2 64rqwZ8La1q3z+BmUehH0KZwXcnzXl2lF8he1uceIizbhMuqmoro8g1SXBmQ248PzRzZ3G fX/17bvXJIbXf63bk3K6xO0uL66Uxd+v6SlPXD6huCaUcjhPXSPcBs7W/AsDzQ2ITlSdH5 DOw5dohHKq7gZ4o3C/PmMLywX+p0NVWE7uLmu4NJExuXGw4gUYlq0vQVI5al1dcpgI0wdO 3RYdjw7Xeq2BH5H6z3bs45pxLTcNURNtUe2gWQzYuj6FOsKtgQ+O4KP+FkDnAg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VTxzW1TjKzKnp; Wed, 1 May 2024 13:01:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 441D1x2v000627; Wed, 1 May 2024 13:01:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 441D1xEc000624; Wed, 1 May 2024 13:01:59 GMT (envelope-from git) Date: Wed, 1 May 2024 13:01:59 GMT Message-Id: <202405011301.441D1xEc000624@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: aa34b1d20e44 - main - vmrun.sh: Add arm64 support List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: aa34b1d20e44141749ffdecf16908fc1e5db4db6 Auto-Submitted: auto-generated The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=aa34b1d20e44141749ffdecf16908fc1e5db4db6 commit aa34b1d20e44141749ffdecf16908fc1e5db4db6 Author: Mark Johnston AuthorDate: 2024-05-01 12:36:30 +0000 Commit: Mark Johnston CommitDate: 2024-05-01 12:36:30 +0000 vmrun.sh: Add arm64 support For now, we enumerate disk devices before network devices. This is to work around a problem wherein u-boot remaps BARs during boot in a way that bhyve does not handle. Some discussion and experiments suggest that this can be handled by having bhyve not map BARs during boot on arm64; until a solution is implemented, however, this workaround is sufficient for simple usage and doesn't have any real downsides. The console and bootrom are specified slightly differently versus amd64, and a few of vmrun.sh's command-line options are amd64-only. Reviewed by: corvink, jhb Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D44933 --- share/examples/bhyve/vmrun.sh | 100 ++++++++++++++++++++++++++++++------------ 1 file changed, 71 insertions(+), 29 deletions(-) diff --git a/share/examples/bhyve/vmrun.sh b/share/examples/bhyve/vmrun.sh index 19eed87af0bf..52935363023a 100755 --- a/share/examples/bhyve/vmrun.sh +++ b/share/examples/bhyve/vmrun.sh @@ -72,29 +72,29 @@ usage() { echo " -C: console device (default: ${DEFAULT_CONSOLE})" echo " -d: virtio diskdev file (default: ${DEFAULT_VIRTIO_DISK})" echo " -e: set FreeBSD loader environment variable" - echo " -E: Use UEFI mode" - echo " -f: Use a specific UEFI firmware" + echo " -E: Use UEFI mode (amd64 only)" + echo " -f: Use a specific boot firmware (e.g., EDK2, U-Boot)" echo " -F: Use a custom UEFI GOP framebuffer size" \ - "(default: ${DEFAULT_VNCSIZE})" + "(default: ${DEFAULT_VNCSIZE}) (amd64 only)" echo " -G: bind the GDB stub to the specified address" echo " -H: host filesystem to export to the loader" echo " -i: force boot of the Installation CDROM image" echo " -I: Installation CDROM image location" \ "(default: ${DEFAULT_ISOFILE})" - echo " -l: the OS loader to use (default: /boot/userboot.so)" + echo " -l: the OS loader to use (default: /boot/userboot.so) (amd64 only)" echo " -L: IP address for UEFI GOP VNC server" \ "(default: ${DEFAULT_VNCHOST})" echo " -m: memory size (default: ${DEFAULT_MEMSIZE})" echo " -n: network adapter emulation type" \ "(default: ${DEFAULT_NIC})" echo " -p: pass-through a host PCI device (e.g ppt0 or" \ - "bus/slot/func)" + "bus/slot/func) (amd64 only)" echo " -P: UEFI GOP VNC port (default: ${DEFAULT_VNCPORT})" echo " -t: tap device for virtio-net (default: $DEFAULT_TAPDEV)" - echo " -T: Enable tablet device (for UEFI GOP)" + echo " -T: Enable tablet device (for UEFI GOP) (amd64 only)" echo " -u: RTC keeps UTC time" echo " -v: Wait for VNC client connection before booting VM" - echo " -w: ignore unimplemented MSRs" + echo " -w: ignore unimplemented MSRs (amd64 only)" echo "" [ -n "$msg" ] && errmsg "$msg" exit 1 @@ -111,6 +111,12 @@ if [ $? -ne 0 ]; then exit 1 fi +platform=$(uname -m) +if [ "${platform}" != amd64 -a "${platform}" != arm64 ]; then + errmsg "This script is only supported on amd64 and arm64 platforms" + exit 1 +fi + force_install=0 isofile=${DEFAULT_ISOFILE} memsize=${DEFAULT_MEMSIZE} @@ -121,7 +127,6 @@ tap_total=0 disk_total=0 disk_emulation=${DEFAULT_DISK} loader_opt="" -bhyverun_opt="-H -P" pass_total=0 # EFI-specific options @@ -133,7 +138,21 @@ vncport=${DEFAULT_VNCPORT} vncsize=${DEFAULT_VNCSIZE} tablet="" -while getopts aAc:C:d:e:Ef:F:G:hH:iI:l:L:m:n:p:P:t:Tuvw c ; do +# arm64 only +uboot_firmware="/usr/local/share/u-boot/u-boot-bhyve-arm64/u-boot.bin" + +case ${platform} in +amd64) + bhyverun_opt="-H -P" + opts="aAc:C:d:e:Ef:F:G:hH:iI:l:L:m:n:p:P:t:Tuvw" + ;; +arm64) + bhyverun_opt="" + opts="aAc:C:d:e:f:F:G:hH:iI:L:m:n:P:t:uv" + ;; +esac + +while getopts $opts c ; do case $c in a) bhyverun_opt="${bhyverun_opt} -a" @@ -161,7 +180,7 @@ while getopts aAc:C:d:e:Ef:F:G:hH:iI:l:L:m:n:p:P:t:Tuvw c ; do efi_mode=1 ;; f) - efi_firmware="${OPTARG}" + firmware="${OPTARG}" ;; F) vncsize="${OPTARG}" @@ -246,12 +265,25 @@ if [ ${pass_total} -gt 0 ]; then bhyverun_opt="${bhyverun_opt} -S" fi -if [ ${efi_mode} -gt 0 ]; then - if [ ! -f ${efi_firmware} ]; then - echo "Error: EFI Firmware ${efi_firmware} doesn't exist." \ - "Try: pkg install edk2-bhyve" - exit 1 +if [ -z "$firmware" ]; then + case ${platform} in + amd64) + firmware="${efi_firmware}" + firmware_pkg="edk2-bhyve" + ;; + arm64) + firmware="${uboot_firmware}" + firmware_pkg="u-boot-bhyve-arm64" + ;; + esac +fi + +if [ -n "${firmware}" -a ! -f "${firmware}" ]; then + echo "Error: Firmware file ${firmware} doesn't exist." + if [ -n "${firmware_pkg}" ]; then + echo " Try: pkg install ${firmware_pkg}" fi + exit 1 fi make_and_check_diskdev() @@ -317,7 +349,7 @@ while [ 1 ]; do installer_opt="" fi - if [ ${efi_mode} -eq 0 ]; then + if [ ${platform} = amd64 -a ${efi_mode} -eq 0 ]; then ${LOADER} -c ${console} -m ${memsize} ${BOOTDISKS} \ ${loader_opt} ${vmname} bhyve_exit=$? @@ -329,15 +361,19 @@ while [ 1 ]; do # # Build up args for additional tap and disk devices now. # - nextslot=2 # slot 0 is hostbridge, slot 1 is lpc - devargs="" # accumulate disk/tap args here - i=0 - while [ $i -lt $tap_total ] ; do - eval "tapname=\$tap_dev${i}" - devargs="$devargs -s $nextslot:0,${nic},${tapname} " - nextslot=$(($nextslot + 1)) - i=$(($i + 1)) - done + devargs="-s 0:0,hostbridge" # accumulate disk/tap args here + case ${platform} in + amd64) + console_opt="-l com1,${console}" + devargs="$devargs -s 1:0,lpc " + nextslot=2 # slot 0 is hostbridge, slot 1 is lpc + ;; + arm64) + console_opt="-o console=${console}" + devargs="$devargs -o bootrom=${firmware} " + nextslot=1 # slot 0 is hostbridge + ;; + esac i=0 while [ $i -lt $disk_total ] ; do @@ -349,6 +385,14 @@ while [ 1 ]; do i=$(($i + 1)) done + i=0 + while [ $i -lt $tap_total ] ; do + eval "tapname=\$tap_dev${i}" + devargs="$devargs -s $nextslot:0,${nic},${tapname} " + nextslot=$(($nextslot + 1)) + i=$(($i + 1)) + done + i=0 while [ $i -lt $pass_total ] ; do eval "pass=\$pass_dev${i}" @@ -372,16 +416,14 @@ while [ 1 ]; do if [ ${efi_mode} -gt 0 ]; then efiargs="-s 29,fbuf,tcp=${vnchost}:${vncport}," efiargs="${efiargs}${vncsize}${vncwait}" - efiargs="${efiargs} -l bootrom,${efi_firmware}" + efiargs="${efiargs} -l bootrom,${firmware}" efiargs="${efiargs} ${tablet}" fi ${FBSDRUN} -c ${cpus} -m ${memsize} ${bhyverun_opt} \ - -s 0:0,hostbridge \ - -s 1:0,lpc \ ${efiargs} \ ${devargs} \ - -l com1,${console} \ + ${console_opt} \ ${installer_opt} \ ${vmname}