From nobody Mon May 06 20:41:16 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VYCx937J7z5JHqf; Mon, 06 May 2024 20:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VYCx90tSRz4GHm; Mon, 6 May 2024 20:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1715028077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v6lTLEqhSLxqHrmGBl/0BjXuXVsYPDnyoxx7eiytvrU=; b=SSfUA47xlqmefUs+8wHeeKfHcCahmCbfxIEzpCv83A0hvID0BaWyNI1VvUNGyUnOfzGrQD po3YL671XR1SD2RZ1C0d9KcuXo9nrFCz8A+fSh/pBr/WLP8aUcI9U2YOcl1IvovDQH8Tj1 O/S8ZN22TD3NSGe1LCtnJAsShDxShkTHSlsGkAt/E4J1TGINjD7W/MT6qPugaaEpilg9Lg 4ovS3D/mVh9Q3+y+PLk+3Jyl5H9hcZKYLNdse575iVzrFKes3SJPp6anEZD6wYfQLm1imH 36saKQvUtR7BptK+dtUD2NstbylF2bNep52eBWw4k3fm1uGhx3gxeOuP8t4E7A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1715028077; a=rsa-sha256; cv=none; b=iRFDXR9Q/RKY7FUboGjJ0PdHKQLHsjBy1teqnx6EwZTBjqWWECqG316fopMYLWKAAQupl1 eZK7B+if1/t5/E9I0YJxWhrYnMTghTHddAvkX8Zr5IvWpS7wcI3pQcliPXzbEfATLyzL1N DohbQ52Mv8EaZZKk5XFynILh/y9pOYerEEHAM6D54hKEYgTxbJmd1vLzp1RYa3jnAUJOdv ScKsvdb/kZMYmy0MxckV16izDogZiWzGCAudHkIJLY9s2NrSCYD5bXXSdpG2TJFoRfLyNU wSEr8uzteDediKk/PEddMmDGSEY8iJRGWoa3pD9Ep8rSoqaVjVK4S0meaxGkpw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1715028077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v6lTLEqhSLxqHrmGBl/0BjXuXVsYPDnyoxx7eiytvrU=; b=K866d024v7OT3LS92vlwVTBDW6MqbB0C9US33JhSWebi8Y1gBIhUx7TJzyqz5BxrcR+qmj qekC1Ml6i7LrdOofIY9AqSsOXisWSItwEuBFwqQB61lHP+DlO4Y0bF75WTtfGNF16kyjlw Xq7NXyOhXXVxbbRx/HaZh1XaRZsl7mbEc9K2WatFiPt29JOJw8YNBzwLKtcD7z6uqUhwgo 8fmlSg9B1QbmNBajCMrdQ7Js1X00fnK7ZQds1uai79WB0k52TiM1IY3SNby2Xf4J/aYqe4 tj5Ta+y0qcuZ31wmLx1uDO74HcHqXkEFv/+htx1yS9ssB2sRKdHf9P3yCu5xAQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VYCx90P6jz1BjZ; Mon, 6 May 2024 20:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 446KfGop076354; Mon, 6 May 2024 20:41:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 446KfGti076351; Mon, 6 May 2024 20:41:16 GMT (envelope-from git) Date: Mon, 6 May 2024 20:41:16 GMT Message-Id: <202405062041.446KfGti076351@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Colin Percival Subject: git: 0b1c5628c74a - main - release: Rework vm_extra_pre_umount List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cperciva X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0b1c5628c74a37e2aa2aa3769c606d3e826302f8 Auto-Submitted: auto-generated The branch main has been updated by cperciva: URL: https://cgit.FreeBSD.org/src/commit/?id=0b1c5628c74a37e2aa2aa3769c606d3e826302f8 commit 0b1c5628c74a37e2aa2aa3769c606d3e826302f8 Author: Colin Percival AuthorDate: 2024-05-06 20:26:52 +0000 Commit: Colin Percival CommitDate: 2024-05-06 20:40:47 +0000 release: Rework vm_extra_pre_umount The vm_extra_pre_umount function in vmimage.subr served two purposes: It removed /etc/resolv.conf and /qemu (if cross-building), and it provided a function for cloudware to override in order to make cloud specific changes to the filesystem before constructing a disk image. This resulted in a number of bugs: 1. When cross-building, the emulator binary was left as /qemu in the Azure, GCE, Openstack and Vagrant images. 2. The build host's resolv.conf was left as /etc/resolv.conf in the basic-ci and basic-cloudinit images. 3. When building GCE images, a Google-specific resolv.conf file was constructed, and then deleted before the disk image was created. Move the bits needed for running code inside a VM staging directory from vm_install_base into a new vm_emulation_setup routine, and move the corresponding cleanup bits from vm_extra_pre_umount to a new vm_emulation_cleanup routine. Remove the /qemu and /etc/resolv.conf cleanups from the cloudware configuration files (where they exist) since we will now be running vm_emulation_cleanup to remove those even when vm_extra_pre_umount has been overridden. Override vm_emulation_cleanup in gce.conf since in that one case (and *only* that one case) we don't want to clean up resolv.conf (since it was constructed for the VM image rather than copied from the host). releng/14.1 candidate. MFC after: 1 week Sponsored by: https://www.patreon.com/cperciva --- release/scripts/mk-vmimage.sh | 3 ++- release/tools/azure.conf | 2 -- release/tools/ec2.conf | 5 ----- release/tools/gce.conf | 10 +++++++++- release/tools/oci.conf | 5 ----- release/tools/openstack.conf | 2 -- release/tools/vagrant-virtualbox.conf | 1 - release/tools/vagrant-vmware.conf | 1 - release/tools/vmimage.subr | 23 +++++++++++------------ 9 files changed, 22 insertions(+), 30 deletions(-) diff --git a/release/scripts/mk-vmimage.sh b/release/scripts/mk-vmimage.sh index ca6a359affbd..dfd91f43140d 100755 --- a/release/scripts/mk-vmimage.sh +++ b/release/scripts/mk-vmimage.sh @@ -97,13 +97,14 @@ main() { vm_create_base vm_install_base + vm_emulation_setup vm_extra_install_base vm_extra_install_packages vm_extra_install_ports vm_extra_enable_services vm_extra_pre_umount vm_extra_pkg_rmcache - cleanup + vm_emulation_cleanup vm_copy_base vm_create_disk || return 0 vm_extra_create_disk diff --git a/release/tools/azure.conf b/release/tools/azure.conf index b6526f21e474..9da2b19de694 100644 --- a/release/tools/azure.conf +++ b/release/tools/azure.conf @@ -53,7 +53,5 @@ EOF touch ${DESTDIR}/firstboot - rm -f ${DESTDIR}/etc/resolv.conf - return 0 } diff --git a/release/tools/ec2.conf b/release/tools/ec2.conf index f8bfe0911759..989e8a05a318 100644 --- a/release/tools/ec2.conf +++ b/release/tools/ec2.conf @@ -102,10 +102,5 @@ EOF # The first time the AMI boots, run "first boot" scripts. touch ${DESTDIR}/firstboot - if ! [ -z "${QEMUSTATIC}" ]; then - rm -f ${DESTDIR}/${EMULATOR} - fi - rm -f ${DESTDIR}/etc/resolv.conf - return 0 } diff --git a/release/tools/gce.conf b/release/tools/gce.conf index aad4a74d2e39..5ad83bcded53 100644 --- a/release/tools/gce.conf +++ b/release/tools/gce.conf @@ -118,7 +118,15 @@ EOF touch ${DESTDIR}/firstboot - rm -f ${DESTDIR}/etc/resolv.conf + return 0 +} +# Do everything except deleting resolv.conf since we construct our own +# Googlized resolv.conf file in vm_extra_install_base. +vm_emulation_cleanup() { + if ! [ -z "${QEMUSTATIC}" ]; then + rm -f ${DESTDIR}/${EMULATOR} + fi + umount_loop ${DESTDIR}/dev return 0 } diff --git a/release/tools/oci.conf b/release/tools/oci.conf index c57a7683a500..a4fe54ad3031 100644 --- a/release/tools/oci.conf +++ b/release/tools/oci.conf @@ -90,10 +90,5 @@ EOF touch ${DESTDIR}/firstboot - if ! [ -z "${QEMUSTATIC}" ]; then - rm -f ${DESTDIR}/${EMULATOR} - fi - rm -f ${DESTDIR}/etc/resolv.conf - return 0 } diff --git a/release/tools/openstack.conf b/release/tools/openstack.conf index 4c905fbae4ba..05d2d13bbb39 100644 --- a/release/tools/openstack.conf +++ b/release/tools/openstack.conf @@ -35,8 +35,6 @@ vm_extra_pre_umount() { echo 'ALL ALL=(ALL) NOPASSWD:ALL' >> \ ${DESTDIR}/usr/local/etc/sudoers.d/cloud-init - rm -f ${DESTDIR}/etc/resolv.conf - # The console is not interactive, so we might as well boot quickly. echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf echo 'beastie_disable="YES"' >> ${DESTDIR}/boot/loader.conf diff --git a/release/tools/vagrant-virtualbox.conf b/release/tools/vagrant-virtualbox.conf index 4dd7ca8953ad..9e0e430bbc88 100644 --- a/release/tools/vagrant-virtualbox.conf +++ b/release/tools/vagrant-virtualbox.conf @@ -14,5 +14,4 @@ vm_extra_pre_umount () { # Setup the Vagrant common items vagrant_common - rm -f ${DESTDIR}/etc/resolv.conf } diff --git a/release/tools/vagrant-vmware.conf b/release/tools/vagrant-vmware.conf index 52ff7f1aac1a..fff929829222 100644 --- a/release/tools/vagrant-vmware.conf +++ b/release/tools/vagrant-vmware.conf @@ -18,5 +18,4 @@ vm_extra_pre_umount () { # Setup the Vagrant common items vagrant_common - rm -f ${DESTDIR}/etc/resolv.conf } diff --git a/release/tools/vmimage.subr b/release/tools/vmimage.subr index 93658b9a03da..9a60be57acf3 100644 --- a/release/tools/vmimage.subr +++ b/release/tools/vmimage.subr @@ -80,8 +80,14 @@ vm_install_base() { echo "zfs_enable=\"YES\"" >> ${DESTDIR}/etc/rc.conf echo "zpool_reguid=\"zroot\"" >> ${DESTDIR}/etc/rc.conf echo "zpool_upgrade=\"zroot\"" >> ${DESTDIR}/etc/rc.conf + echo "kern.geom.label.disk_ident.enable=0" >> ${DESTDIR}/boot/loader.conf + echo "zfs_load=YES" >> ${DESTDIR}/boot/loader.conf fi + return 0 +} + +vm_emulation_setup() { if ! [ -z "${QEMUSTATIC}" ]; then export EMULATOR=/qemu cp ${QEMUSTATIC} ${DESTDIR}/${EMULATOR} @@ -91,15 +97,8 @@ vm_install_base() { mount -t devfs devfs ${DESTDIR}/dev chroot ${DESTDIR} ${EMULATOR} /usr/bin/newaliases chroot ${DESTDIR} ${EMULATOR} /bin/sh /etc/rc.d/ldconfig forcestart - umount_loop ${DESTDIR}/dev - cp /etc/resolv.conf ${DESTDIR}/etc/resolv.conf - if [ "${VMFS}" = zfs ]; then - echo "kern.geom.label.disk_ident.enable=0" >> ${DESTDIR}/boot/loader.conf - echo "zfs_load=YES" >> ${DESTDIR}/boot/loader.conf - fi - return 0 } @@ -133,15 +132,12 @@ vm_extra_install_packages() { if [ -z "${VM_EXTRA_PACKAGES}" ]; then return 0 fi - mkdir -p ${DESTDIR}/dev - mount -t devfs devfs ${DESTDIR}/dev chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg bootstrap -y for p in ${VM_EXTRA_PACKAGES}; do chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg install -y ${p} done - umount_loop ${DESTDIR}/dev return 0 } @@ -156,13 +152,16 @@ vm_extra_install_ports() { vm_extra_pre_umount() { # Prototype. When overridden, performs additional tasks within the # virtual machine environment prior to unmounting the filesystem. - # Note: When overriding this function, removing resolv.conf in the - # disk image must be included. + return 0 +} + +vm_emulation_cleanup() { if ! [ -z "${QEMUSTATIC}" ]; then rm -f ${DESTDIR}/${EMULATOR} fi rm -f ${DESTDIR}/etc/resolv.conf + umount_loop ${DESTDIR}/dev return 0 }