From nobody Sat May 07 15:32:37 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 887DA1ABD51B for ; Sat, 7 May 2022 15:32:52 +0000 (UTC) (envelope-from michaelsprivate@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KwWfC3Wq0z3pvJ for ; Sat, 7 May 2022 15:32:51 +0000 (UTC) (envelope-from michaelsprivate@gmail.com) Received: by mail-il1-x136.google.com with SMTP id z12so6598693ilp.8 for ; Sat, 07 May 2022 08:32:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jWUlQRxMnrZGj7Eix01X95Bo15Q82y/E0D6FrjvLX1I=; b=ALySjmYIVkfsgEZm5taYL9WU7AyyUiA8DxhefiMGfNiD+iWP+RfcyY8Lvao39ajJ43 NDa4t753Ow3cWLCpJvOvseSmErj8LlZo6j/KOOqEDwslQa6SBE5D3E+0+svVNLUQV8iT d8vgzz+oE4TSVtoIt1pDgbvRrTePyZMQvaekgkGeQLYXegHey7jSbj0FhEHG98iwYgH3 sFG5J9GNlctCgDx3M8HZVZTVVFxYbp9ff0O7Vb17TR4JlKPmEU+MoA4mBDJv2ikbO6/g watuYFbzojpDZYR9SOuvXs2YbS72e1pl8s+n0TKXVtTIXANEKfCQiJC+54dNVcaMWH0f GKPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jWUlQRxMnrZGj7Eix01X95Bo15Q82y/E0D6FrjvLX1I=; b=b9ObS4TOQufc2meH+72ikkxaww6a6dKUzeUlcQ/UhnA6XS3jhB5OHkyHJWWxpJIHib v8SoJRudCVlx8PZ4hvBaC00dyptO+1Mjn5OTj8IcTcNg9He7Pa2qQdyG1juIjqmcbokf v6y1wIrsB35GmAgH/gvPlUoIQC5bCTyi6A9kfjM+u1xcYSfexpgHToiI3rjygq/rDJ/H m4qdGtQd05V+u9HPMpcGUZnV00IIJbhnnirKGyr9jbKsAezWXZZ/MWvMOPypr9O22yFW jskX8U9Rs7h0hAJl9sK5KaX7u7tDDsqjnBXdwAn3NW89TjbdhjFBbgtWOf9XxKsFD8sN UIWg== X-Gm-Message-State: AOAM531DJiphUnS6zqDxlZ3rHP3JO3kMrenakuPJfcpxuACVoNI+vzF3 kzG7i99s6hUhTkB8iZCnMb1PEVKt8DhpqFlDqaez4PWBdd5lV3Y6 X-Google-Smtp-Source: ABdhPJyOstY//Phms8MTDERgIALKLQXxtZfCP9xnV9QwoiOkyrhJB1JZITOy4vcxaVwErAnGf0nzUHdvvmkt/7EncGo= X-Received: by 2002:a05:6e02:5a2:b0:2cf:8e6e:a5ab with SMTP id k2-20020a056e0205a200b002cf8e6ea5abmr1569137ils.63.1651937565372; Sat, 07 May 2022 08:32:45 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Michael Schuster Date: Sat, 7 May 2022 17:32:37 +0200 Message-ID: Subject: Re: FreeBSD, boot environments and /dev To: Wes Maag Cc: FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4KwWfC3Wq0z3pvJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=ALySjmYI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of michaelsprivate@gmail.com designates 2607:f8b0:4864:20::136 as permitted sender) smtp.mailfrom=michaelsprivate@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::136:from]; MLMMJ_DEST(0.00)[freebsd-current]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi Wes, finally I get round to responding (below) ... On Fri, May 6, 2022 at 12:15 AM Wes Maag wrote: > > > > On Thu, May 5, 2022 at 4:10 PM Michael Schuster wrote: >> >> Hi all, >> >> while still working (slowly) on an answer to my own question on the >> right workflow to keep current up to date reliably with boot >> environments, I noticed that after creating and mounting a new BE, >> that new BE's /dev (eg /mnt/dev) is very sparsely populated: >> >> [22:44:45: ~ 0] $ ll /mnt/dev >> total 20 >> 9 dr-xr-xr-x 4 root wheel 7 Apr 18 00:35 . >> 9 drwxr-xr-x 23 root wheel 30 May 5 22:44 .. >> 1 drwxr-xr-x 2 root wheel 2 Apr 18 00:35 fd >> 1 lrwxr-xr-x 1 root wheel 12 Apr 17 20:41 log -> /var/run/log >> 1 -rw-r--r-- 1 root wheel 63 May 3 22:19 null >> 1 -rw-r--r-- 1 root wheel 1 May 3 22:18 null.bak >> 1 drwxr-xr-x 2 root wheel 2 Apr 18 00:35 shm >> [22:44:48: ~ 0] $ >> >> (no matter whether I use "beadm create" or "bectl create -r") >> >> I can then mount devfs: >> # mount -t devfs devfs /mnt/dev >> >> and it will look (and work) as expected (eg for "pkg -c /mnt update" >> (*)) ... as long as the BE is mounted. >> When I try to boot from (into?) that BE though, it fails, and on the >> console I can see messages about missing /dev/... entries. I sneaked a >> peek into beinstall.sh, found no inspiration there, I'm afraid. >> >> I believe I noticed this apparent requirement to mount devfs >> separately not so long ago ... definitely this year, while I've been >> experimenting with boot environments for quite a bit longer. Was I >> just lucky all along, or did something change ... and, more >> importantly, how do I make my boot environments bootable again? >> >> TIA for hints, pointers, advice >> Michael >> >> *) I first noticed something was amiss when "pkg -c /mnt ... " >> complained about /dev/null missing, IIRC ... >> -- >> Michael Schuster >> http://recursiveramblings.wordpress.com/ >> recursion, n: see 'recursion' >> > > getting /dev/null errors with pkg -c makes sense. I wouldn't expect /dev to contain anything after a create. as I wrote, I *thought* that worked automatically ... apparently not. > Why it is not populated on boot seems odd to me though... It's possible, if you created a deep boot environment, you created a pool/ROOT/dev dataset > and it is overwriting the initial /dev from boot (I have not tested this theory). Tough to say without more information about the BE layout, and maybe a look at fstab. (after beadm create and beadm mount): $ zfs list -tall [...] tank/ROOT/BE_20220507_171901_adm 184K 397G 17.9G / tank/ROOT/BE_20220507_171901_adm/ports 8K 397G 1.92G /usr/ports tank/ROOT/BE_20220507_171901_adm/src 8K 397G 2.57G /usr/src $ mount | grep dev devfs on /dev (devfs) /dev/nvd0p1 on /boot/efi (msdosfs, local) devfs on /compat/linux/dev (devfs) fdescfs on /compat/linux/dev/fd (fdescfs) tmpfs on /compat/linux/dev/shm (tmpfs, local) $ cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/nvd0p1 /boot/efi msdosfs rw 2 2 /dev/nvd0p3 none swap sw 0 0 proc /proc procfs rw 0 0 > FWIW I typically run "pkg -r /tmp/be_mount.XXXX" after mounting with "bectl mount" which should give you the same effect as -c without the rooted devfs requirement good point ... until now, "pkg -c" worked as expected, I never looked further ;-) I'll keep it in mind! I expected (perhaps naively) that with -c, pkg would be using the "new" stuff I'd installed in the previous step (see below), not the currently running bits ... Any thoughts on that? One more data point, perhaps that's relevant: I've been updating these BEs using a sequence of installing freshly built current into /mnt (using "DESTDIR=/mnt"), following by a few ports (drm-devel, primarily), and then doing pkg update/upgrade. thx Michael -- Michael Schuster http://recursiveramblings.wordpress.com/ recursion, n: see 'recursion'