Re: difficulties replacing a ZFS installer zroot pool with a new zroot pool on a new disk
- Reply: Tomasz CEDRO : "Re: difficulties replacing a ZFS installer zroot pool with a new zroot pool on a new disk"
- Reply: David Christensen : "Re: difficulties replacing a ZFS installer zroot pool with a new zroot pool on a new disk"
- Reply: parv/FreeBSD: "Re: difficulties replacing a ZFS installer zroot pool with a new zroot pool on a new disk"
- In reply to: David Christensen : "Re: difficulties replacing a ZFS installer zroot pool with a new zroot pool on a new disk"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Wed, 30 Mar 2022 22:22:56 UTC
Greetings, I am going to top post because all of the previous discussion is moot given what I have found out to get the problem solved, where the problem is to simply replace a ZFS system pool's drive with a new drive. To begin from the start: I installed the new NVMe SSD drive and I was able to boot the USB install image and install a new FreeBSD system on it. On reboot I first tried keeping the old SATA drive as it was. However the motherboard BIOS (CSM enabled, legacy, ASUS Prime X570-PRO) refused all of my efforts to set the boot drive to the new SSD. I finally resorted to disconnecting the data cable of the old SATA drive, and the new SSD booted fine. I then powered down the motherboard, reattached the old SATA data cable, and booted. The motherboard again refused to boot the new NVMe SSD. After about an hour of fighting the BIOS, I gave up, set the SATA drive as "hot pluggable" in the BIOS, and rebooted with the SATA data cable disconnected. Once the NVMe SSD was booted, I reattached the SATA data cable and it showed up in the 'zpool import' list. 'zpool import zroot' was not a happy solution as it collided with the new SSD zroot pool. I eventually worked out that I should rename the old pool zroot.old on import. That was also not a happy solution as it continued to automatically mount itself on top of the new SSD zroot pool. I then worked out that I need to specify an altroot: zpool import -o altroot=/mnt/zroot.old zroot.old This at first glance appeared to work. But it did NOT. I was left with the complaint I made in my original email to the freebsd-questions list: Where are my subsidiary datasets, and especially their data? zfs mount -a, for instance, did nothing, very quietly. I tried zfs-mount(8)ing several of the subsidiary datasets (eg zroot.old/usr/src), and that worked! But still I was missing some important stuff, like /root and /usr/local. To get /root, I made a wild guess, and tried: zfs mount zroot.old/ROOT/default And that brought all my subsidiary datasets (and the data) back. I would submit that looking back on 30+ years of successfully performing this exercise, that having the old SSD zroot.orig automatically import and mount everything, but not automatically import and mount everything when set given an altroot, is confusing. I would add that manually performing zfs mount zroot.old/ROOT/default (but not, say zfs unmount zroot.old/usr; zfs mount zroot.old/src) mount everything I was missing, is also confusing and unintuitive. I have two more ZFS system pools to upgrade to SSDs. I am going to try the following procedure: 1) unplug the old SATA drive and install FreeBSD to the new SSD. 2) shutdown, reattach old drive, reboot If the new drive boots (instead of the old drive), I boot to single user. I then try zpool import -o altroot /mnt/zrool.old zroot zroot.old 3) If that works, I try: zfs mount zpool.old/ROOT/default 4) If it doesn't work, I'll probably have to zpool export the pool and iterate. I forget exactly how I got the old pool renamed in the above. Anyway, onwards. Russell On 3/29/22 19:12, David Christensen wrote: > On 3/29/22 16:16, Russell L. Carter wrote: >> Greetings, >> After many hours, I am stuck trying to replace my spinning rust >> drive with a new SSD. >> >> Basically I have renamed the old drive pool 'zroot.old' and imported >> it so that it mounts to /mnt/zroot2: >> >> root@bruno> zfs list | grep zroot.old >> zroot.old 89.6G 523G 96K >> /mnt/zroot2/mnt/zroot.old >> zroot.old/ROOT 37.6G 523G 96K >> none >> zroot.old/ROOT/default 37.6G 523G 37.6G >> /mnt/zroot2 >> zroot.old/export 264K 523G 88K >> /mnt/zroot2/mnt/zroot.old/export >> zroot.old/export/packages 176K 523G 88K >> /mnt/zroot2/mnt/zroot.old/export/packages >> zroot.old/export/packages/stable-amd64-default 88K 523G 88K >> /mnt/zroot2/mnt/zroot.old/export/packages/stable-amd64-default >> zroot.old/tmp 144K 523G 144K >> /mnt/zroot2/tmp >> zroot.old/usr 37.8G 523G 96K >> /mnt/zroot2/usr >> zroot.old/usr/home 582M 523G 582M >> /mnt/zroot2/usr/home >> zroot.old/usr/obj 6.14G 523G 6.14G >> /mnt/zroot2/usr/obj >> zroot.old/usr/ports 27.8G 523G 27.8G >> /mnt/zroot2/usr/ports >> zroot.old/usr/src 3.27G 523G 3.27G >> /mnt/zroot2/usr/src >> zroot.old/var 1.89M 523G 96K >> /mnt/zroot2/var >> zroot.old/var/audit 96K 523G 96K >> /mnt/zroot2/var/audit >> zroot.old/var/crash 96K 523G 96K >> /mnt/zroot2/var/crash >> zroot.old/var/log 1.32M 523G 1.32M >> /mnt/zroot2/var/log >> zroot.old/var/mail 120K 523G 120K >> /mnt/zroot2/var/mail >> zroot.old/var/tmp 176K 523G 176K >> /mnt/zroot2/var/tmp >> zroot.old/vm 14.1G 523G 615M >> /mnt/zroot2/vm >> zroot.old/vm/debianv9base 3.79G 523G 120K >> /mnt/zroot2/vm/debianv9base >> zroot.old/vm/debianv9base/disk0 3.79G 523G 3.57G - >> zroot.old/vm/debianv9n2 9.70G 523G 160K >> /mnt/zroot2/vm/debianv9n2 >> zroot.old/vm/debianv9n2/disk0 9.70G 523G 11.3G - >> root@bruno> zfs mount -a >> root@bruno> >> >> The problem is that /mnt/zroot2/usr/home, /mnt/zroot2/usr, >> /mnt/zroot2/usr/src are all empty: >> >> root@bruno> ls /mnt/zroot.old/usr >> root@bruno> >> >> Even though I can look at the individual datasets and theey're >> still using the same amount data as original. This is a bit >> unhelpful for migrating over the old configuration. >> >> The oddball mounting is just the result of several 10s of attempts to >> import and mount so that a) the original zroot pool doesn't clobber the >> new one, and b) attempts to make the datasets visible. >> >> So can someone enlighten me on the proper way to do this, and possibly >> give a hint how I can get those original datasets visible? This is >> definitely a new wrinkle for a geezer who has been doing such things >> without (nontrivial) problems for 30 years now. >> >> Yeah yeah, this is also my backup drive and I should have replicated >> infra over to another system... I'm a gonna do that next. >> >> Thanks very much, >> Russell > > > I recall attempting to install two ZFS FreeBSD OS disks in the same > machine at the same time, and the results were very confusing. I > suggest that you install only one ZFS FreeBSD OS disk at any given time. > > > If you need to work on the FreeBSD OS disk without booting it, I would > boot FreeBSD installer media and use the live system/ shell to access > the ZFS pools and datasets. I expect that you will want to set the > "altroot" property when you import any pools. I am unclear if you will > need to export the ZFS boot pool ("bootpool") or the ZFS root pool > ("zroot.old"?) if you import them (?). > > > If the HDD and SSD both have the same interface (e.g. SAS or SATA), if > the SSD is the same size or larger than the HDD, and if you can revert > your changes to the HDD so that it is a working FreeBSD instance again, > you should be able to use a live distribution to clone the HDD to the > SSD using dd(1), power down, remove the HDD and live media, connect the > SSD to the interface port the HDD was connected to, and boot the SSD. I > would use a Linux live distribution without ZFS support, to ensure that > the live distribution does not interact with any ZFS content on the HDD > or SSD before or after the clone. > > > David > > >