zroot won't mount after 9.0-RC2 -> 9.0-RELEASE upgrade

[Joshua Isom]

On 2/14/2012 6:31 PM, Daniel Shahaf wrote:
> One of our amd64 servers runs 9.0-RC2 (releng/9.0 at r228325) with a zfs root.
>
> It fails to boot the 9.0-RELEASE (releng/9.0 at r229305) GENERIC kernel
> (self compiled) with a mountfrom error:
>
>      http://people.apache.org/~danielsh/infra/loki-20120215-mountfrom.png
>      mountfrom>  zfs:zroot
>      Trying to mount root from zfs:zroot []...
>      Mounting from zfs:zroot failed with error 2.
>
> We've tried to upgrade the zpool format 15->28; the symptoms are
> unchanged.  (The zroot fs is at version 4.)
>
> Why does 9.0-RC2 boot while 9.0-RELEASE (as /boot/testkernel) doesn't?
> What can do to boot 9.0-RELEASE from our zfs root filesystem?
>
> Thanks.
>

Try booting a fixit CD/USB and mounting the zfs pool.  It looks like 
your pool is set up like mine.  If it mounts, the way zfs mounts 
everything, you'll overlay your fixit file system with your zroot. 
It'll make it annoying to fix, especially dealing with zpool.cache.  If 
it mounts, make a copy of the cache file.

I switched from a zpool with two disks to a zmirror, with a third disk 
to help hold data.  It wasn't fun, but it can be done.  One problem I 
had was getting zpool.cache to work properly on reboot.  I had to do the 
overlay mount, and I think I killed the power to prevent zfs from 
removing the file.  I'd suspect a zpool.cache problem.  ZFS doesn't 
think your zroot was mounted so doesn't try to mount it even though the 
bootloader finds the kernel.