Upgrade to 12-Release system not booting kernel panic

[Kaya Saman]

Hi,


I've been digging around a little on this one and it seems that there is 
some ZFS wierdness going on.


My original install was an 11.2-RELEASE fresh install and upon an 
upgrade to 12-RELEASE the system is having issues booting.


I have a ZFS root pool called zroot; by default this mounts at 
zroot/ROOT/default and a few other non-root / bootable ZFS pools.


The system is a SuperMicro SC216 chassis with LSI non-RAID HBA.


I have the boot disks ada0 and ada1 plugged into the rear of the chassis 
and directly into the systemboard which is also a SuperMicro. These 
drives are both Samsung SSD's.

The 22 drive slots at the front of the chassis are occupied by the other 
various data pools.


So here is some strangeness... if I remove all 22 drives from the front, 
the system boots fine but straight after boot goes into kernel panic 
mode and reboots before I can even look at the error or get to the login 
prompt.


With the non-root pools installed at the BTX loader after scanning 
through all the bios drives I get a bunch of:

read 264 from ... to 0x...., error 0x10 errors

then:

ZFS: i/o error - all block copies unavailable

ZFS: can't read MOS of pool <non-root-pool-names>


after this the system simply hangs?


I have tried looking around but everything mentioning the MOS error is 
talking about the root pool, a particularly good reference is here: 
http://freebsd.1045724.x6.nabble.com/ZFS-i-o-error-in-recent-12-0-td6245865.html


In fact I did try to boot with a USB stick and go into Live mode then 
import all the pools on the system. This works without any issue! The 
pools are fine the data is there everything looks normal.

- I also rebuilt the zpool.cache according to the link just incase there 
was some kind of corruption there, however upon reboot I still get the 
same issue??


Looking at a bug report with a kernel panic: 
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220923#c17


I have attempted to add:


kern.cam.scsi_delay="50000"

kern.cam.boot_delay="50000"


into the /boot/loader.conf file but unfortunately the issue still 
continues :-(


I wonder if there is a way to tell to tell the system to only look at 
certain drives for booting??


There is this line in my loader.conf:

vfs.root.mountfrom="zfs:zroot"


It maybe the wrong hunch I have but it seems like the system is looking 
for "zroot" on all pools instead of the actual root pool hence the above 
errors??


Would anyone be able to suggest anything or have any ideas about how to 
get the system back online and booting??


Thanks.


Kaya