Encrypted (GELI) root on ZFS troubles

[Karl Denninger]

On 10/2/2014 3:45 AM, Andriy Gapon wrote:
> On 02/10/2014 07:50, Karl Denninger wrote:
>> 1. Having the kernel able to cross pools and examine the bootfs property
>> would be fairly useful.
> I don't understand your suggestion.  Every pool has bootfs property (if it's not
> explicitly set then that means that it just has its default value).  So which of
> the bootfs properties must be honored and why?  At present we honor bootfs of
> the "current" pool, which is the only distinguished pool.
Except default is "from whence one came" and yes, I understand the
problem if there's a conflict.  I would argue that if the parameter is
set on the pool the kernel booted from it should be honored, but if it
is NOT set (that is, if it is defaulted) then if it is validly set on a
different pool that one should be honored.

We already do this in that if /boot/loader.conf is set that's honored,
and this sounds inviolate but it in fact is not for the below reason.

I do understand the problem in the general sense with "well what happens
if there are two or more bootfs parameters set on other pools but none
on the pool the kernel loaded from" but here's the thing -- that risk is
already there, in that there's already all sorts of "fun" possible with
adapters that don't enumerate reliably in order, especially if the
intended boot device goes offline.  My LSI adapters, in particular
(which are a great example because lots of people use these with SAS
expanders and lots of plugged in drives, myself included, as they work
well and are pretty fast), will happily look for another bootable device
that happens to be plugged in if your primary(s) are offline and, if you
have one in the machine, it will attempt to boot it.  Sucks to be you if
that turns out to be a bootable environment.

So what I think makes sense is the following:

1. If /boot/loader.conf has vfs.root.mountfrom set, honor it (yes, there
is a risk to this too; you might have booted from an unintended disk!)

2. If we booted from zfs and bootfs is set non-default on the pool you
booted from then honor it, otherwise honor the first non-default bootfs
setting you find on any pool that is mounted during kernel init (which
for a geli-encrypted pool means one that had the "-b" flag set on its
members, otherwise it's not available until after init starts.)  I
understand this has risks but so does an adapter deciding to boot from
the "wrong" disk; there's no reason for bootfs to be set non-default on
a pool that isn't used for booting.

3. If neither (1) or (2) is true then mount root from the pool/device
you loaded the kernel from.

I can certainly understand that not being the default (you could get
reamed pretty good if you booted a non-corresponding kernel .vs. the
rest of the environment) but IMHO it violated the principle of least
astonishment when I went to set this up and the implication that bootfs
on *a* pool was sufficient as a hint to the kernel.

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2711 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141002/095edee4/attachment.bin>