Encrypted (GELI) root on ZFS troubles

[Karl Denninger]

On 10/1/2014 4:58 PM, Karl Denninger wrote:
> On 10/1/2014 4:52 PM, Andriy Gapon wrote:
>> On 02/10/2014 00:27, Karl Denninger wrote:
>>> So here's the fun part of what I'm trying to do (and getting frustrated
>>> with)
>>>
>>> I have set up a GPT disk with the following setup:
>>>
>>> =>       34  625142381  da2  GPT  (298G)
>>>          34          6       - free -  (3.0K)
>>>          40       1024    1  freebsd-boot  (512K)
>>>        1064    4194304    2  freebsd-zfs  [bootme]  (2.0G)
>>>     4195368  134217728    3  freebsd-swap  (64G)
>>>   138413096  486729312    4  freebsd-zfs  (232G)
>>>   625142408          7       - free -  (3.5K)
>>>
>>> Then on freebsd-boot I have written the bootloaders.
>>>
>>> The "bootme" filesystem has *only* the /boot directory copied over from
>>> the rest of the system's root directory (that is, the kernel, loadables,
>>> /boot/loader.conf, etc); that pool is called "zboot"
>>>
>>> Partition 4 has the label "root0" on it, and thus shows up in /dev/gpt. 
>>> I have initialized that with geli, set the boot option flag (that is,
>>> prompt on boot) and created a pool called "root" on the resulting .eli
>>> device and then put the system on that.  That's all ok.
>>>
>>> Finally, I set the bootfs on that latter pool.  There is no bootfs set
>>> on /zboot:
>>>
>>> # zpool get bootfs zboot
>>> NAME   PROPERTY  VALUE   SOURCE
>>> zboot  bootfs    -       default
>>>
>>> It is set on the root pool to the proper filesystem:
>>>
>>> # zpool get bootfs root
>>> NAME  PROPERTY  VALUE              SOURCE
>>> root  bootfs    root/R/10.1-CLEAN  local
>>>
>>> The problem is that when the system boots geli "finds" the raw device
>>> (in this case /dev/da0p4), prompts for the password and attaches there
>>> instead of in /dev/gpt.  The gpt label is missing --- and equally bad
>>> the "root" pool does not appear to import at boot time either.
>>>
>>> As a result the system tries to mount root from /zboot (even though it's
>>> not been told to, and HAS been told where to mount off the root pool),
>> As far as *I* can see, you have not told the kernel what your root fs should be,
>> so it is using a default root filesystem which the same filesystem from where
>> the kernel itself was loaded.
>>
>>> but there's no init in there (or anything else other than the boot
>>> filesystem itself) and as a result I get an immediate panic.
>>>
> Various wikis on setting this up have strongly suggested that
> /boot/loader.conf no longer needs to have the root filesystem declared
> explicitly as it is able to locate it via looking in the pool metadata. 
> Is this wrong in this specific case?
>
> (Not a huge deal if so, but it's not at all clear that's true -- and it
> doesn't do anything for the issue of geli grabbing the base device
> rather than the /dev/gpt one.)
>
Ah, the kernel will not cross a zpool to look for bootfs; if it's not
set on the pool it comes from it will not look further.  Setting it
explicitly in /boot/loader.conf worked.

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2711 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141001/dbffa6aa/attachment.bin>