How to change rootfs from official RPI3 image

Tue Aug 27 18:45:23 UTC 2019

On Sat, 24 Aug 2019 12:15:35 +0200, Ricky Zhang <zhangchaowang at gmail.com>  
wrote:

> Hi Ronald,
>
> After migrating to UFS, I read from BSD Journal May/2019  
> (https://www.freebsdfoundation.org/past-issues/freebsd-for-makers/).  
> RPI3 can boot from ZFS rootfs.
>
> So I tried it on my own (see the step details below in method 2):
>
> https://github.com/rickyzhang82/FreeBSDWiki/wiki/1.-Migrate-SD-card-rootfs-to-SSD-for-RPI3#method-2-sd-card-kernel-boot--ssd-zfs-rootfs
>
> Unfortunately, it hangs when try to mount root from dataset tank/rootfs.
>
> I read between lines that the the author in the BSD Journal only mount  
> /var and /usr from ZFS dataset. The rest of rootfs still comes from SD  
> card ufs partition.
>
> My way moved the whole rootfs and home to tank/rootfs and tank/home  
> dataset. When it boots, I saw my USB SSD LED is flashing periodically.  
> The screen prints “Try to mount root from tank/roofs[]...". It seems to  
> me that the kernel struggled to find the zpool from SSD. I saw that zfs  
> module gets loaded properly from the screen. I have added two module in  
> /boot/loader.conf as Internet suggest:
>
> ...
> opensolaris_load="YES"
> zfs_load="YES”
> ...
>
> If I unplug USB SSD without any settings change, hit Ctrl-C. I can  
> override mount root from command line at run time with "ufs:ufs/rootfs”  
> rathe than “zfs:tank/rootfs". Then I can boot from SD card as before.  
> Once I login, I can run zfs list and show both tank/rootfs and tank/home  
> data set. So that confirmed my ZFS pool can be found. But somehow the  
> boot process messed up somewhere.
>
> Do you have any suggestion?
>
> thanks
> Ricky

Mmm, I'm not sure. It is a pretty trivial setup. I think the error is in  
the details. Trying this question on the freebsd-fs@ mailinglist might  
help.
Do you get an error? Or is it waiting indefinitely?

What is the output of 'zfs get mountpoint'?

If you set vfs.root.mountfrom="zfs:tank/rootfs" than you don't need to  
specify / in /etc/fstab.
And ZFS will automatically mount the subdirs of rootfs depending on the  
mountpoint setting of the ZFS volumes.

You might need to set:
zfs set mountpoint=legacy tank/rootfs
zfs set mountpoint=/home tank/home

If you want to test this you can use 'zpool import -R /mnt/tank tank' to  
temporarily mount everything under the /mnt/tank subdirs.

BTW opensolaris_load is not necessary. It is a dependency of zfs_load.

Ronald.

>
>
>> On Aug 21, 2019, at 8:15 AM, Ronald Klop <ronald-lists at klop.ws> wrote:
>>
>> Thanks. I will not take the credits for the idea though. :-)
>> The FreeBSD wiki has some more examples of similar setups:
>> https://wiki.freebsd.org/ZFSOnRoot describes how to do /boot on UFS and  
>> the rest on ZFS, which is similar to /boot on SD-card and the rest on  
>> SSD.
>> https://wiki.freebsd.org/RootOnZFS/UFSBoot
>>
>> Nice that you made a write up also. More exposure for FreeBSD is always  
>> a good thing. :-)
>>
>> Good luck and keep hacking.
>>
>> Ronald.
>> Van: Ricky Zhang <zhangchaowang at gmail.com>
>> Datum: woensdag, 21 augustus 2019 13:16
>> Aan: Ronald Klop <ronald-lists at klop.ws>
>> CC: freebsd-arm at freebsd.org
>> Onderwerp: Re: How to change rootfs from official RPI3 image
>>
>> Your idea is brilliant. It solved the dilemma.
>>
>> Ricky at router ~ $ sudo cat /etc/fstab
>> # Custom /etc/fstab for FreeBSD embedded images
>> /dev/ufs/rootfs   /bootdir       ufs     rw      1       1
>> /dev/label/gpt/ssdrootfs   /       ufs     rw      1       1
>> /dev/msdosfs/MSDOSBOOT /boot/msdos msdosfs rw,noatime 0 0
>> tmpfs /tmp tmpfs rw,mode=1777,size=50m 0 0
>>
>> Ricky at router ~ $ mount
>> /dev/label/gpt/ssdrootfs on / (ufs, local, soft-updates)
>> devfs on /dev (devfs, local, multilabel)
>> /dev/ufs/rootfs on /bootdir (ufs, local, soft-updates)
>> /dev/msdosfs/MSDOSBOOT on /bootdir/boot/msdos (msdosfs, local, noatime)
>> tmpfs on /tmp (tmpfs, local)
>>
>> I wrote the whole thing down in my wiki:  
>> https://github.com/rickyzhang82/FreeBSDWiki/wiki/1.-Migrate-SD-card-rootfs-to-SSD-for-RPI3
>>
>> Thanks
>>
>> Ricky
>>
>> On Aug 21, 2019, at 5:56 AM, Ronald Klop <ronald-lists at klop.ws> wrote:
>>
>> Sorry, I did only reply to the mailinglist. I will use reply-all now.
>>
>> You are right about uboot finding /boot/kernel and /boot/loader.conf.
>>
>> What you need to do is mount the SD-card on /bootdir; in fstab:
>> /dev/yoursdcard /bootdir ufs rw,noatime 1 2
>>
>> And a symlink from /boot -> /bootdir/boot on your SSD.
>>
>> Then installworld/installkernel will do the proper thing.
>> So the complete /boot stays on the SD-card.
>>
>> Regards,
>>
>> Ronald.
>> Van: Ricky Zhang <zhangchaowang at gmail.com>
>> Datum: woensdag, 21 augustus 2019 04:58
>> Aan: freebsd-arm at freebsd.org
>> Onderwerp: Re: How to change rootfs from official RPI3 image
>>
>> > BTW: Yes, u-boot is opensource: /usr/ports/sysutils/u-boot-rpi3
>> > There are sysutils/u-boot-* ports for different system. With  
>> sysutils/u-boot-master as the main part of it.
>> >
>> > https://www.freshports.org/sysutils/u-boot-rpi3  
>> <https://www.freshports.org/sysutils/u-boot-rpi3>
>> > http://www.denx.de/wiki/U-Boot <http://www.denx.de/wiki/U-Boot>
>> >
>> > Regards,
>> > Ronald.
>>
>>
>> Hi Ronald,
>>
>> Sorry, if I messed up the mailing list thread. I didn’t receive your  
>> email directly. Instead, I got your reply from daily digest. I have to  
>> copy subject and quote your reply manually in my email client. I have  
>> no idea how to fix it after reading all FAQ  
>> (https://www.freebsd.org/doc/en_US.ISO8859-1/articles/mailing-list-faq/article.html#etiquette  
>> <https://www.freebsd.org/doc/en_US.ISO8859-1/articles/mailing-list-faq/article.html#etiquette>).  
>> Any other mailing list I subscribed didn’t work this way...
>>
>> In any case, the magic works. I did rsync:
>>
>> rsync -aAXvr --progress --delete /* /mnt/USB \
>>     --exclude='/boot/msdos/*' \
>>     --exclude='/dev/*' \
>>     --exclude='/proc/*' \
>>     --exclude='/net/*' \
>>     --exclude='/tmp/*' \
>>     --exclude='/mnt/*' \
>>     --exclude='/media/*'
>>
>> I confirmed that it mount ssd as rootfs:
>>
>> Ricky at router ~ $ df -h
>> Filesystem                  Size    Used   Avail Capacity  Mounted on
>> /dev/label/gpt/ssdrootfs    407G    5.5G    369G     1%    /
>> devfs                       1.0K    1.0K      0B   100%    /dev
>> /dev/msdosfs/MSDOSBOOT       50M     13M     37M    26%    /boot/msdos
>> tmpfs                        50M    4.0K     50M     0%    /tmp
>>
>> As you said, kernel still comes from SD card. I don’t fully understand  
>> the FreeBSD boot process. Neither am I familiar with UEFI.
>>
>> - I guess /boot/msdos/uboot.bin finds the SD card roofs system. Load  
>> the kernel from /boot/kernel in SD card and scan /boot/loader.conf to  
>> find the rootfs. Please correct me if I’m wrong.
>> - Should I remove /boot folder from SSD to avoid confusion?
>> - Are there any guide how to compile and deploy kernel?
>>
>>
>>
>>
>> _______________________________________________
>> freebsd-arm at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"