Running BSD from RAM memory using zroot

[Gary Jennejohn]

On Sun, 23 Aug 2020 05:51:39 -0700
guyvert at secmail.pro wrote:

> Hello list,
> 
> I am trying to get FreeBSD to operate entirely within RAM, thus no disks.
> Either boot via PXE/NFS/TFTP or via a SD card or eMMC that is read-only.
> 
> I now have:
> Filesystem	Size	Used	Avail	Capacity Mounted on
> /dev/md0	18M	16M	956K	95%	/
> zroot		1.7G	512M	1.2G	29%	/
> devfs		1.0K	1.0K	0B	100%	/dev
> 
> The /dev/md0 is of type preloaded, and loaded from ISO CD using mfsroot
> variables in /boot/loader.conf. The ZFS pool is called 'zroot' and has
> only one filesystem, though more can be created like zroot/var/log and set
> quota's on that filesystem so logs cannot claim too much RAM memory.
> 
> With mount we can see the FSID, but we cannot unmount the /dev/md0 because
> unmounting root filesystems appears to be explicitely forbidden, as 'man
> umount' says:
> "The root file system cannot be forcibly unmounted."
> 
> But that is only by current design, right? We already have another root
> filesystem (the zroot) and no open files should be in use on the original
> root filesystem (/dev/md0). Interestingly, while /dev/md0 cannot be
> unmounted, the actual block device can be forcibly destroyed. This appears
> to work without problems, because the mounted /dev/md0 is not read from or
> written to. But of course, i would like to get rid of it altogether, and
> just have the zroot filesystem and no more /dev/md0.
> 
> My question to you guys is, would any of you like to work on changing this
> behaviour? It would allow for a neat setup where, once booted, the system
> runs inside RAM using a memory disk (mdconfig) with a ZFS pool on top.
> Once the ZFS root filesystem is populated with the base and kernel, the
> mountpoint is changed to root:
> 
> zfs set mountpoint=/ zroot
> 
> That simple, now the system runs on RAM. To reduce RAM usage, compression
> can be enabled, and soon zstd compression will be available.
> 
> Advantages:
> - No more unstable system if the system disk has problems, crashes, bad
> sectors, corruption, whatever.
> - Potentially faster because we read from RAM and not from disk.
> - Does not wear out MicroSD/eMMC/SSD flash because it is used read-only.
> - Better than tmpfs because of compression options, as well as many other
> cool features of ZFS like snapshots etc.
> - RAM can be encrypted if desired by using 'geli onetime /dev/md1' and
> using md1.eli as the zroot disk. This would help against Cold Boot Attack.
> 
> Disadvantage would be slightly longer boot time, complexity and by default
> about 0.5GB RAM used, which can grow as the system is used. But that is
> worth it for me. And you can always mount something persistent on say /var
> and /home and leave the rest as volatile in-RAM filesystem.
> 
> To avoid double RAM caching (first the md block device, then ZFS ARC
> cache) the primarycache and secondarycache can be changed to prevent ZFS
> from caching anything and read everything from block device instead.
> 
> Even better would be to adapt ZFS to allow for in-memory filesystems. Then
> the ARC would be the actual backing and those caches would be persistent
> until the pool is destroyed or the system is reboot. But that requires
> more work, currently i'm already excited about running BSD from RAM.
> 
> But my question: is there any way to force unmount of the unneeded
> /dev/md0 which is preloaded md (mfsroot) ? I see two approaches:
> 
> 1. Adapt the umount utility to allow force unmount of root filesystem,
> which is currently forbidden. Should be a one-line patch?
> 2. Rewrite the init program so that no preloaded md is required, but init
> does the work of creating md disk, zroot pool and populating the base
> system  and kernel from archive (base.txz and kernel.txz which reside on
> MicroSD/eMMC/ISO).
> 
> Anyone thinks this is interesting?
> 

Take a look at mount.conf(5).  It seems like this may do what you want.

-- 
Gary Jennejohn