Re: How to upgrade custom release

On 9/5/24 11:08, Sad Clouds wrote:
> Hello, could anyone please suggest a good way to upgrade FreeBSD base
> system which was built with custom options in src.conf.
> 
> So, reading the official documentation, I'm presented with the
> following issues:
> 
> 1. The freebsd-update(8) seems to only work with the official release
> versions. Any custom or development releases built locally from source
> are not likely to work?

   I believe you would need to do one more rebuild+reinstall without 
your changes to begin using freebsd-update. Otherwise, 
https://docs.freebsd.org/en/articles/freebsd-update-server/ describes 
self hosting it and the warning at the top implies efforts will need to 
go to pkgbase in the future. Vermaden attempted it and ran into issues 
which went unanswered for a while so switched to doing the same but with 
pkgbase and had success. Vermaden has posts about most everything in 
this topic if I recall.

> 2. Another way to upgrade is to do this locally:
> 
> 1) make buildworld buildkernel
> 2) make installkernel && reboot

   To single user mode is fine, like you get with `shutdown now`
   2.1) etcupdate -p

> 3) make installworld && reboot

   that reboot is not 'yet' necessary

> 4) etcupdate

   /usr/src/UPDATING recommends -B parameter though I don't know when it 
should/shouldn't be used.

> 5) make check-old delete-old

   Don't need to run check to run delete; its used to list what would be 
done instead of doing it. `make delete-old -DBATCH_DELETE_OLD_FILES` is 
much less tedious than answering yes to everything.
   Now I'd add that reboot from step 3.
   Step 6 should be done 'after' packages have been upgraded; add 
relevant `pkg upgrade` effort here first. Restart programs/services as 
needed or reboot computer.

> 6) make check-old-libs delete-old-libs

   Again the check-... isn't necessary and -DBATCH_DELETE_OLD_FILES will 
help by answering yes.

> But this gets quite cumbersome with many different FreeBSD VMs. I'm
> looking for an upgrade process similar to how NetBSD does it.
> 
> Specifically:
> 
> 1) Build release. This creates binary tarballs: base.tgz, etc.tgz,
> games.tgz, man.tgz and so on.
> 2) Copy tarballs to target VM and unpack them into / except etc.tgz and
> xetc.tgz.
> 3) Run "etcupdate -s /path/to/etc.tgz -s /path/to/xetc.tgz" The NetBSD
> etcupdate script performs interactive update/merge of /etc files and
> also generates commands to delete/update old libraries.

   I'm not familiar with etcupdate steps for use on tgz file data; it is 
a 3-way diff used to compare configuration changes between an original 
state, your modifications, and an upgraded state so I presume a workflow 
could be done.
   I don't know pkgbase workflow yet but seems 'in development' for 
documents I find despite some people being regular users and testers of 
it. The most minimal steps seem to involve an install not being modified 
much when converting to pkgbase, even going so far as to convert to it 
before completing configuration steps in the installer and performing 
those after. The more that has changed, the more has to be reviewed and 
the initial migration steps haven't sounded nearly as nice as the usual 
etcupdate workflow. etcupdate already knows what you have changed; it 
can be helpful to note/save those details for consideration when 
migrating to pkgbase. If using pkgbase, it is recommended to keep 
backups of your configuration.

> Can something similar be achieved with FreeBSD? I don't want to be
> building and running make commands on every single VM. I just want to
> copy tarballs, unpack them, update /etc and remove old/obsolete files.

   The fastest+simplest way will likely become pkgbase to perform those 
steps assuming etc migration is well managed. 
https://wiki.freebsd.org/PkgBase has some good information but I don't 
know how up to date it is (wiki is usually not up to date when I need it 
for topics). https://lists.freebsd.org/subscription/freebsd-pkgbase is 
also likely good to follow.
   If every system is not a different version or built with its own
custom options then I see 3 possibilities:
   1. Run it once and share the build directory skipping to step "2)" 
for each machine. This sounds like it maintains your current workflow 
with the least amount of changes
   2. Share a self hosted pkgbase repo among the machines if you require 
'some' customization from what you can find on pkgbase repos for 
RELEASE, STABLE, and CURRENT builds. Again, I'm not familiar with 
configuration update steps as pkgbase is a newer system. I do not know 
if old libs are kept around or software requires an update at that time 
to function properly; since the primary goal is `pkg` updates base and 
other packages with the same tool (though not yet at the same time) I 
presume keeping old libraries is deemed unnecessary or is likely a 
pkgbase bug.
   3. Share resulting tgz installer packages that you built. `man 
release` to learn to begin such build steps. Cleaning out old files and 
merging changes is not part of the toolset as these are used for 
new+fresh installations by the installer and no notes are created about 
what was installed or where; this is why this would be the least 
desirable and trying to create fixes for this scenario is what leads to 
pkgbase as an effort. I believe it is planned to have the installer use 
pkgbase from the start of a system being installed. A workaround to do 
it cleanly would be using boot environments to make clean 
partition(s)/dataset(s) to extract them to and copy/edit your 
configuration into the new system.

   Though not part of your original question, if you have to build your 
own system due to desired changes to how base and/or kernel are built, 
then if you haven't already you can speed up the build using ccache or 
ccache4 and additional speedup setting up WITH_META_MODE (`man 
src-env.conf` for more details). If any of the VMs are on separate 
hardware, you may get additional performance using distcc depending on 
the hardware of each and networking performance. These are all 
set-and-forget steps other than periodically manually cleaning up ccache 
repo if used on ZFS.

> Thanks.
>