Can not build kernel on 1GB VM

I have a VM with 1GB RAM running FreeBSD 12.1-RELEASE-p3

I'm trying to upgrade the machine to 12.3 and having swap failures.

This machine runs bird to advertise BGP, ssh and not much else so
the small amount of RAM is (usually) fine.

For a long time, there was a 1 GB swap file which handled the
occasional time when excess memory got used.

Machine needs a custom kernel for BGP, the conf file consists of:
   include GENERIC
   ident ROUTING
   options TCP_SIGNATURE


Today, while building the 12.3 kernel with:
   cd /usr/src
   sudo make toolchain
   sudo make buildkernel KERNCONF=ROUTING
the machine ran out of swap. with a bunch of messages like:
   Apr 15 12:11:26 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 240593, size: 4096
   Apr 15 12:11:35 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 236224, size: 16384
   Apr 15 12:11:37 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 245, size: 12288
   Apr 15 12:11:46 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 240593, size: 4096
   Apr 15 12:11:55 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 236224, size: 16384
   Apr 15 12:11:57 g1 kernel: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 245, size: 12288

Thinking it was a sawp space issue, I increased the swap to 4 GB and
tried again with the same results. Boot gave the kern.maxswzone message,
I ignored it as I had planned to change as soon as I completed the build.

So I pulled up top in a console window and watched swap during the
build.  About 400 MB of RAM was free and about 3 MB of swap was
used when the machine started linking the kernel:
   ctfmerge -L VERSION -g -o kernel.full ...
While this command was running, I saw swap usage go to ~5MB (so
just over 1%), then started seeing processes being killed due to
out of swap space.

So, how to proceed?