Re: WITH_BEARSSL: -8112 bytes available

On Sat, Jun 3, 2023 at 9:03 AM FreeBSD User <freebsd@walstatt-de.de> wrote:

> Am Wed, 31 May 2023 12:15:12 -0600
> Warner Losh <imp@bsdimp.com> schrieb:
>
> Sorry for the late response.
>
>
> > On Mon, May 29, 2023 at 2:59 AM FreeBSD User <freebsd@walstatt-de.de>
> wrote:
> >
> > > Hello,
> > >
> > > on CURRENT, enabling in /etc/src.conf
> > >
> > > WITH_BEARSSL=
> > >
> > > seems to result in a slightly enlarged loader binary, which seems to
> have
> > > a fixed size
> > > supposed on the error I get. See below.
> > >
> > > The system is amd64 (64 bit), for the record.
> > >
> > > Somewhere in the past developers mentioned this upcoming problem and
> > > provided a knob to adjust
> > > the used size - I forgot about that knob and I couldn't find it even in
> > > the loader docs - or
> > > looked at the wrong places.
> > >
> > > Can someone help me out here?
> > >
> > > The first error stops compileing world/kernel, but taking a second run,
> > > the error goes away.
> > >
> > > Kind regards and thanks in advance,
> > >
> > > oh
> > >
> > >
> > >
> > > [...]
> > > --- all_subdir_stand/efi ---
> > > SOURCE_DATE_EPOCH=1451606400  objcopy -j .peheader -j .text -j .sdata
> -j
> > > .data  -j .dynamic -j
> > > .dynsym -j .rel.dyn  -j .rela.dyn -j .reloc -j .eh_frame -j
> > > set_Xcommand_set  -j
> > > set_Xficl_compile_set  --output-target=efi-app-x86_64 loader_4th.sym
> > > loader_4th.efi ---
> > > all_subdir_stand/i386 ---
> > >
> > > -8112 bytes available 7.71 real        12.86 user         3.08 sys
> >
> >
> > bummer. I hate it when it's that close.
> >
> > You can try setting LOADERSIZE=560000 in your environment. We currently
> set
> > the maximum to 550,000
>
> I tried to find find anything related to LOADER or SIZE in the docs. I
> remember you mentioned
> the existence of that variable months ago, but with no clue what to look
> after, it is almost
> impossible yo figure out as a non developer what the right knob might be.
>

Fair point.


> A grep -r on /usr/src shows up only in
>
> [...]
> ./stand/i386/loader/Makefile:LOADERSIZE?=       550000          # Largest
> known safe size for
> loader.bin
> ./stand/i386/loader/Makefile:   @set -- `ls -l ${.TARGET}` ;
> x=$$((${LOADERSIZE}-$$5)); \
> [...]
>
> There is no sign/trace of it in any man page related to loader and
> sibblings. I found the
> variable rather quickly after knowing what to look after.
>

To be fair, this is a very under documented area, semi on purpose...  But
that's a good point that it is
under documented. And now that the size is creeping up and we have options
that explode things,
maybe it should be better documented.


> > bytes because that's the most conservative number due to variation in the
> > available BIOS space available.
> > This likely can be set even higher if you don't have add-in cards that
> are
> > consuming space in the lower 640k
> > of memory. 640k is the absolute limit, but you need 20-30k of stack for
> the
> > loader so pushing this much past
> > 625,000 or maybe 630,000 increases the risk of run-time crashes as the
> > stack smashes through the top of
> > the loader program. You may also have to disable the lua build, since it
> > uses more stack and is just a smidge
> > larger than the forth build. _simp will be the smallest of them all. On
> my
> > system, without bearssl, I see:
> > -r-xr-xr-x  3 root  wheel  503808 May 22 15:25 /boot/loader_lua
> > -r-xr-xr-x  1 root  wheel  446464 May 22 15:25 /boot/loader_4th
> > -r-xr-xr-x  1 root  wheel  385024 May 22 15:25 /boot/loader_simp
>
> In my case, with supposedly blewn up loader size by BEARSSL enables, it is:
>
> -r-xr-xr-x  1 root  wheel  - 503808  3 Juni 12:33 /boot/loader_4th*
> -r-xr-xr-x  1 root  wheel  - 643584  3 Juni 12:33 /boot/loader_4th.efi*
> -r-xr-xr-x  1 root  wheel  - 503808  3 Juni 07:45 /boot/loader_4th.old*
> -r-xr-xr-x  3 root  wheel  - 569344  3 Juni 12:33 /boot/loader_lua*
> -r-xr-xr-x  2 root  wheel  - 737280  3 Juni 12:33 /boot/loader_lua.efi*
> -r-xr-xr-x  1 root  wheel  - 569344  3 Juni 07:45 /boot/loader_lua.old*
> -r-xr-xr-x  1 root  wheel  - 446464  3 Juni 12:33 /boot/loader_simp*
> -r-xr-xr-x  1 root  wheel  - 589312  3 Juni 12:33 /boot/loader_simp.efi*
> -r-xr-xr-x  1 root  wheel  - 446464  3 Juni 07:45 /boot/loader_simp.old*
>
> on FreeBSD 14.0-CURRENT #58 main-n263387-556b43492297: Fri Jun  2 20:19:55
> CEST 2023 amd64.
> > which suggests a ~60k bump for adding forth and ~115k bump for lua. So
> the
> > 560,000 may need to be 625,000
> > which is living life on the edge for 4th, and simply too big for lua.
> >
> > I'd be open to adding docs on this, since I don't think this option is
> > currently documented since I added it
> > to experiment around with a good value.
>
> See above, personally I'd like to see some hints on that variable, even if
> I do not fiddle
> around with it.
>

OK. that makes sense. In fact, it may make sense to disable the build of
the BIOS
loader entirely sometimes, which we can't easily do today.


> >
> > And no, I really do not want to support 'loadable modules'. BIOS booting
> is
> > on the way out, and people
> > that want to do complex stuff in the boot loader will simply have to do
> > that in UEFI or maybe kboot/LinuxBoot.
> > There's low RoI on adding this complexity, imho. We'd be better off,
> imho,
> > making things like the graphics
> > console optional since the fonts and code for that free up about 30k in
> > stupid experiments that I've done
> > (it's hard since vidconsole has a lot of calls into the graphics system
> > that aren't optional and easy to disable,
> > so I've had to do hack and slash to produce a super ugly result that is
> > only suggestive of the final savings):
> > -rw-r--r--  1 imp  imp  352256 May 31 12:04 loader_simp
> > I don't know if I slashed too much, or not enough since the code is
> rather
> > hard to separate out, so if you
> > really wanted to go down this path, it would take a lot of work and
> patient
> > understanding to make it so with
> > the low end of savings 20k and the high end on the order of maybe 40k.
> >
> > There's likely other ways to conserve space. We've not had space issues
> > with loader, et al, in the past,
> > so it's not well setup for subsetting. Though the different filesystem
> > support might also net you a fair amount:
> > LOADER_NET_SUPPORT?=    yes
> > LOADER_NFS_SUPPORT?=    yes
> > LOADER_TFTP_SUPPORT?=   yes
> > LOADER_CD9660_SUPPORT?= yes
> > LOADER_EXT2FS_SUPPORT?= yes
> > LOADER_MSDOS_SUPPORT?=  yes
> > LOADER_UFS_SUPPORT?=    yes
> > LOADER_GZIP_SUPPORT?=   yes
> > LOADER_BZIP2_SUPPORT?=  yes
> > as would compiling w/o ZFS, which uses its own method (eg
> > WITHOUT_LOADER_ZFS). Tuning the loader
> > at this level does start to get into the weeds a bit, but can offer ~40k
> > savings turning off all but NET and UFS:
> > -rw-r--r--  1 imp  imp  344064 May 31 12:11 loader_simp
> > you get even about ~100k when you disable ZFS support with
> > -DWITHOUT_LOADER_ZFS:
> > -rw-r--r--  1 imp  imp  241664 May 31 12:12 loader_simp
> > (both of these are with the graphics console enabled without the silly
> > hacks to see how much that takes up).
> > Without the extras and ZFS, you might have bearssl and lua together
> even...
> >
> > Hope this helps.
>
> Very much appreciated, thanks.
>
> I'm not so much concerned by the space sonsumption or a corrupted boot
> process, I don't even
> know whether I really need this i386 stuff mentioned here, since all of my
> boxes are UEFI,
> except the APU2, which are driven by SeaBIOS and bootin  non UEFI.
>

Yea,  I'm starting to think that the best way forward should include some
way to turn off
the BIOS loader if you know you won't need it and it's too big.


> The cosmetic issue is that whenever anything reagrding the loader gets
> touched by the
> compiler, out build system drops out with  an error - a second run then
> performs clean (which
> is a kind of weird I guess).
>

Yea, the check should cause it to be deleted when it fails, but doesn't.

Warner


> >
> > Warner
>
> Thank you very much,
>
> Kind regards
> Oliver
>
>
> --
> O. Hartmann
>