Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr

On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <jhb@freebsd.org> wrote:

> On 8/10/22 8:31 PM, Warner Losh wrote:
> > The branch main has been updated by imp:
> >
> > URL:
> https://cgit.FreeBSD.org/src/commit/?id=39fdad34e220c52a433e78f20c8c39412429014e
> >
> > commit 39fdad34e220c52a433e78f20c8c39412429014e
> > Author:     Warner Losh <imp@FreeBSD.org>
> > AuthorDate: 2022-08-11 03:19:01 +0000
> > Commit:     Warner Losh <imp@FreeBSD.org>
> > CommitDate: 2022-08-11 03:29:20 +0000
> >
> >      stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr
> >
> >      The BIOS method of booting imposes an absolute limit of 640k for the
> >      size of the program being run due to btx. In practice, this means
> that
> >      programs larger than about 500kiB will fail in odd ways as the
> stack /
> >      heap will overflow.
>
> Technically the heap is now always above 1MB, the issue is the stack
> growing
> down and overwriting .bss.
>

Fair point. I realized that after I pushed as well...

Some compilers I've used in the past have 'stack overflow' checks, but that
was done
at the userland/kernel boundary and likely would be hard to pull off here...


> >      Pick 510,000 as the cutoff line semi-arbitrarily. loader_lua is now
> >      almost too big and we want to break the build when it crosses this
> >      threshold. In my experience, below 500,000 always works, above
> 520,000
> >      always seems to fail with things getting bad somewhere between
> 512,000
> >      to 515,000. 510,000 is as close to the line as I think we can go,
> though
> >      experience may dictate we need to lower this in the future.
> >
> >      This is at-best a stop-breakage until we have a better way to
> subset the
> >      boot loader for BIOS booting to allow better, more fined-tuned
> >      /boot/loaders for the many different environments they have to run
> >      in. This likely means we'll have a graphical loader than
> understands a
> >      few filesystmes for installation, and a non-graphical loader that
> >      understands the most filesystems possible for everything else in the
> >      future. Our build infrastructure needs some work before we can do
> that,
> >      however.
> >
> >      At this late date, it likely isn't worth the efforts to move parts
> of
> >      the loader into high memory. There's a number of assumptions about
> where
> >      the stack is, where buffers reside, etc that are fulfilled when it
> lives
> >      in the first 640k that would need bounce buffers and/or other
> counter
> >      measures if we were to split it up. All BIOS calls are done in
> 16-bit
> >      mode with SEG:OFF addresses, requiring them to be in the first 640k
> of
> >      RAM. And nearly all machines in the last decade can boot with UEFI
> >      (though there's some exceptions, so it isn't worth killing outright
> >      yet).
>
> Fully agree that we just want to keep the BIOS loader on a sufficient
> feature
> diet.
>

Yes.


> >      Sponsored by:           Netflix
> >      Reviewed by:            kevans
> >      Differential Revision:  https://reviews.freebsd.org/D36129
>
> You really want to apply the size check to loader.bin, not loader.  The
> memory
> layout down in the first 1MB for boot loaders is roughly:
>
> 0x0000: real-mode IDT
> 0x0400: BIOS data
> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc.
> 0x1000: various BTX global data like GDT, TSS, IDT, stacks
> 0x9000: BTX kernel
> 0xa000: BTX client (loader.bin)
> 0xa0000: top of BTX client stack (though this can be a bit lower for cases
> like
>           PXE booting)
>
> The real size constraint is on the BTX client (loader.bin) and the fact
> that
> it's text/data/bss plus stack need to fit into that 576k window (give or
> take).
> btxldr isn't stored in low memory, so its size isn't relevant, and BTX's
> code
> always takes up a full page even though it is much smaller.
>

Where does 576k come from? That's 589824 bytes, but a0000-a000 is 614400
bytes. The delta is 24k (24576). My 'observed' value of about 515,000 is
another
75k below that, suggesting we are needing 100k of stack? Can that be right?
I knew
lua ate a lot of stack, but wow!

In theory pxeboot's total size needs to fit into the window at 0x7c00 -
> 0xa0000,
> but in practice that limit is much larger since the size of pxeldr plus
> the BTX
> kernel is much smaller than 0xa000 - 0x7c00.
>
> I would say that you might want the PXE size to be even lower (maybe 4k or
> so?)
> than the "plain" disk loader as PXE ROMs grab some of the memory ending at
> 0xa0000 to use for data buffers.  I don't have a firm number I can recall
> of how
> much they grab hence my guess of about 4k or so.
>

Hmmm... Good point on that. It likely is better to get well below the limit
than to necessarily
check for a smaller pxeldr. Or put another way, I'd rather we push the
limit down all
the time so we don't wind up in that awkward place where /boot/loader fits,
but /boot/pxeldr
doesn't.

Warner