Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr
Date: Fri, 12 Aug 2022 17:18:17 UTC
On Fri, Aug 12, 2022 at 11:05 AM John Baldwin <jhb@freebsd.org> wrote: > On 8/11/22 6:45 PM, Warner Losh wrote: > > On Thu, Aug 11, 2022 at 6:24 PM John Baldwin <jhb@freebsd.org> wrote: > > > >> On 8/11/22 11:18 AM, Warner Losh wrote: > >>> On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <jhb@freebsd.org> wrote: > >>>> You really want to apply the size check to loader.bin, not loader. > The > >>>> memory > >>>> layout down in the first 1MB for boot loaders is roughly: > >>>> > >>>> 0x0000: real-mode IDT > >>>> 0x0400: BIOS data > >>>> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc. > >>>> 0x1000: various BTX global data like GDT, TSS, IDT, stacks > >>>> 0x9000: BTX kernel > >>>> 0xa000: BTX client (loader.bin) > >>>> 0xa0000: top of BTX client stack (though this can be a bit lower for > >> cases > >>>> like > >>>> PXE booting) > >>>> > >>>> The real size constraint is on the BTX client (loader.bin) and the > fact > >>>> that > >>>> it's text/data/bss plus stack need to fit into that 576k window (give > or > >>>> take). > >>>> btxldr isn't stored in low memory, so its size isn't relevant, and > BTX's > >>>> code > >>>> always takes up a full page even though it is much smaller. > >>>> > >>> > >>> Where does 576k come from? That's 589824 bytes, but a0000-a000 is > 614400 > >>> bytes. The delta is 24k (24576). My 'observed' value of about 515,000 > is > >>> another > >>> 75k below that, suggesting we are needing 100k of stack? Can that be > >> right? > >>> I knew > >>> lua ate a lot of stack, but wow! > >> > >> Hmm, I converted 0xa000 to 64k instead of 40k in my head which is where > >> the 576k came from. Yeah, the total size is 600k. 100k stack does seem > >> like a lot. It is possible that other BIOS ROMs besides just PXE might > >> steal data from the stack top. You could maybe try looking at some of > >> your existing BIOS-boot machines to check the 16-bit word at physical > >> address 0x413. That gives you the actual top of the 640k window. On > >> my current desktop (albeit booted via UEFI and not BIOS) it is 629k: > >> > >> % sudo dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > >> 2+0 records in > >> 2+0 records out > >> 2 bytes transferred in 0.000026 secs (75643 bytes/sec) > >> 00000000 75 02 |u.| > >> 00000002 > >> % gdb > >> ... > >> (gdb) p/d 0x275 > >> $2 = 629 > >> Still, that's only 11k gone. > >> > > > > So on the machine that I'm having issues with... > > > > # dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > > 00000000 37 02 > > # echo $((0x237)) > > 567 > > > > Super Yikes! > > Dear goodness what in the world. There must be a 64k bounce buffer or > something weird. Maybe for an HBA ROM of some sort? Oof. That would > explain why you are seeing issues that we aren't normally seeing from > other users though. > Yes. It may be because PXE booting is enabled on these machines or something. I'll have to see if playing around with that makes a difference. I'll also search for an HBA ROM that's stealing space as well. Thankfully it's only on 4 really old revisions of our hardware. But it does mean that I'll have to slowly make things optional so that I can keep these machines working until their retirement date... It's a rather substantial number today, but may be 0 this time next year, so there's a limited time horizon... Warner