Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr

On 8/10/22 8:31 PM, Warner Losh wrote:
> The branch main has been updated by imp:
> 
> URL: https://cgit.FreeBSD.org/src/commit/?id=39fdad34e220c52a433e78f20c8c39412429014e
> 
> commit 39fdad34e220c52a433e78f20c8c39412429014e
> Author:     Warner Losh <imp@FreeBSD.org>
> AuthorDate: 2022-08-11 03:19:01 +0000
> Commit:     Warner Losh <imp@FreeBSD.org>
> CommitDate: 2022-08-11 03:29:20 +0000
> 
>      stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr
>      
>      The BIOS method of booting imposes an absolute limit of 640k for the
>      size of the program being run due to btx. In practice, this means that
>      programs larger than about 500kiB will fail in odd ways as the stack /
>      heap will overflow.

Technically the heap is now always above 1MB, the issue is the stack growing
down and overwriting .bss.
       
>      Pick 510,000 as the cutoff line semi-arbitrarily. loader_lua is now
>      almost too big and we want to break the build when it crosses this
>      threshold. In my experience, below 500,000 always works, above 520,000
>      always seems to fail with things getting bad somewhere between 512,000
>      to 515,000. 510,000 is as close to the line as I think we can go, though
>      experience may dictate we need to lower this in the future.
>      
>      This is at-best a stop-breakage until we have a better way to subset the
>      boot loader for BIOS booting to allow better, more fined-tuned
>      /boot/loaders for the many different environments they have to run
>      in. This likely means we'll have a graphical loader than understands a
>      few filesystmes for installation, and a non-graphical loader that
>      understands the most filesystems possible for everything else in the
>      future. Our build infrastructure needs some work before we can do that,
>      however.
>      
>      At this late date, it likely isn't worth the efforts to move parts of
>      the loader into high memory. There's a number of assumptions about where
>      the stack is, where buffers reside, etc that are fulfilled when it lives
>      in the first 640k that would need bounce buffers and/or other counter
>      measures if we were to split it up. All BIOS calls are done in 16-bit
>      mode with SEG:OFF addresses, requiring them to be in the first 640k of
>      RAM. And nearly all machines in the last decade can boot with UEFI
>      (though there's some exceptions, so it isn't worth killing outright
>      yet).

Fully agree that we just want to keep the BIOS loader on a sufficient feature
diet.

>      Sponsored by:           Netflix
>      Reviewed by:            kevans
>      Differential Revision:  https://reviews.freebsd.org/D36129

You really want to apply the size check to loader.bin, not loader.  The memory
layout down in the first 1MB for boot loaders is roughly:

0x0000: real-mode IDT
0x0400: BIOS data
0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc.
0x1000: various BTX global data like GDT, TSS, IDT, stacks
0x9000: BTX kernel
0xa000: BTX client (loader.bin)
0xa0000: top of BTX client stack (though this can be a bit lower for cases like
          PXE booting)

The real size constraint is on the BTX client (loader.bin) and the fact that
it's text/data/bss plus stack need to fit into that 576k window (give or take).
btxldr isn't stored in low memory, so its size isn't relevant, and BTX's code
always takes up a full page even though it is much smaller.

In theory pxeboot's total size needs to fit into the window at 0x7c00 - 0xa0000,
but in practice that limit is much larger since the size of pxeldr plus the BTX
kernel is much smaller than 0xa000 - 0x7c00.

I would say that you might want the PXE size to be even lower (maybe 4k or so?)
than the "plain" disk loader as PXE ROMs grab some of the memory ending at
0xa0000 to use for data buffers.  I don't have a firm number I can recall of how
much they grab hence my guess of about 4k or so.

-- 
John Baldwin