Re: git: 46ea2ffc3fbc - main - stand: Reduce limit to 500k for x86 loader

On Fri, Aug 2, 2024 at 8:29 AM Mark Johnston <markj@freebsd.org> wrote:

> On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote:
> > The branch main has been updated by imp:
> >
> > URL:
> https://cgit.FreeBSD.org/src/commit/?id=46ea2ffc3fbc42089d8322a65fdee8476d2b00d6
> >
> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6
> > Author:     Warner Losh <imp@FreeBSD.org>
> > AuthorDate: 2024-08-01 21:24:51 +0000
> > Commit:     Warner Losh <imp@FreeBSD.org>
> > CommitDate: 2024-08-01 21:30:26 +0000
> >
> >     stand: Reduce limit to 500k for x86 loader
> >
> >     The largest loader that works for PXE boot is about 500k. PXE needs
> low
> >     memory for packets and other driver state, so the largest safe size
> for
> >     the loader is about 500k. Reduce the size from 560k to 500k so we
> don't
> >     accidentally break PXE in the future.
> >
> >     Add a comment for people with special needs. If you control the
> >     hardware, it can be safe to have boot loaders as large as 580k or
> 600k
> >     in some cases. Since the BIOS loader is becoming more and more of a
> >     legacy item, the build variable LOADERSIZE isn't documented. This
> change
> >     doesn't change that: there's been little demand for this
> documentation
> >     and in general, users shouldn't change it lightly.
> >
> >     PR: 257018
> >     Sponsored by: Netflix
> > ---
> >  stand/i386/loader/Makefile | 7 ++++++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile
> > index a4aa3a3c4d45..efd442977780 100644
> > --- a/stand/i386/loader/Makefile
> > +++ b/stand/i386/loader/Makefile
> > @@ -32,7 +32,12 @@ VERSION_FILE=      ${.CURDIR}/../loader/version
> >  #
> >  # will tell you how many kiB of lomem are available.
> >  #
> > -LOADERSIZE?= 560000          # Largest known safe size for loader.bin
> > +# We further reduce this to 500k, though, to give PXE an additional 64k
> of space
> > +# so pxeloader will fit. If you have special needs that do not include
> pxeboot,
> > +# you can safely set this as high as 560000 generally, or a bit higher
> if you
> > +# have tight control over the machines you are booting on.
> > +#
> > +LOADERSIZE?= 500000          # Largest known safe size for loader.bin
>
> Hi Warner,
>
> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build.
> When enabled, the loader ends up being just slightly larger than the
> limit.
>

I sent another reply, but don't see it now. I hope it doesn't go out since
its tone was off.

So for me, on main, this combination works. I'm able to build everything
with those two options.
One can't build with just WITH_LOADER_VERIEXEC, though, you need both. Even
if it had been
too big, one can fix this with LOADERSIZE=510000. Though doing that
automatically is tricky,
for reasons below. At the very least, I should make a note of it in the
WITH_xxx options. So are
there other options you are using, or maybe a different clang version than
I have (which I think is
tip of main, but might be tip of may from early july).

However, as  the comment states: this is a special needs case. Veriexec for
BIOS is a fringe
activity compared to PXEBOOT using BIOS, so we have to draw some hard lines
in the default
settings. This is the first of the hard lines: The boot loader can't be
larger than 500k, at least until
we can get good data that larger sizes work with pxeboot. So this limit
isn't going to change until
there's new data that comes in, since there's way more pxeboot users. The
build breakage for
pxeboot is silent, and its users can be slow to resport it through the
right channels. So this
is the lessor evil: Strict limits and working pxeboot at the cost (evil)
that some people that add
in non-default options will need to do other special things.

But what to do. Do we warn when these options are enabled? Do we bump the
size? Do we
disable pxeboot? Do we just do that if it's larger than a certain amount
(or fail so unless you've
disabled it, you'll see it's too big)?

I'm inclined to (1) document you may need to bump LOADERSIZE with these
options and
(2) fail the pxeboot  build if loader_lua is > 500,000 and maybe (3) have a
WITHOUT_LOADER_PXEBOOT
option as there isn't one already. That way, if you bump the size, you may
have to also disable
pxeboot to have a successful build, but it will all be explicit and we'll
document that sometimes
too many options results in size issues that need careful testing after
overriding the safety limits.

There's also a dark horse in this: There's a bug / pull request (I can't
recall which) that uses LTO
and other weird, but clever, tricks to reduce the size of the loader. IIRC,
it was a big win for loader.efi,
but much smaller win (or maybe even a slight pessimization) for
/boot/loader. I'll look into that to
see if any of the techniques used there can eke us out some space here.

Warner