From nobody Thu Aug 11 19:38:52 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M3cYz5smzz4YYdM for ; Thu, 11 Aug 2022 19:39:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vs1-xe2d.google.com (mail-vs1-xe2d.google.com [IPv6:2607:f8b0:4864:20::e2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M3cYz5d4Fz3HtX for ; Thu, 11 Aug 2022 19:39:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vs1-xe2d.google.com with SMTP id 66so19307977vse.4 for ; Thu, 11 Aug 2022 12:39:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=diZCHArAyrD9pgSIAT072WjUzuDsX5pxnPZEVVDrJFk=; b=GQhO2DgOziapTIn8jrZpU/oP+IkyAEpOHFdAHl6KY+NxCGJqhWMGRKiesQMe2H4UpB LSsG5LbLM6T4luH4fxGJW+Nsl0HBZvPownIxHAuRjnFl6adX48FLlNGOE/nay0P8o/fZ tdxm/ywg3cDQiLzvcaAsD9UfcVwcMjnw8ANl1OOozntflIcssehDthAbbViIQnkHalX9 ua0LbC/d6S5CYNJKZ0g6E/QrqRXXwh4RI84S7rEbb1jRrJVXQNzXDiE90OdbS4yEVveq cPEmB79Re+teVv9YemgQPfWfp7GbqhtEsXI4YuqrEWlVMEHpexValuI/uWP/3pfbchM2 1JKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=diZCHArAyrD9pgSIAT072WjUzuDsX5pxnPZEVVDrJFk=; b=TBqNId4gS/lbBiBuBtMSASGIckbzMSChyN2HXTx8K8ckzVIp2KC9GL3aqfVvAZBTOQ WAj+DWw1i1DptZQkk8htlScKc6lC9mpBEYlW8vyl+0+LG8a9NLn4ofB6jK+PHrRfN7nu jIGecHb5fLGnkNL42Svqdrduv3kczYNjTtSLVMT48lB+SBgmcju2mTz2FojDsZLhNgN2 usmP90h5i+FuhwlyrzlCDwPeg+wCrW1SEAPZxvSyz/vadL8VSW7BM3iNFZNfGS81ns4r 0I8t/E1ROsssbcEEqsZRDpO3OBA+T0W/vgY5sRA44NoicpclWBmDZUwTQG+schNSG7VN WPVQ== X-Gm-Message-State: ACgBeo26KDl66/4UVxHxpbBgMal0eTkIaM5oB35zqBTKXCJwjlvQHq32 pEDx6QgPJocASwnR7KgEba/wQ1NfKPR+duLN7U1x9Q== X-Google-Smtp-Source: AA6agR4A7BheIDkPVMXD3G71nrDeo6rsQDiDOS0XwVlbgICOAAX4wlPZFIqLK9s8/abFjGzyIxDwDkdv9WNsYMLni4Y= X-Received: by 2002:a67:ac45:0:b0:388:a1ff:7e89 with SMTP id n5-20020a67ac45000000b00388a1ff7e89mr400916vsh.42.1660246743078; Thu, 11 Aug 2022 12:39:03 -0700 (PDT) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 References: <202208110331.27B3Va7M007335@gitrepo.freebsd.org> <20220811211807.0b9c8a75@thor.intern.walstatt.dynvpn.de> In-Reply-To: <20220811211807.0b9c8a75@thor.intern.walstatt.dynvpn.de> From: Warner Losh Date: Thu, 11 Aug 2022 13:38:52 -0600 Message-ID: Subject: Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr To: FreeBSD User Cc: Warner Losh , src-committers , "" , dev-commits-src-main@freebsd.org Content-Type: multipart/alternative; boundary="0000000000005c6b1005e5fc51d5" X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Rspamd-Queue-Id: 4M3cYz5d4Fz3HtX X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N --0000000000005c6b1005e5fc51d5 Content-Type: text/plain; charset="UTF-8" On Thu, Aug 11, 2022 at 1:18 PM FreeBSD User wrote: > Am Thu, 11 Aug 2022 03:31:36 GMT > Warner Losh schrieb: > > > The branch main has been updated by imp: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=39fdad34e220c52a433e78f20c8c39412429014e > > > > commit 39fdad34e220c52a433e78f20c8c39412429014e > > Author: Warner Losh > > AuthorDate: 2022-08-11 03:19:01 +0000 > > Commit: Warner Losh > > CommitDate: 2022-08-11 03:29:20 +0000 > > > > stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr > > > > The BIOS method of booting imposes an absolute limit of 640k for the > > size of the program being run due to btx. In practice, this means > that > > programs larger than about 500kiB will fail in odd ways as the stack > / > > heap will overflow. > > > > Pick 510,000 as the cutoff line semi-arbitrarily. loader_lua is now > > almost too big and we want to break the build when it crosses this > > threshold. In my experience, below 500,000 always works, above > 520,000 > > always seems to fail with things getting bad somewhere between > 512,000 > > to 515,000. 510,000 is as close to the line as I think we can go, > though > > experience may dictate we need to lower this in the future. > > > > This is at-best a stop-breakage until we have a better way to subset > the > > boot loader for BIOS booting to allow better, more fined-tuned > > /boot/loaders for the many different environments they have to run > > in. This likely means we'll have a graphical loader than understands > a > > few filesystmes for installation, and a non-graphical loader that > > understands the most filesystems possible for everything else in the > > future. Our build infrastructure needs some work before we can do > that, > > however. > > > > At this late date, it likely isn't worth the efforts to move parts of > > the loader into high memory. There's a number of assumptions about > where > > the stack is, where buffers reside, etc that are fulfilled when it > lives > > in the first 640k that would need bounce buffers and/or other counter > > measures if we were to split it up. All BIOS calls are done in 16-bit > > mode with SEG:OFF addresses, requiring them to be in the first 640k > of > > RAM. And nearly all machines in the last decade can boot with UEFI > > (though there's some exceptions, so it isn't worth killing outright > > yet). > > > > Sponsored by: Netflix > > Reviewed by: kevans > > Differential Revision: https://reviews.freebsd.org/D36129 > > --- > > stand/i386/loader/Makefile | 5 +++++ > > stand/i386/pxeldr/Makefile | 3 +++ > > 2 files changed, 8 insertions(+) > > > > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile > > index 3685281ffd2c..cde1513aac06 100644 > > --- a/stand/i386/loader/Makefile > > +++ b/stand/i386/loader/Makefile > > @@ -19,6 +19,8 @@ PROG= ${LOADER}.sym > > INTERNALPROG= > > NEWVERSWHAT?= "bootstrap loader" x86 > > VERSION_FILE= ${.CURDIR}/../loader/version > > +LOADERSIZE= 510000 # Largest known safe size > > + > > > > .PATH: ${BOOTSRC}/i386/loader > > > > @@ -79,9 +81,12 @@ CFLAGS+= -I${BOOTSRC}/i386 > > 8x16.c: ${SRCTOP}/contrib/terminus/ter-u16b.bdf > > vtfontcvt -f compressed-source -o ${.TARGET} ${.ALLSRC} > > > > + > > ${LOADER}: ${LOADER}.bin ${BTXLDR} ${BTXKERN} > > btxld -v -f elf -e ${LOADER_ADDRESS} -o ${.TARGET} -l ${BTXLDR} \ > > -b ${BTXKERN} ${LOADER}.bin > > + @set -- `${SIZE} ${.TARGET} | tail -1` ; > x=$$((${LOADERSIZE}-$$4)); \ > > + echo "$$x bytes available"; test $$x -ge 0 > > > > ${LOADER}.bin: ${LOADER}.sym > > ${STRIPBIN} -R .comment -R .note -o ${.TARGET} ${.ALLSRC} > > diff --git a/stand/i386/pxeldr/Makefile b/stand/i386/pxeldr/Makefile > > index a44dc0de2885..f8bc1eae9a31 100644 > > --- a/stand/i386/pxeldr/Makefile > > +++ b/stand/i386/pxeldr/Makefile > > @@ -13,6 +13,7 @@ BOOT= pxeboot > > LDR= pxeldr > > ORG= 0x7c00 > > LOADER= loader > > +PXELDRSIZE= 510000 # Largest known safe size > > > > .if defined(BOOT_PXELDR_PROBE_KEYBOARD) > > CFLAGS+=-DPROBE_KEYBOARD > > @@ -41,5 +42,7 @@ CLEANFILES+= ${LOADER} > > ${LOADER}: ${LOADERBIN} ${BTXLDR} ${BTXKERN} > > btxld -v -f elf -e ${LOADER_ADDRESS} -o ${.TARGET} -l ${BTXLDR} \ > > -b ${BTXKERN} ${LOADERBIN} > > + @set -- `${SIZE} ${.TARGET} | tail -1` ; > x=$$((${PXELDRSIZE}-$$4)); \ > > + echo "$$x bytes available"; test $$x -ge 0 > > > > .include > > > > To make a long story short: > > Having WITH_BEARSSL=YES > > in /etc/src.conf blow off the loader. Can this be fixed? > I think so. But now you have to either choose not to do that, use the loader_4th or wait for things like making the frame buffer code optional to come to fruition. If you aren't actually using this, the latest changes in the tree allow you to override the limit to keep it 'building' but being over by 56k means it can't possibly work. Warner --0000000000005c6b1005e5fc51d5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Aug 11, 2022 at 1:18 PM FreeB= SD User <freebsd@walstatt-de.d= e> wrote:
Am Thu, 11 Aug 2022 03:31:36 GMT
Warner Losh <imp@FreeBSD.org> schrieb:

> The branch main has been updated by imp:
>
> URL: https://= cgit.FreeBSD.org/src/commit/?id=3D39fdad34e220c52a433e78f20c8c39412429014e<= /a>
>
> commit 39fdad34e220c52a433e78f20c8c39412429014e
> Author:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org>
> AuthorDate: 2022-08-11 03:19:01 +0000
> Commit:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org>
> CommitDate: 2022-08-11 03:29:20 +0000
>
>=C2=A0 =C2=A0 =C2=A0stand: impose 510,000 byte limit for /boot/loader a= nd /boot/pxeldr
>=C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0The BIOS method of booting imposes an absolute limi= t of 640k for the
>=C2=A0 =C2=A0 =C2=A0size of the program being run due to btx. In practi= ce, this means that
>=C2=A0 =C2=A0 =C2=A0programs larger than about 500kiB will fail in odd = ways as the stack /
>=C2=A0 =C2=A0 =C2=A0heap will overflow.
>=C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0Pick 510,000 as the cutoff line semi-arbitrarily. l= oader_lua is now
>=C2=A0 =C2=A0 =C2=A0almost too big and we want to break the build when = it crosses this
>=C2=A0 =C2=A0 =C2=A0threshold. In my experience, below 500,000 always w= orks, above 520,000
>=C2=A0 =C2=A0 =C2=A0always seems to fail with things getting bad somewh= ere between 512,000
>=C2=A0 =C2=A0 =C2=A0to 515,000. 510,000 is as close to the line as I th= ink we can go, though
>=C2=A0 =C2=A0 =C2=A0experience may dictate we need to lower this in the= future.
>=C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0This is at-best a stop-breakage until we have a bet= ter way to subset the
>=C2=A0 =C2=A0 =C2=A0boot loader for BIOS booting to allow better, more = fined-tuned
>=C2=A0 =C2=A0 =C2=A0/boot/loaders for the many different environments t= hey have to run
>=C2=A0 =C2=A0 =C2=A0in. This likely means we'll have a graphical lo= ader than understands a
>=C2=A0 =C2=A0 =C2=A0few filesystmes for installation, and a non-graphic= al loader that
>=C2=A0 =C2=A0 =C2=A0understands the most filesystems possible for every= thing else in the
>=C2=A0 =C2=A0 =C2=A0future. Our build infrastructure needs some work be= fore we can do that,
>=C2=A0 =C2=A0 =C2=A0however.
>=C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0At this late date, it likely isn't worth the ef= forts to move parts of
>=C2=A0 =C2=A0 =C2=A0the loader into high memory. There's a number o= f assumptions about where
>=C2=A0 =C2=A0 =C2=A0the stack is, where buffers reside, etc that are fu= lfilled when it lives
>=C2=A0 =C2=A0 =C2=A0in the first 640k that would need bounce buffers an= d/or other counter
>=C2=A0 =C2=A0 =C2=A0measures if we were to split it up. All BIOS calls = are done in 16-bit
>=C2=A0 =C2=A0 =C2=A0mode with SEG:OFF addresses, requiring them to be i= n the first 640k of
>=C2=A0 =C2=A0 =C2=A0RAM. And nearly all machines in the last decade can= boot with UEFI
>=C2=A0 =C2=A0 =C2=A0(though there's some exceptions, so it isn'= t worth killing outright
>=C2=A0 =C2=A0 =C2=A0yet).
>=C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0Sponsored by:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0Netflix
>=C2=A0 =C2=A0 =C2=A0Reviewed by:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 kevans
>=C2=A0 =C2=A0 =C2=A0Differential Revision:=C2=A0 https://revie= ws.freebsd.org/D36129
> ---
>=C2=A0 stand/i386/loader/Makefile | 5 +++++
>=C2=A0 stand/i386/pxeldr/Makefile | 3 +++
>=C2=A0 2 files changed, 8 insertions(+)
>
> diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile > index 3685281ffd2c..cde1513aac06 100644
> --- a/stand/i386/loader/Makefile
> +++ b/stand/i386/loader/Makefile
> @@ -19,6 +19,8 @@ PROG=3D=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0${LOADER}.sym
>=C2=A0 INTERNALPROG=3D
>=C2=A0 NEWVERSWHAT?=3D=C2=A0 =C2=A0 =C2=A0 =C2=A0 "bootstrap loade= r" x86
>=C2=A0 VERSION_FILE=3D=C2=A0 =C2=A0 =C2=A0 =C2=A0 ${.CURDIR}/../loader/= version
> +LOADERSIZE=3D=C2=A0 510000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Larges= t known safe size
> +
>=C2=A0
>=C2=A0 .PATH:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0${B= OOTSRC}/i386/loader
>=C2=A0
> @@ -79,9 +81,12 @@ CFLAGS+=3D=C2=A0 =C2=A0-I${BOOTSRC}/i386
>=C2=A0 8x16.c: ${SRCTOP}/contrib/terminus/ter-u16b.bdf
>=C2=A0 =C2=A0 =C2=A0 =C2=A0vtfontcvt -f compressed-source -o ${.TARGET}= ${.ALLSRC}
>=C2=A0
> +
>=C2=A0 ${LOADER}: ${LOADER}.bin ${BTXLDR} ${BTXKERN}
>=C2=A0 =C2=A0 =C2=A0 =C2=A0btxld -v -f elf -e ${LOADER_ADDRESS} -o ${.T= ARGET} -l ${BTXLDR} \
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-b ${BTXKERN} ${= LOADER}.bin
> +=C2=A0 =C2=A0 =C2=A0@set -- `${SIZE} ${.TARGET} | tail -1` ; x=3D$$((= ${LOADERSIZE}-$$4)); \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "$$x bytes available"= ;; test $$x -ge 0
>=C2=A0
>=C2=A0 ${LOADER}.bin: ${LOADER}.sym
>=C2=A0 =C2=A0 =C2=A0 =C2=A0${STRIPBIN} -R .comment -R .note -o ${.TARGE= T} ${.ALLSRC}
> diff --git a/stand/i386/pxeldr/Makefile b/stand/i386/pxeldr/Makefile > index a44dc0de2885..f8bc1eae9a31 100644
> --- a/stand/i386/pxeldr/Makefile
> +++ b/stand/i386/pxeldr/Makefile
> @@ -13,6 +13,7 @@ BOOT=3D=C2=A0 =C2=A0 =C2=A0 =C2=A0pxeboot
>=C2=A0 LDR=3D pxeldr
>=C2=A0 ORG=3D 0x7c00
>=C2=A0 LOADER=3D=C2=A0 =C2=A0 =C2=A0 loader
> +PXELDRSIZE=3D 510000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# Larges= t known safe size
>=C2=A0
>=C2=A0 .if defined(BOOT_PXELDR_PROBE_KEYBOARD)
>=C2=A0 CFLAGS+=3D-DPROBE_KEYBOARD
> @@ -41,5 +42,7 @@ CLEANFILES+=3D ${LOADER}
>=C2=A0 ${LOADER}: ${LOADERBIN} ${BTXLDR} ${BTXKERN}
>=C2=A0 =C2=A0 =C2=A0 =C2=A0btxld -v -f elf -e ${LOADER_ADDRESS} -o ${.T= ARGET} -l ${BTXLDR} \
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-b ${BTXKERN} ${LOADERBIN}
> +=C2=A0 =C2=A0 =C2=A0@set -- `${SIZE} ${.TARGET} | tail -1` ; x=3D$$((= ${PXELDRSIZE}-$$4)); \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "$$x bytes available"= ;; test $$x -ge 0
>=C2=A0
>=C2=A0 .include <bsd.prog.mk>
>

To make a long story short:

Having WITH_BEARSSL=3DYES

in /etc/src.conf blow off the loader. Can this be fixed?

I think so. But now you have to either choose not to do t= hat, use the loader_4th or wait for things like making the frame buffer cod= e optional to come to fruition. If you aren't actually using this, the = latest changes in the tree allow you to override the limit to keep it '= building' but being over by 56k means it can't possibly work.
=

Warner
--0000000000005c6b1005e5fc51d5--