Deterministic rescue buildworld error with custom make.conf/src.conf/MAKEOBJDIRPREFIX

[Lawrence Stewart]

Hi Ian,

On 12/03/2017 10:29, Ian Lepore wrote:
> On Sun, 2017-03-12 at 10:22 +1100, Lawrence Stewart wrote:
>> Hi all,
>>
>> I'm unable to complete buildworld with 2 recent svn revs I've tried
>> (r314838 and r315059). I'm building for a slightly resource
>> constrained
>> production system so am specifying custom settings and a different
>> obj
>> tree location so I can copy it to the target system. The error
>> persists
>> after an "rm -rf /usr/obj/*", and if parallel building is disabled.
>>
>> The underlying build system built from r314838 via simple "make -C
>> /usr/src -s -j6 buildworld buildkernel" built and installed fine, so
>> the
>> problem seems to be around the use of the build customisations.
>>
>> Any clues?
>>
>> Cheers,
>> Lawrence
>>
>>
>> root at builder-head-amd64:/usr/src # cat cust_make.conf
>> KERNCONF=GENERIC-NODEBUG
>> MALLOC_PRODUCTION=YES
>>
>> root at builder-head-amd64:/usr/src # cat cust_src.conf
>> WITHOUT_PROFILE=1
>>
>> root at builder-head-amd64:/usr/src # make
>> __MAKE_CONF=/usr/src/cust_make.conf SRCCONF=/usr/src/cust_src.conf
>> MAKEOBJDIRPREFIX=/usr/obj/cust buildworld buildkernel
>> [...]
>> MK_AUTO_OBJ=no MK_TESTS=no  UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>> CC="cc -target x86_64-unknown-freebsd12.0
>> --sysroot=/usr/obj/cust/usr/src/tmp
>> -B/usr/obj/cust/usr/src/tmp/usr/bin
>> -O2 -pipe   -std=gnu99    -Qunused-arguments  "  CXX="c++  -target
>> x86_64-unknown-freebsd12.0 --sysroot=/usr/obj/cust/usr/src/tmp
>> -B/usr/obj/cust/usr/src/tmp/usr/bin -O2 -pipe -Qunused-arguments
>> -Wno-c++11-extensions  "  make .MAKE.MODE="normal curdirOk=yes"
>> .MAKE.META.IGNORE_PATHS=""  -f rescue.mk exe
>> cc -target x86_64-unknown-freebsd12.0
>> --sysroot=/usr/obj/cust/usr/src/tmp
>> -B/usr/obj/cust/usr/src/tmp/usr/bin
>> -O2 -pipe   -std=gnu99    -Qunused-arguments   -nostdlib -Wl,-dc -r
>> -o
>> cat.lo cat_stub.o
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/cat/cat.o
>> cc: error: no such file or directory:
>> '/usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/cat/cat.o'
>> *** Error code 1
>>
>> There appear to be a lot of missing .o files under the rescue obj
>> tree:
>>
>> root at builder-head-amd64:/usr/src # find
>> /usr/obj/cust/usr/src/rescue/rescue//usr -type f
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mksyntax.o
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mksyntax
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mknodes.o
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mknodes
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/sh.err.h
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/tc.const.h
>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/gethost
>>
>> compared with an obj tree on a different head system:
>>
>> find /usr/obj/usr/src/rescue/rescue/usr/ -type f | wc -l
>>     1552
>> _______________________________________________
>> freebsd-current at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd
>> .org"
> 
> The MAKEOBJDIRPREFIX variable must be set in the environment, not in
> make.conf or on the make command line (documented in build(7)).

Your assertion seems at odds with my past experience and my reading of
the man page... from build(7):

	The build may be controlled by defining make(1) variables
	described in the ENVIRONMENT section below, and by the
	variables documented in make.conf(5).

... which indicates they are make variables, not environment variables
specifically. As a concrete example, TARGET and DESTDIR are listed under
the "ENVIRONMENT" section of the man page, yet "EXAMPLES" shows:

           make TARGET=sparc64 buildworld
           make TARGET=sparc64 DESTDIR=/clients/sparc64 installworld

I've certainly always set build vars documented in the "ENVIRONMENT"
section of the man page on the make command line without issue. Pretty
sure I've set MAKEOBJDIRPREFIX from the make command line also in the
past, though perhaps it has been working for me "by accident" and a
documentation tweak is in order if the distinction you make is in fact
relevant...

Cheers,
Lawrence