FYI: x11/xscreensaver appears to have a build-race

[Mark Millard]

On 2019-Aug-18, at 01:27, Mark Millard <marklmi at yahoo.com> wrote:



> On 2019-Aug-18, at 01:03, Niclas Zeising <zeising at freebsd.org> wrote:
> 
>> On 2019-08-18 09:48, Mark Millard wrote:
>>> On 2019-Aug-18, at 00:34, Niclas Zeising <zeising at freebsd.org> wrote:
>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote:
>>>>> I ran two separate devel/poudriere-devel amd64->aarch64
>>>>> cross builds on the same system (head -r 351178 based)
>>>>> with the same /usr/ports/ tree (ports head -r509171),
>>>>> building the same 97 ports each, mostly overlapping
>>>>> in time, and one got:
>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2
>>>>> while building x11/xscreensaver but the other
>>>>> completed building x11/xscreensaver just fine.
>>>>> There was also a armv7-targeting poudriere-devel ports
>>>>> update running, also mostly overlapping in time. With
>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was
>>>>> frequently over 90. (The armv7 x11/xscreensaver build
>>>>> also completed fine.)
>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=yes and all 3 set
>>>>> up to have "Building 97 packages using 28 builders".
>>>>> devel/llvm90 was one of the ports being built. All 3
>>>>> poudriere's had native-xtools in use. The context has
>>>>> ECC DRAM, 96 GiBytes worth.
>>>> 
>>>> I'm just about to update x11/xscreensaver, but through multiple poudriere runs I haven't noticed any issues.  I have a much smaller machine to do builds on though.
>>>> It would be interesting to see more context of the error as well.
>>> Here you go:
>> 
>> Thank you
>> 
>>> The overall build is still in progress. So I've not had
>>> a chance to run poudriere again to have it retry just
>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest
>>> build okay.)
>>> A race suggests that MAKE_JOBS_UNSAFE=yes may well be appropriate.
>> 
>> I'll look into it.  However, this is the only report I have of this issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow down the build quite a lot.
>> 
>> The error is a little strange, since it looks like it is when generating apple_png.h.  I wonder which file it is that it complains about not being there...
> 
> From what I can tell:
> 
> ../../utils/bin2c apple.png gen/apple_png.h
> 
> is supposed to produce gen/apple_png.h from the binary
> file apple.png . Presuming that is true, the:
> 
> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2
> 
> indicates that the Makefile tried to reference
> gen/apple_png.h before ../../utils/bin2c had gotten
> around to creating gen/apple_png.h (the race).

I looked up with "the Makefile" would be for apple_png.h
and it was:

/wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile

via its lines:

bsod.o: images/gen/apple_png.h

and:

bsod:           bsod.o          $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG)
        $(CC_HACK) -o $@ $@.o   $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) $(PNG_LIBS) $(THRL)

and:

# Make sure the images have been packaged. This is the first one hit:
images/gen/som_png.h:
        cd $(srcdir)/images && $(MAKE)
images/gen/6x10font_png.h:
        cd $(srcdir)/images && $(MAKE)


The presumption that images/gen/som_png.h and images/gen/6x10font_png.h
will have their actions (including sub-makes) completed before other
things such as "bsod.o: images/gen/apple_png.h" are attempted ( without
MAKE_JOBS_UNSAFE=yes ) is false. This is the source of the race that I
find in looking around.

This kind of structure needs MAKE_JOBS_UNSAFE=yes for reliability.
Otherwise a different structure is needed for reliability.


> Similar points go for gen/atm_png.h gen/atari_png.h and
> gen/earth_png.h .
> 
> 
> The original 3 builds completed so I started poudriere again
> for the failing one, without the machine being otherwise busy.
> The result was:
> 
> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | xscreensaver-5.42
> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | xscreensaver-5.42: Success
> 
> This too suggests a race condition.
> 
> If the FreeBSD port-build servers report this sort of failure
> at some point, you may then want to change things to avoid
> the failures on those servers so that dependent ports are not
> skipped.




===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)