FYI: x11/xscreensaver appears to have a build-race

[Mark Millard]

[I missed an important word: "always" in front
of "completed".]

On 2019-Aug-18, at 02:05, Mark Millard <marklmi at yahoo.com> wrote:


> 
> On 2019-Aug-18, at 01:27, Mark Millard <marklmi at yahoo.com> wrote:
> 
> 
> 
>> On 2019-Aug-18, at 01:03, Niclas Zeising <zeising at freebsd.org> wrote:
>> 
>>> On 2019-08-18 09:48, Mark Millard wrote:
>>>> On 2019-Aug-18, at 00:34, Niclas Zeising <zeising at freebsd.org> wrote:
>>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote:
>>>>>> I ran two separate devel/poudriere-devel amd64->aarch64
>>>>>> cross builds on the same system (head -r 351178 based)
>>>>>> with the same /usr/ports/ tree (ports head -r509171),
>>>>>> building the same 97 ports each, mostly overlapping
>>>>>> in time, and one got:
>>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2
>>>>>> while building x11/xscreensaver but the other
>>>>>> completed building x11/xscreensaver just fine.
>>>>>> There was also a armv7-targeting poudriere-devel ports
>>>>>> update running, also mostly overlapping in time. With
>>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was
>>>>>> frequently over 90. (The armv7 x11/xscreensaver build
>>>>>> also completed fine.)
>>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=yes and all 3 set
>>>>>> up to have "Building 97 packages using 28 builders".
>>>>>> devel/llvm90 was one of the ports being built. All 3
>>>>>> poudriere's had native-xtools in use. The context has
>>>>>> ECC DRAM, 96 GiBytes worth.
>>>>> 
>>>>> I'm just about to update x11/xscreensaver, but through multiple poudriere runs I haven't noticed any issues.  I have a much smaller machine to do builds on though.
>>>>> It would be interesting to see more context of the error as well.
>>>> Here you go:
>>> 
>>> Thank you
>>> 
>>>> The overall build is still in progress. So I've not had
>>>> a chance to run poudriere again to have it retry just
>>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest
>>>> build okay.)
>>>> A race suggests that MAKE_JOBS_UNSAFE=yes may well be appropriate.
>>> 
>>> I'll look into it.  However, this is the only report I have of this issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow down the build quite a lot.
>>> 
>>> The error is a little strange, since it looks like it is when generating apple_png.h.  I wonder which file it is that it complains about not being there...
>> 
>> From what I can tell:
>> 
>> ../../utils/bin2c apple.png gen/apple_png.h
>> 
>> is supposed to produce gen/apple_png.h from the binary
>> file apple.png . Presuming that is true, the:
>> 
>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2
>> 
>> indicates that the Makefile tried to reference
>> gen/apple_png.h before ../../utils/bin2c had gotten
>> around to creating gen/apple_png.h (the race).
> 
> I looked up with "the Makefile" would be for apple_png.h
> and it was:
> 
> /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile
> 
> via its lines:
> 
> bsod.o: images/gen/apple_png.h
> 
> and:
> 
> bsod:           bsod.o          $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG)
>        $(CC_HACK) -o $@ $@.o   $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) $(PNG_LIBS) $(THRL)
> 
> and:
> 
> # Make sure the images have been packaged. This is the first one hit:
> images/gen/som_png.h:
>        cd $(srcdir)/images && $(MAKE)
> images/gen/6x10font_png.h:
>        cd $(srcdir)/images && $(MAKE)
> 
> 
> The presumption that images/gen/som_png.h and images/gen/6x10font_png.h
> will have their actions (including sub-makes) completed before other

I should have typed: "always completed" above, not just "completed".

> things such as "bsod.o: images/gen/apple_png.h" are attempted ( without
> MAKE_JOBS_UNSAFE=yes ) is false. This is the source of the race that I
> find in looking around.
> 
> This kind of structure needs MAKE_JOBS_UNSAFE=yes for reliability.
> Otherwise a different structure is needed for reliability.
> 
> 
>> Similar points go for gen/atm_png.h gen/atari_png.h and
>> gen/earth_png.h .
>> 
>> 
>> The original 3 builds completed so I started poudriere again
>> for the failing one, without the machine being otherwise busy.
>> The result was:
>> 
>> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | xscreensaver-5.42
>> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | xscreensaver-5.42: Success
>> 
>> This too suggests a race condition.
>> 
>> If the FreeBSD port-build servers report this sort of failure
>> at some point, you may then want to change things to avoid
>> the failures on those servers so that dependent ports are not
>> skipped.
> 






===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)