qemu-arm-static appears to have problems with signal delivery during (at least) poudrirer-devel based cross builds of some ports with ALLOW_MAKE_JOBS=yes

Thu Jan 26 13:54:14 UTC 2017

On 26.01.2017 5:26, Mark Millard wrote:
> On 2017-Jan-25, at 12:27 PM, Sean Bruno <sbruno at freebsd.org> wrote:
> 
>> Mark:
>>
>> There was a recent update this week that was submitted and accepted to
>> qemu-user-static.
>>
>> Want to give it a spin again and see if you are able to make progress?
>>
>> sean "top poster for maximum effect" bruno
> 
> I updated my /usr/ports to -r432460 (from today) and rebuilt.
> I the tried doing some poudriere -x -a arm.armv6 port builds
> again, with ALLOW_MAKE_JOBS=yes and -J 1 in use.
> 
> Unfortunately the qemu-user-static update did not fix the
> problem I've been seeing.
> 
> An example extracted from a print/texinfo log still shows
> "TCG temporary leak before 00021826":

I just rebuild print/texinfo without single problem.
Well, with slightly different CFLAGS
CFLAGS+= -O2 -munaligned-access -mcpu=cortex-a15 -fno-builtin-sin
-fno-builtin-cos

Michal

....
mv warn-on-use.h-t warn-on-use.h
/bin/mkdir -p sys
rm -f sys/types.h-t sys/types.h && \
{ echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
  sed -e 's|@''GUARD_PREFIX''@|GL|g' \
      -e 's|@''INCLUDE_NEXT''@|include_next|g' \
      -e 's|@''PRAGMA_SYSTEM_HEADER''@|#pragma GCC system_header|g' \
      -e 's|@''PRAGMA_COLUMNS''@||g' \
      -e 's|@''NEXT_SYS_TYPES_H''@|<sys/types.h>|g' \
      -e 's|@''WINDOWS_64_BIT_OFF_T''@|0|g' \
      < ./sys_types.in.h; \
} > sys/types.h-t && \
mv sys/types.h-t sys/types.h
rm -f unistd.h-t unistd.h && \
..


> 
> . . .
> rm -f sys/types.h-t sys/types.h && \
> { echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
>   sed -e 's|@''GUARD_PREFIX''@|GL|g' \
>       -e 's|@''INCLUDE_NEXT''@|include_next|g' \
>       -e 's|@''PRAGMA_SYSTEM_HEADER''@|#pragma GCC system_header|g' \
>       -e 's|@''PRAGMA_COLUMNS''@||g' \
>       -e 's|@''NEXT_SYS_TYPES_H''@|<sys/types.h>|g' \
>       -e 's|@''WINDOWS_64_BIT_OFF_T''@|0|g' \
>       < ./sys_types.in.h; \
> } > sys/types.h-t && \
> mv sys/types.h-t sys/types.h
> TCG temporary leak before 00021826
> qemu: uncaught target signal 4 (Illegal instruction) - core dumped
> Illegal instruction
> gmake[2]: *** [Makefile:1174: all-recursive] Error 1
> gmake[2]: Leaving directory '/wrkdirs/usr/ports/print/texinfo/work/texinfo-6.1'
> gmake[1]: *** [Makefile:1113: all] Error 2
> gmake[1]: Leaving directory '/wrkdirs/usr/ports/print/texinfo/work/texinfo-6.1'
> ===> Compilation failed unexpectedly.
> Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
> the maintainer.
> *** Error code 1
> 
> Stop.
> make: stopped in /usr/ports/print/texinfo
> ====>> Cleaning up wrkdir
> ===>  Cleaning for texinfo-6.1.20160425,1
> build of print/texinfo ended at Wed Jan 25 20:08:32 PST 2017
> build time: 00:06:57
> !!! build failure encountered !!!
> 
> 
> ===
> Mark Millard
> markmi at dsl-only.net
> 
> On 01/15/17 07:09, Mark Millard wrote:
>> On 2017-Jan-14, at 10:53 PM, Mark Millard <markmi at dsl-only.net> wrote:
>>
>>> [Context: head (12) -r312009 and ports head -r431413.]
>>>
>>> I've been experimenting on amd64 with poudriere-devel with -x
>>> for -a arm.armv6 and I ran into:
>>>
>>>> TCG temporary leak before 00021826
>>>> qemu: uncaught target signal 4 (Illegal instruction) - core dumped
>>>
>>> in 3 of the 31 ports for the build, but 4 skipped so 3 of 27
>>> attempted. The 00021826 is the same number in all the examples
>>> so far (whatever its base).
>>>
>>> These seem to be the only TCG messages and each failure starts with
>>> one and then reports the qemu message. (Also true for the below.)
>>> As far as I can tell the TCG notice is the report of an internal
>>> qemu problem that is then translated into an Illegal instruction.
>>>
>>> This was with ALLOW_MAKE_JOBS=yes but -J 1 for poudriere.
>>>
>>> For 2 of the problem ports retries worked, still using
>>> ALLOW_MAKE_JOBS=yes and -J 1 .
>>>
>>> But the 3rd port failed each time tried with ALLOW_MAKE_JOBS=yes
>>> --but in a different step each time.
>>>
>>> In all failure cases it was gmake that got the "illegal instruction".
>>>
>>> But disabling ALLOW_MAKE_JOBS=yes appears (so far) to avoid the
>>> issue. For example, that 3rd failing port built fine. (I've
>>> been doing more ports since, with ALLOW_MAKE_JOBS=yes repeatedly
>>> failing and lack of it working.)
>>>
>>> My guess is SIGCHLD delivery sometimes touches something (or a timing)
>>> that is not handled well in qemu-arm-static. I've had not problems
>>> on an rpi2 or bpim3 in the past.
>>>
>>> (I have seen some analogous "soemtimes" issues on powerpc under
>>> and version of lang that mishandled the stack part of the ABI
>>> FreeBSD uses, SIGCHLD sometimes getting on the stack at a bad-time
>>> for the messed up code generation, leading to stack corruption. Code
>>> not getting signals had no problems.)
>>>
>>> Note: The amd64 context is FreeBSD under VirtualBox under macOS
>>> and it has had no problem for native builds of world, kernel,
>>> or ports.
>>
>> Avoiding ALLOW_MAKE_JOBS=yes is not sufficient to guarantee builds
>> will work. Here is one that got near the end before failing the
>> same way:
>>
>> . . .
>> install -m 0644 /wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/gcc-6.3.0/gcc/cp/type-utils.h /wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/stage/usr/local/lib/gcc/arm-none-eabi/6.3.0/plugin/include/cp/type-utils.h
>> install: DONTSTRIP set - will not strip installed binaries
>> TCG temporary leak before 00021826
>> qemu: uncaught target signal 4 (Illegal instruction) - core dumped
>> gmake[1]: *** [Makefile:4176: install-gcc] Illegal instruction
>> gmake[1]: Leaving directory '/wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/.build'
>> *** Error code 2
>>
>> Stop.
>> make: stopped in /usr/ports/devel/arm-none-eabi-gcc
>> ====>> Cleaning up wrkdir
>> ===>  Cleaning for arm-none-eabi-gcc-6.3.0
>> build of devel/arm-none-eabi-gcc ended at Sun Jan 15 00:04:02 PST 2017
>> build time: 02:52:28
>> !!! build failure encountered !!!
>>
>>
>> Going back to the earlier initial problem (that I happen to have the
>> material for handy): expanding the .tbz of the failed build and finding
>> the core showed:
>>
>> # find . -name "*.core" -exec file {} \;                                                                                                                                                                                                                            ./work/binutils-2.27/ld/qemu_gmake.core: ELF 32-bit LSB core file ARM, version 1 (FreeBSD), FreeBSD-style, from 'ke'
>>
>> [I've not figured out what I can do with that --or how.]
>>
>>
>> One thing unusual on my part is that I use -mcpu=cortex-a7 . That
>> matches how I historically buildworld buildkernel for installation
>> on the rpi2 and bpim3. I've never had problems like this with
>> builds on the rpi2 or the bpim3 (buildworld, buildkernel, port
>> builds). It might be that qemu-arm-static has a problem with
>> -mcpu=cortex-a7 code that is generated --but not always.
>>
>> Using the make.conf as an example:
>>
>> # more /usr/local/etc/poudriere.d/head-cortex-a7-make.conf
>> WANT_QT_VERBOSE_CONFIGURE=1
>> #
>> DEFAULT_VERSIONS+=perl5=5.24
>> WITH_DEBUG=
>> WITH_DEBUG_FILES=
>> MALLOC_PRODUCTION=
>> #
>> #system clang 3.8+ (gcc6 rejects -march=armv7a):
>> #CFLAGS+= -march=armv7-a -mcpu=cortex-a7
>> #CXXFLAGS+= -march=armv7-a -mcpu=cortex-a7
>> #CPPFLAGS+= -march=armv7-a -mcpu=cortex-a7
>> #
>> #lang/gcc6's xgcc stage considers the above conflicting so use just:
>> CFLAGS+= -mcpu=cortex-a7
>> CXXFLAGS+= -mcpu=cortex-a7
>> CPPFLAGS+= -mcpu=cortex-a7
>>
>>
>> For my context poudriere with -x for -a arm.armv6 and the use of
>> qemu-arm-static does not look reliable enough to depend on. It is
>> not obvious that the -x use contributes to the problem: it may well
>> not.
>>
>> ===
>> Mark Millard
>> markmi at dsl-only.net
>>
>>
>> _______________________________________________
>> freebsd-arm at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"
>>
> 
> 
> _______________________________________________
> freebsd-arm at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"
>