Re: Restarting poudriere

On 2021-Jun-14, at 12:03, Mark Millard <marklmi at yahoo.com> wrote:

> bob prohaska <fbsd_at_www.zefox.net> wrote on
> Date: Mon, 14 Jun 2021 09:28:39 -0700 :
> 
>> Thanks to much support on the lists it looks like poudriere _almost_
>> worked on the Pi3. 
> 
> I'm presuming the RPi3B+ is being used as aarch64 instead
> of armv7.
> 
>> In earlier tries builds of llvm10 failed from lack of memory. Restraining
>> poudriere with -J 2 and MAKE_JOBS_NUMBER=2 in /etc/make.conf, along with
>> turning off use of tmpfs appears to have relieved the excess memory pressure.
>> 
>> Restarting the poudriere session still fails with llvm10, reporting
>> 
>> c++: error: no such file or directory: '/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/lib/Target/Hexagon/Disassembler/HexagonDisassembler.cpp'
>> c++: error: no input files
>> ninja: build stopped: subcommand failed.
>> *** Error code 1
>> 
>> Similarly, the build of rust stopped with
>> 
>> 10: fatal error: 'PPCTargetMachine.h' file not found
>> #include "PPCTargetMachine.h"
>>         ^~~~~~~~~~~~~~~~~~~~
>> 1 error generated.
>> ninja: build stopped: subcommand failed.
>> thread 'main' panicked at '
>> command did not execute successfully, got: exit code: 1
>> 
>> It looks like artifacts from the previous build failures are somehow
>> persisting into subsequent work.
> 
> poudriere does not do that. Each builder (job) is started
> from scratch, only reusing already built packages (via
> reinstallation in the temporary jail). That includes its
> jail-local /wrkdirs/. . . content being built up from
> scratch (from distribution files).
> 
>> There are no indications of memory
>> or swap insufficiency. 
>> 
>> Can anybody suggest a fix, short of starting over from scratch?
>> Perhaps building the offending packages individually, or in separate
>> jails? I've been using the same jail for all attempts so far.
> 
> I will note that on arm (aarch64 and armv7) I use:
> 
> # more /usr/local/etc/poudriere.d/options/devel_llvm10/options 
> # This file is auto-generated by 'make config'.
> # Options for llvm10-10.0.1_3
> _OPTIONS_READ=llvm10-10.0.1_3
> _FILE_COMPLETE_OPTIONS_LIST=BE_AMDGPU CLANG DOCS EXTRAS LIT LLD LLDB LLD_LINK OPENMP PYCLANG BE_FREEBSD BE_NATIVE BE_STANDARD
> OPTIONS_FILE_SET+=BE_AMDGPU
> OPTIONS_FILE_SET+=CLANG
> OPTIONS_FILE_SET+=DOCS
> OPTIONS_FILE_SET+=EXTRAS
> OPTIONS_FILE_SET+=LIT
> OPTIONS_FILE_SET+=LLD
> OPTIONS_FILE_SET+=LLDB
> OPTIONS_FILE_SET+=LLD_LINK
> OPTIONS_FILE_SET+=OPENMP
> OPTIONS_FILE_UNSET+=PYCLANG
> OPTIONS_FILE_UNSET+=BE_FREEBSD
> OPTIONS_FILE_SET+=BE_NATIVE
> OPTIONS_FILE_UNSET+=BE_STANDARD
> 
> in part in order to avoid building for targeting non-arm
> architectures. So: no Hexagon or powerpc* . (Less time
> wasted in my context.)
> 
> Because of that use, I'm unsure if I would get the same
> as you report from using BE_FREEBSD or BE_STANDARD
> instead on aarch64.
> 
> It is possible to have poudriere produce a (compressed)
> tar of the /wrkdirs/usr/ports/devel/llvm10/work/ content
> that can later be extracted someplace and looked at.
> Producing these for llvm10 tends to take significant time
> once it starts. The files are put at paths that look like:
> 
> /usr/local/poudriere/data/wrkdirs/JAILNAME/default/*
> 
> Depending on your poudriere.conf content, you might
> already have such a compressed tar. But I'm not sure
> that you would want to see what was under:
> 
> . . ./llvm-10.0.1.src/lib/Target/Hexagon/. . .
> 
> or if . . ./llvm-10.0.1.src/lib/Target/Hexagon/ even
> exists. (I'm not sure what you would do with the
> information beyond reporting it.)

I should also have noted that poudriere leaves
logs around to look at, of the form:

/usr/local/poudriere/data/logs/bulk/JAILNAME/latest-per-pkg/llvm10-10.0.1_5.log

(using llvm10-10.0.1_5 as an example).

With the log you can see the steps that it takes, including
what OPTIONS were in use, extracting from dist files,
installing packages, and so on. Toward the end should be the
errors that stopped the build. The command lines that were
in use are present.

The log shows a list of the targets:

-- Targeting AArch64
-- Targeting AMDGPU
-- Targeting ARM
-- Targeting BPF
-- Targeting Hexagon
-- Targeting Lanai
-- Targeting Mips
-- Targeting MSP430
-- Targeting NVPTX
-- Targeting PowerPC
-- Targeting RISCV
-- Targeting Sparc
-- Targeting SystemZ
-- Targeting WebAssembly
-- Targeting X86
-- Targeting XCore

(The above was from an amd64 context that does build
llvm10 for multiple targets.)


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)