Re: llvm10 build failure on Rpi3

From: bob prohaska <>
Date: Sat, 03 Jul 2021 21:54:45 UTC
On Sat, Jul 03, 2021 at 01:15:19PM -0700, Mark Millard wrote:
> So you still have not tried an artifacts or snapshot kernel+world?
Not yet. 

> > Eventually I resorted to running make in devel/llvm10, to my surprise it
> > ran to completion.
> Interesting.
> Was this -j4? -j1? -j2? Any other interesting characteristics
> for how it was run?
Nothing special was done. IIRC, it was make -DBATCH > make.log in
the background. From top's screen it looked like -j4. 
> It would be interesting to see if building in a chroot
> in that make style also worked (or a non-poudriere jail).

Can you point me to instructions for doing the experiment?

> > It also ran make package successfully. Again I tried to
> > build just devel/llvm10 using poudriere, again getting "expected expression". 
> > 
> > At that point I resized the swap partitions to 1 GB each and tried poudriere
> > on devel/llvm10. That got rid of the excessive swap warnings, but didn't help.
> > Finally I placed 
> > in /usr/local/etc/poudriere.d/make.conf and tried again. That still failed,
> > still with "expected expression". 
> I'll note that the running build build shows Load Averages
> of under 3. So the MAKE_JOBS_NUMBER=2 seems to be working.
> > Since devel/llvm10 had created a package successfully, I tried slipping a copy
> > into poudriere's package directory, hoping it would find and use the package
> > to make further progress. Unfortunately, poudriere seems to remember the failure
> > and won't use the proffered package. 
[large snip which convinced me to give up on tricking poudriere into
using a package constructed by make] 
> Going in a different direction, one way to force a build to
> start over after a failure is to: rm -fr PATH/.building
> before starting a new bulk build. This might be appropriate
I'm missing something here: what does PATH represent? There's
nothing called .building under /usr/local/poudriere, at least
after the run finishes. 

> if one suspects a problem of a kind that did not stop a
> build but produced something for a build that fails to operate
> correctly.
Such as a corrupt llmv-tblgen?
> So lang/rust finished. That is interesting because it includes an
> llvm build internally.

Does that build invoke the same llvm-tblgen?
> Again, poudriere does not control memory initialization in
> the processes in the builders.

For some reason I got the idea that whatever  asked for memory to use
was responsible for initializing it. Certainly not the kernel.....
> > The fact that the stoppage reported looks like
> > a syntax error specific to devel/llmv10 which is unaffected by swap pressure
> > makes it seem unrelated to kernel or swap constraints. 
> The files with the syntax errors are ones generated by llvm-tblgen
> during the build and it is the output of llvm-tblgen that is corrupt,
> showing evidence of having used memory not initialized like it should
> have been.
Wouldn't that point suspicion at llvm-tblgen, of whatever version
LLVM is actually doing the work? 

> > AIUI, the hardware of the Pi4 is considerably different from the Pi3 in terms
> > of memory management, noted from an interview with Eben Upton on YouTube.
> Why would Eben Upton be talking about FreeBSD's memory management?
He was talking about the Pi4 hardware and how it differed from the Pi3

> I suspect that the talk is not about what you think it is about,
> but some narrower aspects than the overall memory managment.
I thought it had something to do with added DMA capablity. The video is at
In light of the discussion about llvm-tblgen I'm doubtful it's relevant,
but it's not the worst way to waste an hour.

> > Is there any sort of sanity test for the poudriere system? If I delete and
> > re-create the existing jail can the existing package library be preserved
> > and re-used? If not, that's OK, I'd just like to know beforehand.
> > 
> # poudriere jail -jNAME -d
> # poudriere jail -c -jNAME -m null -M /WORLDPATH -S /SRCPATH -v 14.0-CURRENT
> should work fine. But really all that you are
> doing is (using an example from my environment)
> is deleting and rewriting a few very small files
> in a directory with the jail's name:
So, in my case /usr/local/poudriere/poudriere-system? 
(using the nomenclature in your sample instructions).
That would leave /usr/local/poudriere/data intact....
I'm starting to understand why you think it unlikely
to help.

> The deletion/replacement of timestamp may have rebuild
> consequences from appearing to have changed (or just
> being missing).
If timestamps guide decisions on what to make and when,
that might be significant. Not sure how I might've screwed
them up, but in my hands anything is possible 8-)

> Nothing about any of those is going to change how memory
> initialization is working in llvm-tblgen's operation
> for generating any * files, other than
> if the timestamp forces some sort of rebuild from scratch
> of some build dependencies first.
Maybe this should be obvious, but which llvm-tblgen is in 
action? the one from the system, (12.0.1) or something

Thanks for writing!

bob prohaska