Re: Restarting poudriere
- Reply: Mark Millard via freebsd-ports : "Re: Restarting poudriere"
- In reply to: Mark Millard via freebsd-ports : "Re: Restarting poudriere"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Tue, 15 Jun 2021 09:20:55 UTC
On Tue, 15 Jun 2021 01:31:32 -0700 Mark Millard <email@example.com> wrote: > Michael Gmelin <freebsd_at_grem.de> wrote on > Date: Tue, 15 Jun 2021 09:31:26 +0200 : > > On Mon, 14 Jun 2021 18:02:33 -0700 > bob prohaska <fbsd_at_www.zefox.net> wrote: > > > > > On Mon, Jun 14, 2021 at 09:52:22PM +0200, Michael Gmelin wrote: > > > > > > > > > > > > > On 14. Jun 2021, at 20:30, bob prohaska > > > > > <fbsd_at_www.zefox.net> wrote: > > > > > > > > > > ???On Mon, Jun 14, 2021 at 06:46:52PM +0200, Michael Gmelin > > > > > wrote: > > > > >> On Mon, 14 Jun 2021 09:28:39 -0700 > > > > >> What do you mean by "restarting"? > > > > >> How do you invoke poudriere exactly? > > > > > As root, > > > > > poudriere bulk -J 2 -j main x11-wm/lxqt www/chromium > > > > > > testbuild.log > > > > > > > > Doesn???t ninja handle parallel builds on its own anyway? Does > > > > it work okay if you comment out ALLOW_*_JOBS in poudriere.conf? > > > > > > > > > > The line ALLOW_MAKE_JOBS is already commented out in > > > /usr/local/etc/poudriere.conf but it's active in > > > /etc/make.conf > > > > > > I remain a bit confused about how poudriere and make coordinate > > > their parallel job spawning activity. In the latest case the -J 2 > > > on the poudriere command line put two packages under > > > construction, but the ALLOW_MAKE_JOBS line in /etc/make.conf > > > didn't result in parallel threads building LLVM10. Clearly I > > > don't understand the relationship between builders, jobs, threads > > > and (much!) more. > > > > /etc/make.conf from the host system is not used by poudriere, as it > > aims to provide clean-room builds and therefore doesn't use > > configurations that also serve a purpose on the host system. > > > > All configurations come from /usr/local/etc/poudriere.conf and > > various files in /usr/local/etc/poudriere.d/. > > I'll explicitly note that one of the files is: > > /usr/local/etc/poudriere.d/make.conf > > It is the most global make.conf in poudriere's activities. > For poudriere, someone writing about using make.conf likely > is implicitly referencing the one at that path (unless they > report something that indicates otherwise). > > Bob: Sorry I did not notice the /etc/make.conf reference in > your earlier material. I should have commented on the > distinction. > > > See also the > > CUSTOMISATION section in poudriere(8) (reading that man pages is > > useful in general). > > > > So if you want to use this feature, you need to enable it in > > poudriere.conf (but I would still recommend using > > ALLOW_MAKE_JOBS_PACKAGES instead and also only if you don't use > > multiple builders, given the limited resources of your machine). > > /usr/local/etc/poudriere.d/make.conf use should work fine > as well. Nope, ALLOW_MAKE_JOBS_PACKAGES is a poudriere setting, used here inside poudriere: https://github.com/freebsd/poudriere/blob/master/src/share/poudriere/common.sh#L4421 poudriere doesn't read make.conf configurations as its own configuration, so setting ALLOW_MAKE_JOBS_PACKAGES in there does nothing (it only appends make.conf settings to the generated /etc/make.conf inside the jail, see "append_make()" in common.sh). Also note that ALLOW_MAKE_JOBS_PACKAGES has no effect for ports tested using "poudriere testport" (but it *does* apply to building the dependencies required for such ports). So, assuming ALLOW_MAKE_JOBS_PACKAGE="someport", "poudriere testport someport" won't make use of MAKE_JOBS, while "poudriere testport otherport-that-depends-on-someport" will do so for building someport. Something that might not be obvious when reading poudriere's config: /usr/local/etc/poudriere.d/poudriere.conf is only read if /usr/local/etc/poudriere.conf doesn't exist. Other files like /usr/local/etc/poudriere.d/130amd64-poudriere.conf *will* be read though (after reading the global config). Cheers, Michael > > My guess is that Bob will explore what various combinations > do, including for hitting resource limits vs. not. He has a > long history of doing such and I've learned some (for me) > handy things from some of his explorations and what they > lead to. > > > > Perhaps I've complicated matters by using testport on earlier, > > > successful, tries. Is there a significant difference if one just > > > wants to make packages? I merely wanted to see how the > > > attempt would fail and hoped for more diagnostics. But it worked. > > > So, I tried bulk and that doesn't seem to work so well. > > > > As someone pointed out earlier in this thread, looking at the > > poudriere build log of a failed build (and maybe sharing it) could > > help to find a solution. > > > Bob has a history of posting large log files someplace for > reference. So I'd not be surprised if he does that again for > this issue at some point. > > === > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) -- Michael Gmelin