Re: Restarting poudriere

On 2021-Jun-15, at 02:20, Michael Gmelin <freebsd at grem.de> wrote:

> On Tue, 15 Jun 2021 01:31:32 -0700
> Mark Millard <marklmi@yahoo.com> wrote:
> 
>> Michael Gmelin <freebsd_at_grem.de> wrote on
>> Date: Tue, 15 Jun 2021 09:31:26 +0200 :
>> 
>> On Mon, 14 Jun 2021 18:02:33 -0700
>> bob prohaska <fbsd_at_www.zefox.net> wrote:
>>> 
>>>> On Mon, Jun 14, 2021 at 09:52:22PM +0200, Michael Gmelin wrote:  
>>>>> 
>>>>> 
>>>>>> On 14. Jun 2021, at 20:30, bob prohaska
>>>>>> <fbsd_at_www.zefox.net> wrote:
>>>>>> 
>>>>>> ???On Mon, Jun 14, 2021 at 06:46:52PM +0200, Michael Gmelin
>>>>>> wrote:    
>>>>>>> On Mon, 14 Jun 2021 09:28:39 -0700
>>>>>>> What do you mean by "restarting"?
>>>>>>> How do you invoke poudriere exactly?    
>>>>>> As root,
>>>>>> poudriere bulk -J 2 -j main x11-wm/lxqt www/chromium >
>>>>>> testbuild.log    
>>>>> 
>>>>> Doesn???t ninja handle parallel builds on its own anyway? Does
>>>>> it work okay if you comment out ALLOW_*_JOBS in poudriere.conf?
>>>>> 
>>>> 
>>>> The line ALLOW_MAKE_JOBS is already commented out in 
>>>> /usr/local/etc/poudriere.conf  but it's active in
>>>> /etc/make.conf

Note the ALLOW_MAKE_JOBS instead of: ALLOW_MAKE_JOBS_PACKAGES .
(But I was still wrong for ALLOW_MAKE_JOBS.)

>>>> I remain a bit confused about how poudriere and make coordinate
>>>> their parallel job spawning activity. In the latest case the -J 2
>>>> on the poudriere command line put two packages under
>>>> construction, but the ALLOW_MAKE_JOBS line in /etc/make.conf
>>>> didn't result in parallel threads building LLVM10. Clearly I
>>>> don't understand the relationship between builders, jobs, threads
>>>> and (much!) more.   
>>> 
>>> /etc/make.conf from the host system is not used by poudriere, as it
>>> aims to provide clean-room builds and therefore doesn't use
>>> configurations that also serve a purpose on the host system.
>>> 
>>> All configurations come from /usr/local/etc/poudriere.conf and
>>> various files in /usr/local/etc/poudriere.d/.  
>> 
>> I'll explicitly note that one of the files is:
>> 
>> /usr/local/etc/poudriere.d/make.conf
>> 
>> It is the most global make.conf in poudriere's activities.
>> For poudriere, someone writing about using make.conf likely
>> is implicitly referencing the one at that path (unless they
>> report something that indicates otherwise).
>> 
>> Bob: Sorry I did not notice the /etc/make.conf reference in
>> your earlier material. I should have commented on the
>> distinction.
>> 
>>> See also the
>>> CUSTOMISATION section in poudriere(8) (reading that man pages is
>>> useful in general).
>>> 
>>> So if you want to use this feature, you need to enable it in
>>> poudriere.conf (but I would still recommend using
>>> ALLOW_MAKE_JOBS_PACKAGES instead and also only if you don't use
>>> multiple builders, given the limited resources of your machine).  
>> 
>> /usr/local/etc/poudriere.d/make.conf use should work fine
>> as well.
> 
> Nope, ALLOW_MAKE_JOBS_PACKAGES is a poudriere setting,

Bob referenced: ALLOW_MAKE_JOBS instead of ALLOW_MAKE_JOBS_PACKAGES.

But I was still wrong in that the logic for ALLOW_MAKE_JOBS is:

		[ -n "${ALLOW_MAKE_JOBS}" ] || \
		    echo "DISABLE_MAKE_JOBS=poudriere" \
		    >> ${MASTERMNT}/etc/make.conf

So the make.conf structure uses DISABLE_MAKE_JOBS (or not)
instead of ALLOW_MAKE_JOBS  notation.

> used
> here inside poudriere:
> https://github.com/freebsd/poudriere/blob/master/src/share/poudriere/common.sh#L4421
> 
> poudriere doesn't read make.conf configurations as its own
> configuration, so setting ALLOW_MAKE_JOBS_PACKAGES in there does
> nothing (it only appends make.conf settings to the
> generated /etc/make.conf inside the jail, see "append_make()" in
> common.sh).

Yep.

> Also note that ALLOW_MAKE_JOBS_PACKAGES has no effect for ports
> tested using "poudriere testport" (but it *does* apply to building
> the dependencies required for such ports). So, assuming
> ALLOW_MAKE_JOBS_PACKAGE="someport", "poudriere testport someport" won't
> make use of MAKE_JOBS, while "poudriere testport
> otherport-that-depends-on-someport" will do so for building someport.

Not something I'd looked into. Interesting.

> Something that might not be obvious when reading poudriere's config:
> /usr/local/etc/poudriere.d/poudriere.conf is only read if
> /usr/local/etc/poudriere.conf doesn't exist. Other files like
> /usr/local/etc/poudriere.d/130amd64-poudriere.conf *will* be read
> though (after reading the global config).

Too bad that is not documented by "man poudriere":

QUOTE
. . .
     POUDRIERE_ETC/poudriere.conf                  See self-documented
                                                   /usr/local/etc/poudriere.conf.sample
                                                   for example.
     POUDRIERE_ETC/poudriere.d/poudriere.conf      The configuration can be
                                                   stored in the poudriere.d
                                                   directory as well.
. . .
   Create optional poudriere.conf
     You can also specify an optional poudriere.conf that is pulled in
     depending on the build.  Any of the following are allowed and will all be
     used in the order shown:

          /usr/local/etc/poudriere.d/poudriere.conf
. . . (all are poudriere.d/ paths) . . .
END QUOTE

I would never get the actual behavior from the wording present.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)