Re: Too many pythons in poudriere

From: Mark Millard via freebsd-ports <freebsd-ports_at_freebsd.org>
Date: Wed, 7 Jul 2021 19:27:54 -0700
On 2021-Jul-7, at 19:11, Mark Millard <marklmi at yahoo.com> wrote:

> On 2021-Jul-7, at 19:04, Mark Millard <marklmi at yahoo.com> wrote:
> 
>> From: bob prohaska <fbsd_at_www.zefox.net> wrote on
>> Date: Wed, 7 Jul 2021 14:53:20 -0700 :
>> 
>>> In trying to compile www/chromium under poudriere on a Pi3 there
>>> comes a point when five python2.7 sessions totaling more than 2 GB
>>> in size are running at once.
> 
> I should have orignially noted right here that such is
> because of the use of: ALLOW_MAKE_JOBS=yes
> 
> You need to avoid that for the www/chromium build.
> 
>>> Obviously, swap is swamped and the Pi3
>>> is running at a crawl. It hasn't givem up, however.
>>> 
>>> Poudriere was started with -J 2 and make is limited to 2 jobs. 
>>> Is there an analogous limit to how many pythons are loosed at
>>> once? It looks like there's only one builder, so it isn't
>>> obvious that -J 1 would help; I'll try it if this job stops
>>> prematurely.
>> 
>> It will not help. There were no competing build jobs.
>> 
>>> Progress, such as it is, can be seen at
>>> 
>>> http://www.zefox.org/~bob/poudriere/data/logs/bulk/main-default/2021-07-05_14h06m26s/build.html
>> 
>> By the time I looked it had run out of swap space:
>> 
>> Swapinfo 100.00%
>> 
>> and had stopped for build/timeout after 50:33:26 .
>> 
>> For reference, including "swap_pager: out of swap space"
>> and "swp_pager_getswapspace(1): failed":
>> 
>> QUOTE
>> 
>> Wed Jul  7 15:20:34 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1831744    11456    99%
>> /dev/mmcsd0s2b    1843200  1832328    10872    99%
>> Total             3686400  3664072    22328    99%
>> . . .
>> Wed Jul  7 15:20:46 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1838356     4844   100%
>> /dev/mmcsd0s2b    1843200  1838928     4272   100%
>> Total             3686400  3677284     9116   100%
>> . . .
>> Wed Jul  7 15:20:56 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1841260     1940   100%
>> /dev/mmcsd0s2b    1843200  1841836     1364   100%
>> Total             3686400  3683096     3304   100%
>> . . .
>> Wed Jul  7 15:21:08 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843000      200   100%
>> /dev/mmcsd0s2b    1843200  1843124       76   100%
>> Total             3686400  3686124      276   100%
>> . . .
>> Jul  7 15:20:58 www kernel: swap_pager: out of swap space
>> . . .
>> Wed Jul  7 15:21:20 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843128       72   100%
>> /dev/mmcsd0s2b    1843200  1843140       60   100%
>> Total             3686400  3686268      132   100%
>> . . .
>> Jul  7 15:20:58 www kernel: swap_pager: out of swap space
>> . . .
>> Wed Jul  7 15:21:30 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843160       40   100%
>> /dev/mmcsd0s2b    1843200  1843116       84   100%
>> Total             3686400  3686276      124   100%
>> . . .
>> Jul  7 15:20:58 www kernel: swap_pager: out of swap space
>> . . .
>> Wed Jul  7 15:21:45 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:20:58 www kernel: swap_pager: out of swap space
>> Jul  7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
>> . . .
>> Wed Jul  7 15:22:05 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:20:58 www kernel: swap_pager: out of swap space
>> Jul  7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
>> . . .
>> Wed Jul  7 15:48:46 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
>> Jul  7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
>> . . .
>> Wed Jul  7 15:57:01 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
>> Jul  7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
>> . . .
>> Wed Jul  7 15:57:21 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
>> Jul  7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
>> . . .
>> Wed Jul  7 16:31:52 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200  1843192        8   100%
>> /dev/mmcsd0s2b    1843200  1843192        8   100%
>> Total             3686400  3686384       16   100%
>> Jul  7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
>> Jul  7 16:13:16 www kernel: swp_pager_getswapspace(3): failed
>> . . .
>> Wed Jul  7 17:47:11 PDT 2021
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/da0s2b       1843200    32696  1810504     2%
>> /dev/mmcsd0s2b    1843200    33572  1809628     2%
>> Total             3686400    66268  3620132     2%
>> 
>> END QUOTE
>> 
>> It looks like for the configuration as it is, the
>> bulk build needs to build such that the load
>> average stays near 1 or less, avoiding near 2 or
>> more: no use of ALLOW_MAKE_JOBS=yes during the bulk
>> build is one way to do that.
>> 
>> In http://www.zefox.org/~bob/poudriere.conf (modified
>> for illustration):
>> 
>> # By default MAKE_JOBS is disabled to allow only one process per cpu
>> # Use the following to allow it anyway
>> #ALLOW_MAKE_JOBS=yes
>> # List of packages that will always be allowed to use MAKE_JOBS
>> # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports
>> # which holdup the rest of the queue to build more quickly.
>> #ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*"
>> 
>> I'll also note that:
>> 
>> http://www.zefox.org/~bob/poudriere.d/make.conf
>> 
>> should not ever have the "ALLOW_MAKE_JOBS=yes" that
>> is does: it is the wrong symbol for that kind of
>> context. poudriere converts ALLOW_MAKE_JOBS to something
>> else used in make.
>> 
>> QUOTE (not modified for illustration)
>> ALLOW_MAKE_JOBS=yes
>> MAKE_JOBS_NUMBER=2
>> #.if ${.CURDIR:M*www/chromium}
>> #MAKE_JOBS_NUMBER_LIMIT=2
>> #.endif
>> #.if ${.CURDIR:M*databases/sqlite3}
>> #MAKE_JOBS_NUMBER_LIMIT=2
>> #.endif
>> #.if ${.CURDIR:M*www/firefox}
>> #MAKE_JOBS_NUMBER_LIMIT=2
>> #.endif
>> END QUOTE
> 

Looks like you could use:

MAKE_JOBS_NUMBER=2
.if ${.CURDIR:M*www/chromium}
MAKE_JOBS_UNSAFE=
.endif

to have www/chromium build as, essentially, -j1 .
One thing that MAKE_JOBS_UNSAFE= leads to is:
MAKE_JOBS_NUMBER=1

(Or you could put the MAKE_JOBS_UNSAFE= in the
www/chromium makefile .)

>> 
>>> 
>>> The last time www/chromium was compiled (using make) on this machine 
>>> I don't remember seeing such a python jam. If it happened at all the 
>>> Pi3 worked through it faster than presently. 
>> 
>> Which was the old make: -j1 vs. -j2 vs. -j3 vs. -j4 ?
>> 
>> The ALLOW_MAKE_JOBS=yes use is like -j4 in your 4 core
>> context. Lack of ALLOW_MAKE_JOBS=yes is like -j1 .
>> 
>> The -jN is for the number of make processes allowed to be
>> active per builder, including when there is only one builder.
>> 
>> The ALLOW_MAKE_JOBS=yes meant that there was likely a massive
>> amount of paging activity that was taking up much of the
>> time. That would still have been true with a much lager
>> swap space: it is a type of context where Lack of
>> ALLOW_MAKE_JOBS=yes may well take notably less time to
>> finish.
> 



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Received on Thu Jul 08 2021 - 02:27:54 UTC

Original text of this message