Re: Too many pythons in poudriere
Date: Thu, 08 Jul 2021 02:11:59 UTC
On 2021-Jul-7, at 19:04, Mark Millard <marklmi at yahoo.com> wrote:
> From: bob prohaska <fbsd_at_www.zefox.net> wrote on
> Date: Wed, 7 Jul 2021 14:53:20 -0700 :
>
>> In trying to compile www/chromium under poudriere on a Pi3 there
>> comes a point when five python2.7 sessions totaling more than 2 GB
>> in size are running at once.
I should have orignially noted right here that such is
because of the use of: ALLOW_MAKE_JOBS=yes
You need to avoid that for the www/chromium build.
>> Obviously, swap is swamped and the Pi3
>> is running at a crawl. It hasn't givem up, however.
>>
>> Poudriere was started with -J 2 and make is limited to 2 jobs.
>> Is there an analogous limit to how many pythons are loosed at
>> once? It looks like there's only one builder, so it isn't
>> obvious that -J 1 would help; I'll try it if this job stops
>> prematurely.
>
> It will not help. There were no competing build jobs.
>
>> Progress, such as it is, can be seen at
>>
>> http://www.zefox.org/~bob/poudriere/data/logs/bulk/main-default/2021-07-05_14h06m26s/build.html
>
> By the time I looked it had run out of swap space:
>
> Swapinfo 100.00%
>
> and had stopped for build/timeout after 50:33:26 .
>
> For reference, including "swap_pager: out of swap space"
> and "swp_pager_getswapspace(1): failed":
>
> QUOTE
>
> Wed Jul 7 15:20:34 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1831744 11456 99%
> /dev/mmcsd0s2b 1843200 1832328 10872 99%
> Total 3686400 3664072 22328 99%
> . . .
> Wed Jul 7 15:20:46 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1838356 4844 100%
> /dev/mmcsd0s2b 1843200 1838928 4272 100%
> Total 3686400 3677284 9116 100%
> . . .
> Wed Jul 7 15:20:56 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1841260 1940 100%
> /dev/mmcsd0s2b 1843200 1841836 1364 100%
> Total 3686400 3683096 3304 100%
> . . .
> Wed Jul 7 15:21:08 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843000 200 100%
> /dev/mmcsd0s2b 1843200 1843124 76 100%
> Total 3686400 3686124 276 100%
> . . .
> Jul 7 15:20:58 www kernel: swap_pager: out of swap space
> . . .
> Wed Jul 7 15:21:20 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843128 72 100%
> /dev/mmcsd0s2b 1843200 1843140 60 100%
> Total 3686400 3686268 132 100%
> . . .
> Jul 7 15:20:58 www kernel: swap_pager: out of swap space
> . . .
> Wed Jul 7 15:21:30 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843160 40 100%
> /dev/mmcsd0s2b 1843200 1843116 84 100%
> Total 3686400 3686276 124 100%
> . . .
> Jul 7 15:20:58 www kernel: swap_pager: out of swap space
> . . .
> Wed Jul 7 15:21:45 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:20:58 www kernel: swap_pager: out of swap space
> Jul 7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
> . . .
> Wed Jul 7 15:22:05 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:20:58 www kernel: swap_pager: out of swap space
> Jul 7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
> . . .
> Wed Jul 7 15:48:46 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
> Jul 7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
> . . .
> Wed Jul 7 15:57:01 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
> Jul 7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
> . . .
> Wed Jul 7 15:57:21 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:21:33 www kernel: swp_pager_getswapspace(3): failed
> Jul 7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
> . . .
> Wed Jul 7 16:31:52 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 1843192 8 100%
> /dev/mmcsd0s2b 1843200 1843192 8 100%
> Total 3686400 3686384 16 100%
> Jul 7 15:48:44 www kernel: swp_pager_getswapspace(1): failed
> Jul 7 16:13:16 www kernel: swp_pager_getswapspace(3): failed
> . . .
> Wed Jul 7 17:47:11 PDT 2021
> Device 1K-blocks Used Avail Capacity
> /dev/da0s2b 1843200 32696 1810504 2%
> /dev/mmcsd0s2b 1843200 33572 1809628 2%
> Total 3686400 66268 3620132 2%
>
> END QUOTE
>
> It looks like for the configuration as it is, the
> bulk build needs to build such that the load
> average stays near 1 or less, avoiding near 2 or
> more: no use of ALLOW_MAKE_JOBS=yes during the bulk
> build is one way to do that.
>
> In http://www.zefox.org/~bob/poudriere.conf (modified
> for illustration):
>
> # By default MAKE_JOBS is disabled to allow only one process per cpu
> # Use the following to allow it anyway
> #ALLOW_MAKE_JOBS=yes
> # List of packages that will always be allowed to use MAKE_JOBS
> # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports
> # which holdup the rest of the queue to build more quickly.
> #ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*"
>
> I'll also note that:
>
> http://www.zefox.org/~bob/poudriere.d/make.conf
>
> should not ever have the "ALLOW_MAKE_JOBS=yes" that
> is does: it is the wrong symbol for that kind of
> context. poudriere converts ALLOW_MAKE_JOBS to something
> else used in make.
>
> QUOTE (not modified for illustration)
> ALLOW_MAKE_JOBS=yes
> MAKE_JOBS_NUMBER=2
> #.if ${.CURDIR:M*www/chromium}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> #.if ${.CURDIR:M*databases/sqlite3}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> #.if ${.CURDIR:M*www/firefox}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> END QUOTE
>
>
>>
>> The last time www/chromium was compiled (using make) on this machine
>> I don't remember seeing such a python jam. If it happened at all the
>> Pi3 worked through it faster than presently.
>
> Which was the old make: -j1 vs. -j2 vs. -j3 vs. -j4 ?
>
> The ALLOW_MAKE_JOBS=yes use is like -j4 in your 4 core
> context. Lack of ALLOW_MAKE_JOBS=yes is like -j1 .
>
> The -jN is for the number of make processes allowed to be
> active per builder, including when there is only one builder.
>
> The ALLOW_MAKE_JOBS=yes meant that there was likely a massive
> amount of paging activity that was taking up much of the
> time. That would still have been true with a much lager
> swap space: it is a type of context where Lack of
> ALLOW_MAKE_JOBS=yes may well take notably less time to
> finish.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)