Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes?

On May 1, 2023, at 12:57, Mark Millard <marklmi@yahoo.com> wrote:

> On Apr 30, 2023, at 18:57, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On Apr 30, 2023, at 17:44, Mateusz Guzik <mjguzik@gmail.com> wrote:
>> 
>>> can you redo zfs test with:
>>> sysctl vfs.zfs.per_txg_dirty_frees_percent=5
>> 
>> Sure.
>> 
>> Result summary: Seems to have avoided the sustained periods
>> of low load average activity. Much better for the context.
>> 
>> 
>> Context: Original ZFS USB3 media. World or Kernel in use
>> had been built (non-debug style) using:
>> 
>> -mcpu=cortex-a78C+flagm+nofp16fml
>> 
>> 
>> Steps for this test . . .
>> 
>> # poudriere pkgclean -A -jmain-CA78C
>> . . .
>> 
>> # sysctl vfs.zfs.per_txg_dirty_frees_percent=5
>> vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5
>> 
>> # grep USE_TMPFS= /usr/local/etc/poudriere.conf
>> # EXAMPLE: USE_TMPFS="wrkdir data"
>> USE_TMPFS="data"
>> #USE_TMPFS=all
>> 
>> # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt
>> . . .
>> 
>> At 15 minutes into the build:
>> 46 ports in 1st 15 minutes. Load average stayed reasonable
>> for the configuration.
>> 
>> At 30 minutes into the build:
>> 102 ports in 1st 30 minutes. Load average still reasonable
>> for the configuration.
>> 
>> Looks good compared to before.
>> 
>> 
>> I've no clue what optimal would be for the context, but
>> 
>> vfs.zfs.per_txg_dirty_frees_percent=5
>> 
>> is vastly better for the context than the default 30 was.
>> 
>> Thanks.
>> 
>> 
>> I'm going to stop the test and do the conversion to the
>> U2 960GB Optane media in the USB3 adaptor and then
>> compare USE_TMPFS=data vs. USE_TMPFS=all --but using your
>> vfs.zfs.per_txg_dirty_frees_percent=5 assignment.
>> 
> 
> Took a while to actually get around to stopping the test.
> It got 186 of the ports built in the 1st hour. (A from
> scratch build, starting with building pkg.)
> 
> I finally have started the U2 960GB Optane based tests,
> currently USE_TMPFS=data . The initial activity looks
> like it might build about as many ports as the earlier
> USE_TMPFS=all test (for different media,
> vfs.zfs.per_txg_dirty_frees_percent being 30).
> 
> . . . waiting . . .
> 
> It got 222 of the ports built in the 1st hour, again
> starting with pkg. That compares to 262 for the earlier
> USE_TMPFS=all test.
> 
> None of these ports form the first hour are large, long
> running port builds, none using large scale amounts of
> storage space for its builder. (As I build things, rust
> for example, uses 17GiBytes+ of file system space, more
> than half of the size of the RAM in the Windows Dev Kit
> 2023 for USE_TEMPFS=all just for file system content.)
> 
> Now for a USE_TMPFS=all build test with
> vfs.zfs.per_txg_dirty_frees_percent being 5 . I may try
> letting that run to completion. (The configuration has
> 118 GiBytes of swap for paging activity.)


USE_TMPFS=all with the U2 960GB Optane USB3 media:

It built 270 ports in the first hour.

I'm going to let it run to completion (assuming that it
does in under, say, 16 hours total for the overall 480
ports). I'll see what it is like when llvm15, llvm16,
rust, and possibly gcc13 or such will likely build in
overlapping time frames.

(I do not build LTO style and, for gcc* do do not even
use a bootstrap style. My usage of compiler toolchains
between toolchain builds is minor compared to the
toolchain builds themselves for the most part and my
time preferences for builds are fairly strongly biased
to less wait time.)


===
Mark Millard
marklmi at yahoo.com