Re: poudriere bulk -a fails on UFS: "Too many links" under logs/bulk/latest-per-pkg/ and then "Failed: starting"

On Nov 4, 2023, at 16:45, Tatsuki Makino <tatsuki_makino@hotmail.com> wrote:

> Mark Millard wrote on 2023/11/05 07:54:
>> (I've not checked if /usr/local/poudriere/data/logs/bulk/latest-per-pkg/
>> is the only issue. It is just the first context to stop the build.)
> 
> Sorry, I didn't read it very carefully and confused the logs directory with the packages directory :)
> I think we can delete the old logs.
> But just do the following.
> 
> poudriere logclean 30 # If the last 30 days are to be kept.

Even if there are 0 logs before the bulk -a starts,
the bulk -a will fail to finish on a UFS file system.
This happens because there are over 32767 packages
these days that will get logs for an amd64 context
and poudriere chooses to organize its directory tree
such that the UFS limitation will be hit. (Plus
UFS has the limitation in the first place.)

I'll also note that my bulk -a experiments are not
important and would not justify changing anything.
But they have lead to the issue being discovered and
reported so that the limitation is better known now.

> There is a -N count option in poudriere-logclean, but I think it would cause less problems if it were not used.
> Because awk is behaving strangely.
> It occurs that logs with old dates continue to be kept.
> It could also leave directories for packages whose names no longer exist.
> I don't know if awk behavior has been fixed in newer versions of FreeBSD.
> It is strange, at least in 12.x.
> 

None of this matters for the issue with what goes directly
in

/usr/local/poudriere/data/logs/bulk/latest-per-pkg/

The count will still attempt to become larger than 32767
in di_nlink in ufs2_dinode for that directory. UFS will
refuse such attempts.

I'll note that having a few thousand or more packages that
never have an attempted build works fine on UFS. (Over
time a larger subset must not be built in order to stay
under 32768.) Also, the official FreeBSD package build
servers use ZFS for the file system and, so, do not have
the problem.

I do not expect UFS to ever be changed to use more than 16
bits for the di_nlink purpose. I'd be surprised if poudriere
changed its directory tree structure to avoid the issue on
UFS, given that official builds work as-is.

I expect that folks that want to do bulk -a runs will just
have to set up a (probably) ZFS filesystem. This much should
be documented if it stays as a requirement for bulk -a use
to be successful. (And, my ZFS bulk -a experiment worked
just fine, given that I'd configured well for my style of
build.)


===
Mark Millard
marklmi at yahoo.com