Re: list of valid ABI combinations

On Tue, Jan 2, 2024, at 10:46 AM, Dan Langille wrote:
> On Tue, Jan 2, 2024, at 12:45 AM, Philip Paeps wrote:
>> On 2024-01-02 12:38:47 (+0800), Mark Millard wrote:
>>> [FYI: My internet access is limited to a cellphone for some number of 
>>> days.]
>>>
>>>> On Jan 1, 2024, at 19:26, Philip Paeps <philip@freebsd.org> wrote:
>>>>
>>>> On 2023-12-07 23:03:33 (+0800), Mark Millard wrote:
>>>>> Dan Langille <dan_at_langille.org> wrote:
>>>>>> My goal is to automatically maintain this table, from which 
>>>>>> FreshPorts fetches package information:
>>>>>>
>>>>>> freshports.dev=# select * from abi order by name;
>>>>>> id | name | active
>>>>>> ----+----------------------+--------
>>>>>> 9 | FreeBSD:12:aarch64 | t
>>>>>> [...]
>>>>>> 53 | FreeBSD:15:riscv64 | t
>>>>>> (31 rows)
>>>>>
>>>>> The name list in the middle (/bin/sh context):
>>>>>
>>>>> # fetch https://pkg.freebsd.org/index.html
>>>>> index.html                                            3606  B  103 
>>>>> MBps    00s
>>>>>
>>>>> # grep FreeBSD: index.html | sed -e 's@.*\(FreeBSD:[^ <]*\).*@\1@' | 
>>>>> sort
>>>>> FreeBSD:12:aarch64
>>>>> [...]
>>>>> FreeBSD:15:powerpc64le
>>>>>
>>>>> There might be issues at times with 
>>>>> https://pkg.freebsd.org/index.html not
>>>>> being in sync. As far as I can tell, that page is manually 
>>>>> maintained.
>>>>
>>>> clusteradm updates that page to match what we're actually 
>>>> distributing.  If it doesn't reflect reality, that's a bug, and we'd 
>>>> like to know please. :-)
>>>
>>> Does that include the “only ??? Is updated” sorts of notes?
>>
>> Yes.  Please let clusteradm know if the notes don't appear to reflect 
>> reality.
>>
>> clusteradm maintains the pkgsync infrastructure, which includes the 
>> index page on the mirrors.  pkgmgr is responsible for scheduling the 
>> builds.  (And ports committers are responsible for making sure packages 
>> are even buildable).
>>
>>> For example, FreeBSD:13:armv6 Latest dates back to 2021-Jan-25 but 
>>> there is no note about quarterly only. Various FreeBSD:15:* do not 
>>> have notes about Latest only, despite having no quarterly at all. Some 
>>> other cases have notable but shorter differences in the Latest vs. 
>>> Quarterly time frames, suggesting only one is being updated but no 
>>> fixed difference is definitive for that status.
>>
>> The pkgsync infrastructure doesn't really care how often packages are 
>> updated.  If memory serves, we first added a "quarterly only" note when 
>> the thunderxX generation of aarch64 builders broke with distressing 
>> regularity.  The ampereX builders are much more reliable.
>>
>> I think the current situation for armv6 is "updated irregularly".  
>> PowerPC is in the same boat as aarch64 in the ThunderX era: when the 
>> hardware works, packages are updated.
>>
>>> Dan’s activities with the page do not use such notes as far as I 
>>> know.
>>
>> Would it help if we added an easy to parse text file listing the ABIs 
>> we're currently distributing?  That won't tell you anything about the 
>> build schedule, but it'll at least tell you what's live, without having 
>> to parse HTML and hope we don't break it.

Could that extra file be useful to clusteradm, and is the 'single source of
truth', then yes, that would be useful to third parties as well. 

Changes to FreshPorts ABI table just tells it what ABI to attempt to import.
All that is lost/found is the package information. As such, it does not
otherwise affect the service.

Right now, I have a script which works.  It will be deployed on dev.freshports.org
later today. It will refresh the ABI on a daily basis.

> I think this threads provides sufficient information for 'automatically know
> the list of valid ABI' to proceed:
>
> https://github.com/FreshPorts/freshports/issues/508

This is all implemented by short shell scripts, the longest of which is 32 lines.
See the URL above for details.

-- 
  Dan Langille
  dan@langille.org