Re: list of valid ABI combinations

On Tue, Jan 2, 2024, at 12:45 AM, Philip Paeps wrote:
> On 2024-01-02 12:38:47 (+0800), Mark Millard wrote:
>> [FYI: My internet access is limited to a cellphone for some number of 
>> days.]
>>
>>> On Jan 1, 2024, at 19:26, Philip Paeps <philip@freebsd.org> wrote:
>>>
>>> On 2023-12-07 23:03:33 (+0800), Mark Millard wrote:
>>>> Dan Langille <dan_at_langille.org> wrote:
>>>>> My goal is to automatically maintain this table, from which 
>>>>> FreshPorts fetches package information:
>>>>>
>>>>> freshports.dev=# select * from abi order by name;
>>>>> id | name | active
>>>>> ----+----------------------+--------
>>>>> 9 | FreeBSD:12:aarch64 | t
>>>>> [...]
>>>>> 53 | FreeBSD:15:riscv64 | t
>>>>> (31 rows)
>>>>
>>>> The name list in the middle (/bin/sh context):
>>>>
>>>> # fetch https://pkg.freebsd.org/index.html
>>>> index.html                                            3606  B  103 
>>>> MBps    00s
>>>>
>>>> # grep FreeBSD: index.html | sed -e 's@.*\(FreeBSD:[^ <]*\).*@\1@' | 
>>>> sort
>>>> FreeBSD:12:aarch64
>>>> [...]
>>>> FreeBSD:15:powerpc64le
>>>>
>>>> There might be issues at times with 
>>>> https://pkg.freebsd.org/index.html not
>>>> being in sync. As far as I can tell, that page is manually 
>>>> maintained.
>>>
>>> clusteradm updates that page to match what we're actually 
>>> distributing.  If it doesn't reflect reality, that's a bug, and we'd 
>>> like to know please. :-)
>>
>> Does that include the “only ??? Is updated” sorts of notes?
>
> Yes.  Please let clusteradm know if the notes don't appear to reflect 
> reality.
>
> clusteradm maintains the pkgsync infrastructure, which includes the 
> index page on the mirrors.  pkgmgr is responsible for scheduling the 
> builds.  (And ports committers are responsible for making sure packages 
> are even buildable).
>
>> For example, FreeBSD:13:armv6 Latest dates back to 2021-Jan-25 but 
>> there is no note about quarterly only. Various FreeBSD:15:* do not 
>> have notes about Latest only, despite having no quarterly at all. Some 
>> other cases have notable but shorter differences in the Latest vs. 
>> Quarterly time frames, suggesting only one is being updated but no 
>> fixed difference is definitive for that status.
>
> The pkgsync infrastructure doesn't really care how often packages are 
> updated.  If memory serves, we first added a "quarterly only" note when 
> the thunderxX generation of aarch64 builders broke with distressing 
> regularity.  The ampereX builders are much more reliable.
>
> I think the current situation for armv6 is "updated irregularly".  
> PowerPC is in the same boat as aarch64 in the ThunderX era: when the 
> hardware works, packages are updated.
>
>> Dan’s activities with the page do not use such notes as far as I 
>> know.
>
> Would it help if we added an easy to parse text file listing the ABIs 
> we're currently distributing?  That won't tell you anything about the 
> build schedule, but it'll at least tell you what's live, without having 
> to parse HTML and hope we don't break it.

I think this threads provides sufficient information for 'automatically know
the list of valid ABI' to proceed:

https://github.com/FreshPorts/freshports/issues/508

Also, the (much appreciated-here) script provided by Mark on 2023-12-07 still
runs true after the removal of 12.4 from the build page.

[15:42 empty dan ~/tmp] % fetch https://pkg.freebsd.org/index.html
index.html                                            3068  B   37 MBps    00s
[15:42 empty dan ~/tmp] % grep FreeBSD: index.html | sed -e 's@.*\(FreeBSD:[^ <]*\).*@\1@' | sort
FreeBSD:13:aarch64
FreeBSD:13:amd64
FreeBSD:13:armv6
FreeBSD:13:armv7
FreeBSD:13:i386
FreeBSD:13:powerpc
FreeBSD:13:powerpc64
FreeBSD:13:powerpc64le
FreeBSD:14:aarch64
FreeBSD:14:amd64
FreeBSD:14:armv6
FreeBSD:14:armv7
FreeBSD:14:i386
FreeBSD:14:powerpc
FreeBSD:14:powerpc64
FreeBSD:14:powerpc64le
FreeBSD:15:aarch64
FreeBSD:15:amd64
FreeBSD:15:armv7
FreeBSD:15:powerpc
FreeBSD:15:powerpc64
FreeBSD:15:powerpc64le
[15:42 empty dan ~/tmp] % 

FYI, the goal is for FreshPorts to fetch and display package information only for the
ABI listed by this script.

Thank you
-- 
  Dan Langille
  dan@langille.org