Re: git: 0882d238e9b4 - main - Mk/bsd.sites.mk: Update GENTOO entries

On 2023-03-22 06:50, Alexey Dokuchaev wrote:
> On Mon, Mar 20, 2023 at 09:02:18AM +0100, Daniel Engberg wrote:
>> Hi,
> 
> Hi, and thanks for elaborate reply Daniel.  It would have made a better
> commit log if parts of this text were in it. :)

That's fair

> 
>> At least the first 2 were dead and/or didn't connect from different
>> locations at time I decided to redo the list based on
>> https://www.gentoo.org/downloads/mirrors/. I tried to determine and 
>> use
>> sites that have as good connectivity and speed for all regions, both
>> LeaseWeb and Rackspace services in multiple and from my test locations
>> (different regions) they performed overall well.
> 
> Fair enough.  Mirrors do come and go and must be curated, but mixing
> dead and "bad" ones together makes it hard to assess these changes in
> retrospect.  Ideally we should be able to track any particular mirror
> via commit logs, esp. because sometimes they are not totally gone, but
> moved from rotation temporarily, change ISP, colocation, etc.

Ideally I agree however from what I've seen no other project does that 
outside of their own mirror pool and I think we can trust upstream in 
such cases. If we see degraded performance for a long period of time (as 
we all know hardware/Internet breaks sometimes) we can just adjust our 
"local" list and move on.

> 
>> In theory there isn't anything wrong with unofficial ones however such
>> hosts may not be as actively monitored for health and consistency
>> (being in sync) and from what I've seen and looked at feedback in
>> general it's not desirable list such hosts unless needed.
> 
> This part I don't understand.  If a mirror lacks some distfile, the
> next one would be used.  Both Yandex and 163.com might not be official
> but are very popular in their regions of the world (and often broader).

The primary goal should be trying to serve as many as possible 
irregardless of region with decent performance, in this case transfer 
speed since that's what it primarily affects apart from availability. 
The ports framework isn't aware of geolocation and that would serve a 
very niche purpose in practice however and mirror health needs to be 
taken into account which is why I think we should be reluctant add such 
hosts. Inevitable some countries/hosts have worse connectivity and/or 
transfer/peering than others, we can't fix that in ports and we need to 
assume that TCP/IP functionality works pretty much as we assume that the 
hardware you're running FreeBSD on works as intended.

> 
>> The rest of the hosts were selected based on of peering, speed, and
>> regional location.
> 
> From where?  First-world internet may give you quite skewed results.

Your rural ISP might only have one upstream link etc but that's also not 
your everyday user.  I used Telia which uses Arelion (former Telia 
International Carrier, https://www.arelion.com/our-network) in Europe 
and Oracle Cloud Infrastructure in Canada (ASN31898) for evalution while 
trying to  have users in Asia in mind.

> 
>> Connectivity issues and not overall transfer speed for "single/few"
>> hosts is something that's out of scope for the ports tree and needs to
>> be addressed by the user(s). To answer your question regarding
>> mirrors.163.com the reason is simply because there are better options
>> overall and .cn mirrors are in the majority of times very slow to 
>> hosts
>> outside of China (Rackspace lists a node in HK fwiw).
> 
> You could've very well simply moved it downwards if you don't like it,
> but then again, there are FreeBSD users in China, what about them?

It doesn't matter, if 8+ hosts fails adding X more wont help and I'm 
don't understand why you're so hung up on this.
> 
>> Most of tree overall have single hosts listed and afterwards relies on
>> FreeBSD's infrastructure so I presume that you have a lot more issues
>> with the tree overall than just a handful of ports.
> 
> I have no issues with the ports tree, I just have shitty internet, and
> that's why I like to have a plethora of mirrors.

That's a somewhat biased personal opinion and unfortunate circumstances 
regarding your ISP. We still need to assume that a connection works as 
expected and as I mentioned earlier vast majority of ports only have a 
single host (apart from our distcaches) so this doesn't make any 
practical difference it just adds overhead.

> 
>> I think we can safely say that if 8 hosts in different regions, ASNs 
>> etc
>> fails your host has more issues than the amount of mirrors we provide,
>> in fact the vast majority of ports actually provides fewer mirrors 
>> than
>> that including FreeBSD's infrastructure.
> 
> That's beyond my point, which is: more mirrors is better than less.  
> But
> oh well, I guess I could always put some extra MASTER_SITE_GENTOO in my
> /etc/make.conf.

It makes no practical difference if a mirror list is somewhat well 
maintained/curated so I kindly disagree on that one but you can of 
course maintain forks locally if you so choose.

> 
> ./danfe

Best regards,
Daniel