public_distfile replication

[Peter Wemm]

On Thu, Jul 11, 2013 at 2:57 AM, Simon L. B. Nielsen <simon at qxnitro.org> wrote:
> On 11 July 2013 08:31, Jason Helfman <jgh at freebsd.org> wrote:
>> Hey Gang...
>>
>> From looking at some of my public_distfiles, it seems that
>> ftp.{cn,se,tw}.freebsd.org don't seem to be receiving public_distfiles per
>> replication. I've also seen that ru has issues, as well.
>
> Are they on ftp.FreeBSD.org ? If they are, it gets to ftp-master and
> after that it's out of our hands.

Most ftp mirrors are broken because they try to pick out parts of the
rsync archive and expand symlinks.  I've watched some ftp mirrors
rsync 3x to 5x as much data because they fetch a duplicate copy of the
data via all the "compatability" symlinks or because they're not
preserving hard links.

Some of this silliness came about because of rsync all the cvsup and
cvsup checkouts had high overheads but these days most should be able
to mirror exactly as is, with -H to preserve hard links.  A lot of the
time consuming things for rsync were tidied up.

Regardless, this is not a clusteradm thing.  3rd party mirrors might
be reachable via hubs at freebsd.org.  All we can do via clusteradm is
revoke 3rd party zone delegations to de-list broken ftp mirrors.  Most
of the subdomains are not under project control in any way.

-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
UTF-8: So you can \342\200\231 .. for when a ' just won't do
<brueffer> ZFS must be the bacon of file systems. "everything's better with ZFS"