[hubs] cvsup-master changes

[John Marshall]

On Wed, 05 Feb 2014, 10:54 -0800, Peter Wemm wrote:
> In order to tidy up some no-longer-needed complexity with cvsup-master I've
> done a final rebuild of the infrastructure supporting it.  It's now a self
> contained slave of other public data.

Thank you for all of your work on this, and thank you for letting us know.
I _love_ the CNAME target!

> - The source for the www mirror data is now the same rsync pool that is
> used on www.freebsd.org.  The old cvsup data was being gathered from an
> obsolete machine that was only doing a partial build - large chunks were 1+
> years stale.

So now the www mirrors served from www/current replicas are more like
real mirrors.  Lovely!

> - I purged the 1-2 year stale ports and doc trees.  I'll upload tarballs of
> the final versions of each onto ftp shortly.

I hope you don't get too much backlash from this.  All CVSup mirrors and
probably all CVSup ports tree users would have "default delete" in their
supfiles.  So, by deleting the ports tree on cvsup-master, you have
replicated the tree deletion out to the world: their (very stale) ports
trees would have been replaced with your README.ports file.  It looks
like this was your intention but it might come as a bit of a shock to
some.  Did you consider the option of removing the ports-all collection
directory?  Would you consider doing that anyway?  That way the sup
client will see, 'Server message: Unknown collection "ports-all"'
without having its very own copy of the stale CVS ports tree deleted.

In any case, I suggest that some folks might find it helpful to find an
announcement on ports-announce@ and/or ports@ to the effect that the
ports collection has been removed from the CVSup servers.

> After looking at the machine's logs, I'm wondering how much this stuff is
> being used by end users relative to the number of mirrors still in
> operation.  I suspect it is really difficult to tell the difference between
> somebody who's actually using the code from cvsup vs somebody who set up a
> mirror years ago and forgot about it.

Given that 25% of our server's daily synchronization requests are for
the ports-all collection, I would suggest that many of the sync's. fall
into the "set and forgot" category.

> There's a public cvsup machine in the cluster (cvsup14.nyi) that appears to
> be desperately bored lately.  It implements 12 of the cvsupN aliases and
> seems to be doing about 50 client connections per hour.  Is this comparable
> with what other folks are seeing?

We saw a 60% reduction in CVSup traffic early last year when svn exports
to the CVS ports tree ceased.  With the current, release and stable src
branches, ports, doc and gnats no longer being updated via CVSup, I
would guess that the majority of what's left is "set and forgot", apart
from some actual users of the mail-archive/current and www/current
collections - and some of those may be "set and forgot" too.  The bulk
of our remaining CVSup traffic is attributable to about half a dozen
downstream private mirrors (some of which are telco, ISP and government
- I have no knowledge of the degree of usage made downstream of any of
those).

Here are some real numbers from the past 9 months.  The graphs start a
couple of months after the 60% drop-off.  The 10GB/day spikes are now
ancient history.  This server takes all traffic for cvsup.au and
cvsup[2-7].au and is the only FreeBSD CVSup mirror in the South
Pacific/Australasian region.

  http://cvsup3.au.freebsd.org/stats.html

As far as I know, there is still no official svn mirror available this
side of the Pacific, so most of our former CVSup clients have probably
switched to svn and using svn0.us-west (~200ms away).

-- 
John Marshall
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hubs/attachments/20140206/89d2e8ec/attachment.sig>