ports/128754: [port infrastructure] implement master sites randomization

Eygene Ryabinkin rea-fbsd at codelabs.ru
Mon Nov 10 08:00:11 PST 2008

>Number:         128754
>Category:       ports
>Synopsis:       [port infrastructure] implement master sites randomization
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-ports-bugs
>State:          open
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 10 16:00:09 UTC 2008
>Originator:     Eygene Ryabinkin
>Release:        FreeBSD 7.1-PRERELEASE i386
Code Labs

System: FreeBSD 7.1-PRERELEASE i386


Today I was hit by the very bad connectivity with twaren.net and
nchc.dl.sourceforge.net (the first site in the SF mirrors list in the
FreeBSD ports .mk files) is hosted by Taiwan REN.  So, I decided to
implement simple randomization that will enable to evenly distribute
the downloads between SF mirrors.

I had made it only for the SF mirrors, but it can be done for the other
well-known download locations as well.


Look at ${PORTSDIR}/Mk/bsd.sites.mk: it uses statical assignment to


The following patch works for me.  It places simple script rnd.awk to
the ${PORTSDIR}/Mk -- I don't think that this is the most appropriate
place to put it, but I can't think of another place now -- the script is
too ports-specific to put it into, for example, /usr/bin/.

--- 0001-Implement-randomization-of-main-SourceForge-sites.patch begins here ---
>From d48aa3982a5ca6192fad561462c3ba9da2b38cfa Mon Sep 17 00:00:00 2001
From: Eygene Ryabinkin <rea-fbsd at codelabs.ru>
Date: Mon, 10 Nov 2008 18:38:01 +0300

The reasoning behind this is simple: today I had experienced very bad
connectivity with twaren.net and it is the first mirror in the list.
So, for automated upgrades of ports that are using SourceForge mirrors,
the download was very time-consuming.  Such round-robin randomization
of the download sites helps to improve the situation.

Signed-off-by: Eygene Ryabinkin <rea-fbsd at codelabs.ru>
 Mk/bsd.commands.mk |    1 +
 Mk/bsd.sites.mk    |    9 ++++++---
 Mk/rnd.awk         |   14 ++++++++++++++
 3 files changed, 21 insertions(+), 3 deletions(-)
 create mode 100644 Mk/rnd.awk

diff --git a/Mk/bsd.commands.mk b/Mk/bsd.commands.mk
index aa8bde8..0b2a58b 100644
--- a/Mk/bsd.commands.mk
+++ b/Mk/bsd.commands.mk
@@ -69,6 +69,7 @@ PASTE?=		/usr/bin/paste
 PAX?=		/bin/pax
 PRINTF?=	/usr/bin/printf
 PW?=		/usr/sbin/pw
+RANDLIST?=	${AWK} -f ${PORTSDIR}/Mk/rnd.awk
 REALPATH?=	/bin/realpath
 RM?=		/bin/rm
 RMDIR?=		/bin/rmdir
diff --git a/Mk/bsd.sites.mk b/Mk/bsd.sites.mk
index 41b3f93..bd9a2a6 100644
--- a/Mk/bsd.sites.mk
+++ b/Mk/bsd.sites.mk
@@ -1188,20 +1188,23 @@ MASTER_SITE_SAVANNAH+= \
 .for mirror in nchc kent easynews ufpr umn mesh heanet
 # official sf.net mirrors that don't mirror all projects, check
 # http://prdownloads.sourceforge.net/%SUBDIR%/
 .for mirror in easynews switch puzzle belnet osdn ovh keihanna
diff --git a/Mk/rnd.awk b/Mk/rnd.awk
new file mode 100644
index 0000000..fbfcad7
--- /dev/null
+++ b/Mk/rnd.awk
@@ -0,0 +1,14 @@
+# Need to drop a couple of initial rand() values: they tend
+# to be around 0.8 - 0.9, so for fairly small array lenght
+# they will produce identical values at the beginning.
+	srand(); rand(); rand(); rand(); rand();
+	for (i = ARGC - 1; i > 0; i--) {
+		j = int(10*ARGC*rand()) % i + 1;
+		if (j == i) continue;
+		t = ARGV[i]; ARGV[i] = ARGV[j]; ARGV[j] = t;
+	}
+	for (i = 1; i < ARGC; i++)
+		print ARGV[i];
--- 0001-Implement-randomization-of-main-SourceForge-sites.patch ends here ---

More information about the freebsd-ports mailing list