Query about the usage of FreeBSD mirror sites for other projects

[Fabian Wenk]

Hello Thomas

On 21.02.2012 03:39, Thomas Abthorpe wrote:
> Recently, a question came up within our community, should we be using
> other project sites for distfile fetching?  Sites like Debian and Gentoo
> mirror many of the tarballs we use for our ports.

For my local FreeBSD and Gentoo servers I use a shared distfiles 
folder over NFS. All systems are allowed to save new downloads, 
and if the file already exists, they are using it instead of 
downloading.

As far as I see it, in the FreeBSD distfiles there are also sub 
directories, where Gentoo does not use sub directories.

I did a quick check with sysutils/fdupes and there are a few 
files which are saved at two different locations. On my systems 
the following projects are affected from this:
Perl, Ruby, Apache and Gnome

The files with root:wheel have been downloaded from FreeBSD 
Ports, and the files with portage:portage from Gentoo (sorry for 
the line wraps):

# cd /usr/ports/distfiles

# find . -name perl-5.12.4.tar.bz2 -ls
329842 24160 -rw-r--r-- 1 root    wheel   12350353 Jun 20  2011 
./perl/perl-5.12.4.tar.bz2
237981 24160 -rw-rw-r-- 1 portage portage 12350353 Nov 10  2010 
./perl-5.12.4.tar.bz2

# find . -name ruby-1.8.7-p352.tar.bz2 -ls
237747  8256 -rw-r--r-- 1 root    wheel    4207576 Jul  2  2011 
./ruby/ruby-1.8.7-p352.tar.bz2
237991  8256 -rw-rw-r-- 1 portage portage  4207576 Jul  2  2011 
./ruby-1.8.7-p352.tar.bz2

# find . -name httpd-2.2.21.tar.bz2 -ls
282742 10464 -rw-r--r-- 1 root    wheel    5324905 Sep 12 19:02 
./apache22/httpd-2.2.21.tar.bz2
237936 10464 -rw-rw-r-- 1 portage portage  5324905 Sep 15 11:50 
./httpd-2.2.21.tar.bz2

# find . -name glib-2.28.8.tar.xz -ls
259229 10240 -rw-r--r-- 1 root    wheel    5223564 Jun  6  2011 
./gnome2/glib-2.28.8.tar.xz
237741 10240 -rw-rw-r-- 1 portage portage  5223564 Jun  6  2011 
./glib-2.28.8.tar.xz

As far as I see it, it is the same structure on the mirror 
servers too. So I guess to really be able to use each others 
mirrors, some adjustments are needed either in Ports (FreeBSD) or 
in Portage (Gentoo). Else some of the downloads will fail or use 
the fall back download source.


bye
Fabian