Downloading DISTFILES from multiple locations in parallel (torrent or ...)?

Mikhail T. mi+thun at aldan.algebra.com
Sat Nov 22 22:03:32 UTC 2014


Has anybody given the idea any thought? Though in addition to the
"upstream" MASTER_SITES, there is a (fast) collection of FreeBSD
mirrors, the downloads currently aren't happening in parallel -- they
are serial...

One solution would be for the downloader (fetch(1) or whatever) to try
to open multiple connections -- to several possible sites -- and
download different ranges of the same tarball from different locations
in parallel thus affording the total download rate of the actually
fastest mirror (almost), rather than the one first on the list.

The alternative would use one of the existing torrent-clients, but we'd
need to think of how to publish the torrent-information with the ports.
It would seem, this second plan can go further even if the first would
require fewer  changes to the existing infrastructure.

Any thoughts?

    -mi



More information about the freebsd-ports mailing list