bin/119196: [wish] parallel download from multiple servers by fetch

[Mikhail T.]

>Number:         119196
>Category:       bin
>Synopsis:       [wish] parallel download from multiple servers by fetch
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 31 18:30:02 UTC 2007
>Closed-Date:
>Last-Modified:
>Originator:     Mikhail T.
>Release:        FreeBSD 6.3-PRERELEASE amd64
>Organization:
Virtual Estates, Inc.
>Environment:
	FreeBSD

>Description:
	In most cases, when fetch(1) is invoked (or libfetch called
	into), it is to either download sources for a port, or
	binaries for a package.

	In all of those cases multiple servers hosting the same files
	are known to the caller. Unfortunately, the caller must use
	the servers /in sequence/, because fetch does not allow
	downloading different ranges of the same file from different
	sources /in parallel/.

	This is, of course, suboptimal as some servers (such as
	ftp.freebsd.org) are hit by much higher loads. The users
	may also have a (much) wider download bandwidth, than an
	upload bandwidth, that a particular server has (or would
	allow a single client to use).

	It should be possible to implement parallel downloads (over
	different protocols -- http, ftp, etc.). The implementation
	should also be adaptive: if the download from one of the
	servers finishes faster, the program should proceed to
	automatically fetch a not-yet-downloaded sub-range of the
	range, that is being downloaded from another server.

	Various "p2p" downloading programs (torrent, gnuttella)
	implement this feature already -- out of necessity. We
	should do it, because we can.

	No, I don't have a patch. Not yet, anyway... Somebody looking
	for a cool project is most welcome to this idea.

>How-To-Repeat:
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted: