/pub/FreeBSD or /FreeBSD

[Oliver Fromme]

Dominic Fandrey wrote:
 > I think I might have made a mistake here. I checked again and my
 > estimate for half the servers was a) grossly exaggerated and b)
 > the issue is more present over http.
 > 
 > Those don't work:
 > http://ftp.de.freebsd.org/pub/FreeBSD/
 > http://ftp3.de.freebsd.org/pub/FreeBSD/
 > http://ftp4.de.freebsd.org/pub/FreeBSD/
 > http://ftp6.de.freebsd.org/pub/FreeBSD/ (no HTTP at all)
 > 
 > These work:
 > ftp://ftp.de.freebsd.org/pub/FreeBSD/
 > ftp://ftp3.de.freebsd.org/pub/FreeBSD/
 > ftp://ftp6.de.freebsd.org/pub/FreeBSD/
 > 
 > So, I have three distinct problems with half the German mirrors,
 > two offer /pub/FreeBSD only through FTP, but not HTTP.

It is desirable that all FTP mirrors also offer the data
via HTTP (and possibly also rsync or other protocols),
but it is not a "must".  So, it is not surprising that
some mirrors do not support HTTP.

 > One (ftp4.de) does not offer /pub/FreeBSD at all.

I consider that a bug.

By the way, there's a mailing list specifically for the
German mirrors:  de-bsd-hubs at de.freebsd.org  (posting in
either German or English is ok).

 > And one does not offer HTTP access, so if I switched back to FTP,
 > 3 quarters of my problem would disappear and I'd have 8 working
 > mirrors instead of 5 out of 9.
 > 
 > So I should probably justify my use of HTTP:
 > - Less latency (important for small downloads)
 >   - 8-stable/All/automounter-1.4.3.tbz with the same connection:
 >     HTTP: 0.20s, 0.20s, 0.20s
 >     FTP: 0.39s, 0.62s, 0.39s, 0.39s (I'd throw the 0.62s away as a glitch)
 >   - It needs 77 packages to install firefox, 70 of these are below 1.5m,
 >     which is a small file by broadband standards, i.e. for 90% of
 >     the packages latency is an important parameter

It depends very much on how the protocols are implemented
by server and client.  If you use HTTP and *both* server
and client support keep-alive, all files can be downloaded
with a single connection.  Otherwise you need n connections
(e.g. 77 connections for 77 files).  Some clients also open
several connections in parallel.

With FTP, you have to use one control connection, and one
data connection per file, so you have n+1 connections total
(e.g. 78 connections for 77 files).  So it's not that much
different from the non-keep-alive case.  I think FreeBSD's
fetch(3) does not support keep-alive.

Besides the number of connections, the overhead for FTP is
slightly larger.  That's because HTTP is a simple request-
reply protocol, while FTP requires an additional round-trip
because of the negotiation of the port number for the data
connection (PORT or PASV commands, depending on whether
active or passive connections are used).

Since FreeBSD's fetch(3) (used by sysinstall, pkg_add and
the ports framework) does neither support keep-alive nor
parallel downloads, the difference between HTTP and FTP is
very small in practice.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

Python is executable pseudocode.  Perl is executable line noise.