freebsd-update painfully slow - slower than source code build of world and kernel

[Christopher Arnold]

On Tue, 6 Jan 2009, Daniel Bond wrote:

> Regarding portsnap in my previous post, I think you misunderstood me. This is 
> not a new "one time" problem regarding a specific case, portsnap is allways 
> slow. This is observed from heavy usage of it, over a long period of time.
>
This is not my experience, but shure i realise that mileages can vary.

> Great to see that there will be an update2.freebsd.org - unfortunately, that 
> a new release generates more traffic than update-server handles is not 
> acceptable (imho). People should be able to upgrade to a new release, once it 
> is out. Sadly, I don't think one more mirror will cut it. Especially if it is 
> going to be of the same quality as the other portsnap mirrors. Also, sadly CP 
> isn't looking for more mirrors, while a large chunk of users trying to 
> upgrade *are* looking for mirrors.
>
portsnap is extremly lightweight, so it might be just fine.

But then i am not arguing against you, more and better infrastructure is 
always good. Lets wait untill the us has woken up (And maybe add some 
extra time for the right person to look into the current problems) and see 
what kind of feedback we get from people who have more insight into this issue.

> Look at CVSUP mirrors, they have always worked fine, even directly after a 
> new release. We even have a few of them here in Norway, and they are fast as 
> hell. Look how many there are of them, spread around the world.. This works 
> out great!
>
My experience from when i was based in Sweden is the opposit. Shortly 
after a major release cvsup always had problems syncing due to the load on 
the servers.

> However, freebsd-update is closed. I've searched the web for how the protocol 
> works, how the server-part of it works, with metadata, checksums and all. How 
> the mirroring of it works, basicly. There are no public available documents 
> on this. Do we have to reverse-engineer it, or what?
>
If we start off with portsnap it is http-based and the in the manual you 
can find:
"If you wish to use portsnap to keep a large number of machines 
up to date, you may wish to set up a caching HTTP proxy.  Since portsnap 
uses fetch(1) to download updates, setting the HTTP_PROXY environment 
variable will direct it to fetch updates from the given proxy.  This is 
much more efficient than mirroring the files on the portsnap server, since 
the vast majority of files are not needed by any particular client."

So it's straight forward to speed up portsnap. (But then if the central 
servers break like today this dosn't help.)

Im not shure about freebsd-update, but since they are both written by 
Colin and the fact that they seem simmilar in config etc. i would guess 
that the same applies to freebsd-update.

So lets wait for some input from Colin or someone else who know the ins 
and outs of freebsd-update.

 	/Chris

--
http://www.arnold.se/chris/