freebsd-update painfully slow - slower than source code build
of world and kernel
Daniel Bond
db at danielbond.org
Tue Jan 6 12:08:07 UTC 2009
Hi again Christopher,
reading your answer, you are obviously confusing what I am saying
about freebsd-update with the portsnap program. Also, I also wrote in
my first post that HTTP_PROXY / Caching proxy server does not help me
much. This is because I download a lot of "initial tarball
snapshots".. I would rarely see "Cache hits" in my proxy log. I guess
I could set something up to fetch nightly via proxy, to keep the data
in house, for when I need it. I don't want to use a PROXY server, I
feel this is attacking the problem at the wrong end.
I agree, I am interested to hear the views of the wise ones.
Personally I'm going back to CVSup until freebsd-update and portsnap
mirrors are in a more distributed or usable state.
Cheers.
On Jan 6, 2009, at 12:59 PM, Christopher Arnold wrote:
>
>
> On Tue, 6 Jan 2009, Daniel Bond wrote:
>
>> Regarding portsnap in my previous post, I think you misunderstood
>> me. This is not a new "one time" problem regarding a specific case,
>> portsnap is allways slow. This is observed from heavy usage of it,
>> over a long period of time.
>>
> This is not my experience, but shure i realise that mileages can vary.
>
>> Great to see that there will be an update2.freebsd.org -
>> unfortunately, that a new release generates more traffic than
>> update-server handles is not acceptable (imho). People should be
>> able to upgrade to a new release, once it is out. Sadly, I don't
>> think one more mirror will cut it. Especially if it is going to be
>> of the same quality as the other portsnap mirrors. Also, sadly CP
>> isn't looking for more mirrors, while a large chunk of users trying
>> to upgrade *are* looking for mirrors.
>>
> portsnap is extremly lightweight, so it might be just fine.
>
> But then i am not arguing against you, more and better
> infrastructure is always good. Lets wait untill the us has woken up
> (And maybe add some extra time for the right person to look into the
> current problems) and see what kind of feedback we get from people
> who have more insight into this issue.
>
>> Look at CVSUP mirrors, they have always worked fine, even directly
>> after a new release. We even have a few of them here in Norway, and
>> they are fast as hell. Look how many there are of them, spread
>> around the world.. This works out great!
>>
> My experience from when i was based in Sweden is the opposit.
> Shortly after a major release cvsup always had problems syncing due
> to the load on the servers.
>
>> However, freebsd-update is closed. I've searched the web for how
>> the protocol works, how the server-part of it works, with metadata,
>> checksums and all. How the mirroring of it works, basicly. There
>> are no public available documents on this. Do we have to reverse-
>> engineer it, or what?
>>
> If we start off with portsnap it is http-based and the in the manual
> you can find:
> "If you wish to use portsnap to keep a large number of machines up
> to date, you may wish to set up a caching HTTP proxy. Since
> portsnap uses fetch(1) to download updates, setting the HTTP_PROXY
> environment variable will direct it to fetch updates from the given
> proxy. This is much more efficient than mirroring the files on the
> portsnap server, since the vast majority of files are not needed by
> any particular client."
>
> So it's straight forward to speed up portsnap. (But then if the
> central servers break like today this dosn't help.)
>
> Im not shure about freebsd-update, but since they are both written
> by Colin and the fact that they seem simmilar in config etc. i would
> guess that the same applies to freebsd-update.
>
> So lets wait for some input from Colin or someone else who know the
> ins and outs of freebsd-update.
>
> /Chris
>
> --
> http://www.arnold.se/chris/
>
More information about the freebsd-stable
mailing list