Need to build some systems this week. Snapshots?

[Wes Peters]

On Thursday 28 August 2003 07:32 pm, Brett Glass wrote:
> At 02:22 PM 8/28/2003, Colin Percival wrote:
> >  FreeBSD Update only concerns itself with the base FreeBSD
> > distribution -- I simply don't have the resources to build any more
> > than that.  However, one simple approach to the ports problem would
> > be to # find /usr/local/ -perm +111 -type f -exec file {} \; | grep
> > "statically linked" | cut -f 1 -d ':' and rebuild the applicable
> > ports.  Now that I think about it, I might add some sort of
> > functionality like that (providing a listing of ports which need to
> > be rebuilt) into a future version of FreeBSD Update.
>
> This would be a big help. It would be even better if it could also
> identify binary packages that need updating (since this, after all,
> has historically been one of the biggest problems with updating
> FreeBSD.
>
> Of course, the problem with packages is more serious than with ports,
> because the project has always (for no reason that I can see other
> than habit) treated binary packages as "second class citizens"
> compared to ports.

Not really, no.  A binary package is actually the end product of 
building a port.

> Ports can be updated at any time and recompiled.

Which is a major consideration.  The real fly in the binary package 
ointment are the dependencies.  If you have package a installed that 
requires version 1.2 of package b, and package c installed that also 
required version 1.2 of package b, what do you do when you want to 
upgrade c to something that requires version 1.4 of package b but you 
don't want to upgrade package a?

In a perfect world, every port/package would always build, would always 
install cleanly supporting multiple versions, etc.  Here in the real 
world, half or more of the packages are originally created by, er, 
"programmers" who don't even realize how import a $DESTDIR is, let 
alone plan for installing multiple versions side-by-side.

One of the values YOU can add for your customers is to setup a port 
build machine in your office or home and produce releases of the 
packages your customers need, compiled on the exact same version of the 
operating system they are running, and deliver the results either 
on-line or via CD-R.  This is exactly the sort of thing an appliance 
manufacturer does for their customers, plus generally adding a 
mechanism for (mostly) automated on-line installation.

> But if there's a bug in a program which has been installed as a
> package, there's no way for a user to get a freshened package until
> the next release of the OS!

Poppycock.  They can get a freshened package anytime anyone on the whole 
planet builds one for them, including themselves.  It's really not that 
difficult to run CVSup and then 'portupgrade', it really isn't.  I used 
to do it over a modem, then a wireless link, and now a cable modem.  
One of them is a lot faster, but the transfer time is 'background' 
time, you don't have to sit there and wait for it.

> While the project builds binary snapshots
> of the OS itself nightly, it doesn't rebuild and post packages in
> between releases. Yes, a user can sometimes uninstall the package and
> reinstall the same application as a port (after fixing the relevant
> libraries). But if disk space is tight, or the system is embedded or
> doesn't include a compiler (embedded or semi-embedded implementations
> of the BSDs are becoming more and more common), this might not be
> possible.
>
> I'd like to see the project rebuild binary packages regularly so that
> a user (or a utility such as your updater) can fetch repaired
> versions of them as needed. It should be easy to tell which ones need
> rebuilding, so that it's unnecessary to rebuild the entire collection
> every night.

We do, sort of.  The entire ports heirarchy is rebuilt continually on 
the bento cluster.  The next step would be to provide the results of 
these builds to a sufficient number of FTP servers to make them 
generally useful, but there are other structural problems in the way 
that would make that a less than fruitful exercise.

The real problem here is that the entire ports collection is a 
continually changing mass, and a rather big one at that, except during 
the 'ports freeze' prior to a release.  This is the only time we have 
people concentrate on getting ports into a semblance of releasability, 
and has its limitations as it exists now.

The ports manager team is doing a pretty good job of fighting off chaos 
at this moment, but the ports system is getting big enough that at any 
given time a fair percentage of it won't even build, let alone produce 
useful executables, libraries, etc.  This is a natural state for 
something that is essentially a collection of 9,000 ongoing development 
projects with many complicated and varied interactions.  In short, this 
is just not going to happen unless somebody steps up with a half dozen 
full time developers whose 'day job' is to produce ports snapshots.

Now, if you want to discuss a way for the ports builders to say "Wow, 
that seemed like a pretty good one!" when bento builds somewhere north 
of 90% of the packages, so they push a button and produce a snapshot, 
that's a least a possibility.  That will never handle the awful morass 
of dependencies on complicated packages like GNOME or KDE applications, 
but it MIGHT produce occasional snapshots of some use to some people.

-- 
         "Where am I, and what am I doing in this handbasket?"

Wes Peters                                                  Softweyr LLC
wes at softweyr.com                                    http://softweyr.com/