Upgrading through packages: an experience.

[Michel Talon]

Hello,

Since KDE recently appeared in the Latest prebuilt packages, and my 
main desktop ports (running FreeBSD-7.0-RELEASE) had not been upgraded
since more than a year i decided to test my pkgupgrade
(www.lpthe.jussieu.fr/~talon/pkgupgrade) on a machine with many ports
installed (all Gnome and KDE basically, plus many other things) and
going through several important changes in the ports system (the
new modularized Xorg, the new gettext, etc.). I wanted a worst case
test, including using the Latest packages and not a RELEASE. 

The net result is that it went through without problems, at a speed
comparable to that one observes for Debian upgrades, but at the end
some small glitches remained, due to using Latest packages.

Some details follow:

-first the script pkgupgrade crashed. This is due to the fact that the
old python version used libpthread, and there are apparent bugs on 7.0,
while it worked OK on 6.2.  This was solved using libmap.conf to force
use of libthr for threading. Since KSE is now deprecated it is not
useful to describe this bug more fully. Anyways programs doing IO and
threading don't act reliably with libpthread under 7.0.

- after that things were smooth. There has been fantastic progress
in the utility pkg_add, it runs at least twice faster than last year.
This explains the very speedy upgrade i had. More precisely, first the
specs of the machine, it is a 4 years old Pentium 4 with IDE disk, so
nothing particularly fast. There were 744 installed ports initially.
Some of them i put on hold, since they are very heavy to build or install
(java, tetex). So finally pkgupgrade removed 616 old packages and
installed 877 new packages (the difference is of course because of the
new modular Xorg). A few remained to be compiled. At the end i have now
1017 installed ports.

- the speed of this upgrade was really remarkable. The initial analysis
by pkgupgrade took 2 min, the download of necessary packages (for a
total of 1.3 GB) from the french FreeBSD mirror took 5 min (i have a
100Mb/s connection to it, but this also shows the gains of maintaining a
unique ftp connection to the server for the whole download), and was
simultaneous with backing up the old packages (only shared libraries and
config files) which took 28 mn.  So after half an hour i had a shell
script that i reviewed fully to avoid removing stuff that i had
forgotten to put on hold.

- then i runned the shell script UpgradeShell, which removed old
packages (took 18 mn, pkg_delete is still slow, compared to pkg_add)
then installed 877 packages in 48 mn! This is absolutely fantastic, last
year i spent here 2 hours in a much smaller installation. Finally it
compiled a few packages, notably one of them kdewebdev took 2 hours
by itself, that is much more than the complete upgrade procedure. This
illustrates the point that compiling from source is a waste of time.
I have put the Logs here http://www.lpthe.jussieu.fr/~talon/2008-07-11.tgz
so that one can see how it runs in practice.


Now the problems. They come from the fact that the Latest packages are
not always coherent between themselves or with the libraries in
FreeBSD-7.0. So, while most programs runned perfectly well, a few
crashed when i tried them. I traced most problems to the fact that
the use of fcntl() to lock files did not work in some cases. This was
for example the case for tin when it tries to lock the "posted"  file.
I had to recompile such ports, now they run fine. An example which
doesn't run is kdm because it needs to lock /var/run/kdm.lock. This
is very annoying since kdebase is huge to recompile. All other KDE
stuff runs OK. The lesson of this story is that it is *not* a good
idea to use the Latest packages when doing binary upgrades, one should
stick to RELEASE.


Conclusion: binary upgrades, even in very messy situation are very
doable. They take a reasonable time, completely comparable to what one
may expect under Linux. Compiling a single KDE port took longer that
the whole upgrade procedure except the compilation. 
 

-- 

Michel TALON