lang/gcc43 and lang/gcc44 installation procedures broken after updates

[b. f.]

On 10/28/09, Scott Bennett <bennett at cs.niu.edu> wrote:
>      On Tue, 27 Oct 2009 11:28:51 +0000 "b. f." <bf1783 at googlemail.com>
> wrote:
>>Scott Bennet wrote:
...
>
>      With one exception, I do not alter the
> contents of the ports tree manually.
...
> I have not made alterations to any ports in the ports tree by hand.  Any
> changes that may have occurred would have to have happened during runs of
> portmaster, portupgrade, or make(1) (as in "make deinstall && make
> reinstall"
...
> I resorted to a "portsnap fetch extract" in case something
> in my ports tree *had* gotten screwed up somehow.

Right, I wasn't suggesting it was necessarily due to local changes to
the Ports tree, although on the face of it that was possible, but that
it may also have failed because, once in a while, binaries and other
files belonging to the base system or ports get corrupted, and
malfunction.  This is usually due to hardware problems, user error,
and occasionally, an OS or third-party software bug.  The lang/gcc4?
ports are lengthy and demanding builds, and are among the most likely
to fail if such problems exist.

...
>      The only change I made was indicated by a comment that showed where
> a lot of lines were deleted.  If you really want all that junk, which
> contained no error messages, I do still have it and can send it to you.
> Nothing was rearranged into a different order, however.

You may want to save it, so that it will be available if anyone
decides to try to track down  the problem.

>
>      I do not have MAKEFLAGS set when running portmaster or portupgrade.
> If a particular port decides internally to run a parallel make, it appears
> to do it as -j2.  It appears that the lang/gcc?? ports work this way, too.
>

If parallel builds are not disabled in a port Makefile, or by you, and
you have a multiple-cpu or multiple-core machine, then
ports/Mk/bsd.port.mk uses:


# Multiple make jobs support
.if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE)
_MAKE_JOBS=             #
.else
.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
MAKE_JOBS_NUMBER?=      `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS=             -j${MAKE_JOBS_NUMBER}
.if defined(FORCE_MAKE_JOBS)
BUILD_FAIL_MESSAGE+=    "You have chosen to use multiple make jobs
(parallelization) for all ports.  This port was not tested for this
setting.  Please remove FORCE_MAKE_JOBS and retry the build before
reporting the failure to the maintainer."
.endif
.endif
.endif

to do a parallel build.  Since this feature is relatively new, and
people are occasionally finding that it breaks port builds, then it is
an obvious thing to try disabling in a case like this, where you have
a demanding build, and some evidence that things are being done out of
the proper order.  In the future, you can disable this feature for a
build by setting DISABLE_MAKE_JOBS=yes on the make command line, in
the build environment, or in /etc/make.conf, e.g.:

.if${.CURDIR:M*/usr/ports/lang/gcc44*}
DISABLE_MAKE_JOBS=yes
.endif

>
>      portmaster long since created a backup package and deinstalled the
> ports in question.

Ok.  I don't use portmaster often, but portupgrade will often restore
an old installation of the port from the backup package automatically
after a failure.

...

>  I cannot begin to imagine why
> it worked this way, but refused to work under portmaster or portupgrade.

Occasionally a port exposes a bug in portmaster or portupgrade.  This
may be such a case, especially since Doug Barton made some recent
changes to portmaster.  But the most common reason for failure is that
many ports, to enable easy maintenance, use sloppy flags like
LDFLAGS=-L${LOCALBASE}/lib or CPPFLAGS=-I${LOCALBASE}/include, that
may lead them to link against the older, already installed versions of
themselves, or to include old versions of their own headers if they
are present in the system.  So it's always safer to deinstall a port
_before_ attempting to build it, or to build the port in a clean
sandbox as is done on many package-building clusters.  portmaster and
portupgrade choose not to do this, in order to shorten the process of
recovering from a failed build, and to minimize the time during which
a piece of software is unavailable to users, and this can lead to
problems.  I don't say that this happened in this case, but it is a
possibility.

> I guess I will just have to add "-x gcc\*" to the
> "portmaster -x perl\*5.8.9\* -a" runs from now on, which is now possible
> thanks to Doug Barton's portmaster enhancement that allows multiple "-x"
> arguments, and do lang/gcc* updates by the old-fashioned method that worked
> in this case.  I'm not sure what to do if a situation arises like this for
> a port that has many dependencies that would typically be better managed by
> portmaster or portupgrade, however.

You don't have to do it on the command line -- you can add the port to
HOLD_PKGS in pkgtools.conf with portupgrade, or use a
/var/db/pkg/*/+IGNOREME as described in portmaster(8).  It's a bit of
a pain to manage large updates -- you may be better off fashioning
your own tool.  You could, for example, decide on a list of ports to
update, and then determine the ports that depend upon them, and sort
the lot into dependency order via:

pkgdb -L    (to be sure the pkgdb has got the dependency graph right)
pkg_glob -r <insert list of ports to update here> | pkg_sort > /tmp/updatelist

Now you could just feed this sorted update list to your own updating
script, or use one of the existing tools.  For example, if you wanted
to use portupgrade, assuming that lang/gcc44 is midway down the update
list, and you've got lang/gcc44 in HOLD_PKGS in pkgtools.conf:

sed -n '1,/gcc-4.4/p' /tmp/updatelist | xargs portupgrade --batch -cfuv

cd /tmp && pkg_create -b gcc-4.4* && make -C /usr/ports/lang/gcc44
deinstall clean install && make -C /usr/ports/lang/gcc44 clean

sed -n '/gcc-4.4/,$p' /tmp/updatelist | xargs portupgrade --batch -cfuv

and youi've done the update in only a few steps.  If you  place it in
a script, it will be even easier.  With portmaster, using -a, -f, or
-r, you could try interrupting or pausing the build at lang/gcc44,
rebuilding that port by hand, and then resuming the build, with -R and
the other appropriate flags if necessary.

b.