Utility for safe updating of ports in base system

Peter Jeremy peterjeremy at optushome.com.au
Thu Mar 20 23:21:16 PDT 2008

On Thu, Mar 20, 2008 at 12:48:00PM -0700, Doug Barton wrote:
>Michel Talon wrote:
>> Since the compilation will take most of the
>> time, it is not relevant to consider performance questions on the
>> portmaster side.
>Having spent a substantial amount of time doing performance tuning on
>portmaster, I beg to differ.

I also agree with Doug here.  Compilation may take most of the time
but any time used by the ports upgrading infrastructure will directly
add to the total time taken by an upgrade.  And, with the switch to
"modular X.org", systems are likely to have lots more ports than
before so any per-port overhead is much more of issue than before.

The "compilation will take most of the time so don't worry about
anything else" approach leads to the GNU configure and libtool idiocy
- where the port spends several minutes determining the system
configuration and then 30 seconds compiling the port.

>> Several causes of slowness have already been identified. Parsing
>> /var/db/pkg is slow,
>s/is/can be/. There are some things I need to dig out of /var/db/pkg
>that are relatively slow, yes. By far the most expensive is
>determining the name of an installed port by grep'ing for ORIGIN in
>all the +CONTENTS files.
>Now, would having some kind of database be better? Maybe, maybe not.

Note that UFS is a database: If I've understood you correctly, the
main problem is that there is no appropriate index to map a port
directory to an installed package name.  This could be fixed...

There are regular discussions about moving more configuration
information into some sort of database format.  We already use
Berkeley DB for some things and importing SQLite has been suggested
on at least one occasion.  It would be interesting to see if changing
/var/db/pkg into a "traditional" database will improve performance
without adversely affecting administration.

>> this is why the idea of using a database has been
>> advocated. Much worse, running make in a port directory, even only to
>> get the value of a variable is slow.
>Now on that I agree with you completely. But now you're talking about
>a much more fundamental redesign issue that is way outside the scope
>of the project we're discussing.

One option here might be to restrict port Makefiles to a subset of
pmake's functionality and move much of the configuration data
currently embedded in /usr/ports/Mk/* into separate files so that
'make describe' can be implemented using an alternative tool that
doesn't need to parse most of /usr/ports/Mk/*.

>Doing N number of downloads in parallel (where N should be user
>configurable) is preferable.  ...  OTOH, I've
>never had a user complain that I'm swamping their machine with
>distfile downloads, so it's not been a high priority.

I've seen a couple of FTP mirror sites that explicitly state "no more
than one client connection at a time is allowed from a single
computer" as part of their conditions of use.  Swamping a mirror site
with parallel downloads may be seen as unfriendly by the site admins.

>> Probably the present package management system doesn't keep enough state
>> on each installed package so that an upgrade system works reliably.
>Here you and I are in agreement.

<metoo> I doubt we will ever manage to move to a totally binary
upgrade approach for all ports - for licensing reasons if nothing
else.  This means that an end user will probably wind up building some
ports.  It's also unlikely that the "one-size-fits-all" approach
implied by the package system will suit everyone - my favourite gotcha
is the lack of mod_php when you use the php and Apache packages.  I
also occasionally say "make -DWITH_foo -DWITHOUT_bar" and it would be
handy to know what options I actually used when I go to rebuild the
port (or try to work out why a dependency isn't working as expected).

Peter Jeremy
Please excuse any delays as the result of my ISP's inability to implement
an MTA that is either RFC2821-compliant or matches their claimed behaviour.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20080321/7329c14b/attachment.pgp

More information about the freebsd-ports mailing list