poudriere: reduce the number of rebuilt packages?

[Baptiste Daroussin]

On Fri, Jan 02, 2015 at 12:03:54PM +0100, Stefan Ehmann wrote:
> I've recently switched from portmaster to poudriere/'pkg upgrade' to 
> manage my port updates. Basically it works fine, but incremental builds 
> don't quite work as I expected.
> 
> poudriere rebuilds all packages if any dependency has changed. If there 
> are only some ports with new versions, possibly hundreds of packages are 
> rebuilt. So far it looks like I'll end up rebuilding packages like 
> libreoffice/KDE/chromium several times a week. The rebuilt packages 
> won't even be installed by 'pkg upgrade' because their version number 
> has not changed.
> 
> That's a huge waste of resources. With portmaster only ports with 
> increased version numbers are rebuilt.
> 
> Can I use poudriere to rebuild only packages where the version number 
> changed?
> 
The problem here in consistency while in theory we can cherry-pick what we
really want to rebuild based on libraries provided/required in binary packages
poudriere has to deal with the ports tree and compatibility.

The ports tree was a heavy user of pkg_add which became pkg add, this tool was
relying on the version of dependencies as registered in the package creation:
- A-v1 was depending on B-v2
if B-v2 is bumped to B-v3 then A-v1 dependency chain is broken in regard
pkg_add.

Just for that we have no choices but rebuilding everything that depends on B

This can now be "easily" fixed because pkg_install is gone and we do not have to
rely on compatiblity with it anymore, the problem is people willing to work on
that (flexible dependencies and smart dependencies) have been mostly killed by
the nightmare this compatibility has introduced into pkg.

Lots of scripts still rely on the pkg_add behaviour and until all of them are
killed I'm afraid we won't be able to prevent those massive rebuilds.

That is if you are doing the things correctly.

Now there is an alternative.

Introduce a new repository format for file:// kind of url (like Zypper) which
will not need all the metadata and be blazing fast to produce. Use pkg install
instead of pkg add in ports and then we can reduce the massive rebuilds to only
rebuild things that really requires a library and only a library being
removed/upgraded.

Any volunteers?

regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20150113/9c3c7f5c/attachment.sig>