Just a thought

[Baptiste Daroussin]

On Tue, Jul 21, 2015 at 06:36:17PM -0700, Don Lewis wrote:
> On 22 Jul, Michelle Sullivan wrote:
> > I'm using poudriere 3.0.19 (for good reason) so I don't know if this is
> > already done, but from a recent set of mailings I don't think so...
> > 
> > Also pkgng doesn't seem to do it (again based on recent mailings)
> > 
> > You have the following package set:
> > 
> > pkga-2.0.1 => pkgb-1.34.5 => pkgc-34.4.2 => pkgd-1.1.1
> > 
> > where a => b 'a' is a dependency of 'b'.
> > 
> > if the options are change for pkgb are changed, *or* pkgb has a new
> > version 1.35.0 (or both)
> > 
> > Why are both pkgc and pkgd rebuilt/reinstalled?
> > 
> > Surely (logically) pkgc-34.4.2 needs to be rebuild because pkgb's ABI
> > *may* have been changed... however pkgd should *not* need to be
> > rebuilt/reinstalled because what it depends on has only been rebuilt for
> > the current (last) version and should have absolutely no ABI change.
> > 
> > So in the real world if (recent example) OpenSSL is rebuilt because of a
> > new version/patch (1.0.2a -> 1.0.2c) only those ports that actually have
> > OpenSSL as a *direct dependency* (ie those that link to the openssl
> > libraries) need to be rebuilt (they might not actually even need that if
> > it's just a security patch that doesn't change any of the public
> > interface, but one cannot easily tell that so rebuilding is required to
> > be safe) ... but why would you need to rebuild 1000+ ports (my case)
> > because OpenSSL has been patched... ie modssl has been update and
> > therefore apache has been updated then everything thing under the sun
> > that depends on apache gets rebuilt because apache was rebuilt even
> > though the apache version is identical....?
> 
> Poudriere is kind of dumb about this.  If it sees that pkgb needs to be
> rebuilt, then it blows away pkgc, and then it blows away pkgd because it
> sees that pkgd's dependency (pkgc) is missing.
> 
> What you say is mostly true, but there are some cases where rebuiding
> pkgd is necessary because it turns out to directly depend on pkgb but
> the dependency is not listed as a direct dependency.  One potential
> cause of this is the configure script in pkgd noticing the presence of
> pkgb and adding it to the list of libraries to be linked to.  Another
> case can happen if the .pc file for pkgc lists a library from pkgb as a
> library to also link to when linking to a library from pkgc.  Poudriere
> won't catch this when it builds pkgd because when it installs pkgc, pkgb
> also gets installed because it is a dependency of pkgc.  In theory the
> Q/A tools should be able to catch this by looking at the shared
> libraries that are directly referenced by the shared libraries and
> executables in pkgd and verifying that the come from packages that are
> listed as dependencies, but that's not how things are currently
> implemented.


The main reason for the useless rebuild is fairly simple and we are working on a
fix for that: pkg inherited from pkg_tools the was the dependencies are
registered (aka name-version) which is totally insane as a version is hardcoded,
we are working to dropping that version. This version is only used for the add
subcommnd which is only used by the ports tree itself and people that have
switched to pkg but "only" did the quick switch of their tools/script by s/_//g
from pkg_tools to pkg. In pkg we just made a modification which will allow to
stop hardcoded those version hence reduce the over rebuild of package in
incremental build on poudriere.

Note that the other reason for overrebuild is to catch overlinking (a package
not officially directly depending on another one but which end up linked - in a
elf file to it)

The last reason is to catch "forgotten" revision bumped.

You might see after pkg 1.6 is out a reduction of the over rebuild (at least
option to have a less aggressive incremental build).
> 
> > ...even pkg has this issue it seems... the only minor difference is if a
> > package is installed it should (in theory) use the already downloaded
> > (cached) version of that package... but why does it need to
> > deinstall/reinstall? (or is that only what it displays and it doesn't
> > actually do it?)
> 
As explained above pkg will not reinstall all of them but only the one actually
needed. pkg has way more useful information to decided when reinstalling is
needed (like shared libraries used) than the ports tree at build time, so we
can have a saner model of reinstall and mostly only reinstall what we really
need to reinstall.

Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20150722/2bc5d5e1/attachment.bin>