Just a thought

Wed Jul 22 01:37:46 UTC 2015

On 22 Jul, Michelle Sullivan wrote:
> I'm using poudriere 3.0.19 (for good reason) so I don't know if this is
> already done, but from a recent set of mailings I don't think so...
> 
> Also pkgng doesn't seem to do it (again based on recent mailings)
> 
> You have the following package set:
> 
> pkga-2.0.1 => pkgb-1.34.5 => pkgc-34.4.2 => pkgd-1.1.1
> 
> where a => b 'a' is a dependency of 'b'.
> 
> if the options are change for pkgb are changed, *or* pkgb has a new
> version 1.35.0 (or both)
> 
> Why are both pkgc and pkgd rebuilt/reinstalled?
> 
> Surely (logically) pkgc-34.4.2 needs to be rebuild because pkgb's ABI
> *may* have been changed... however pkgd should *not* need to be
> rebuilt/reinstalled because what it depends on has only been rebuilt for
> the current (last) version and should have absolutely no ABI change.
> 
> So in the real world if (recent example) OpenSSL is rebuilt because of a
> new version/patch (1.0.2a -> 1.0.2c) only those ports that actually have
> OpenSSL as a *direct dependency* (ie those that link to the openssl
> libraries) need to be rebuilt (they might not actually even need that if
> it's just a security patch that doesn't change any of the public
> interface, but one cannot easily tell that so rebuilding is required to
> be safe) ... but why would you need to rebuild 1000+ ports (my case)
> because OpenSSL has been patched... ie modssl has been update and
> therefore apache has been updated then everything thing under the sun
> that depends on apache gets rebuilt because apache was rebuilt even
> though the apache version is identical....?

Poudriere is kind of dumb about this.  If it sees that pkgb needs to be
rebuilt, then it blows away pkgc, and then it blows away pkgd because it
sees that pkgd's dependency (pkgc) is missing.

What you say is mostly true, but there are some cases where rebuiding
pkgd is necessary because it turns out to directly depend on pkgb but
the dependency is not listed as a direct dependency.  One potential
cause of this is the configure script in pkgd noticing the presence of
pkgb and adding it to the list of libraries to be linked to.  Another
case can happen if the .pc file for pkgc lists a library from pkgb as a
library to also link to when linking to a library from pkgc.  Poudriere
won't catch this when it builds pkgd because when it installs pkgc, pkgb
also gets installed because it is a dependency of pkgc.  In theory the
Q/A tools should be able to catch this by looking at the shared
libraries that are directly referenced by the shared libraries and
executables in pkgd and verifying that the come from packages that are
listed as dependencies, but that's not how things are currently
implemented.

> ...even pkg has this issue it seems... the only minor difference is if a
> package is installed it should (in theory) use the already downloaded
> (cached) version of that package... but why does it need to
> deinstall/reinstall? (or is that only what it displays and it doesn't
> actually do it?)

I haven't noticed this problem with pkg.  My last poudriere run rebuilt
~500 of ~1400 packages, but pkg only upgraded a handful and only
re-installed one.

# pkg upgrade
Updating myrepo repository catalogue...
Fetching meta.txz: 100%    292 B   0.3kB/s    00:01    
Fetching packagesite.txz: 100%  332 KiB 340.5kB/s    00:01    
Processing entries: 100%
myrepo repository update completed. 1419 packages processed.
Checking for upgrades (563 candidates): 100%
Processing candidates (563 candidates): 100%
The following 21 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	texlive-tlmgr: 20150523
	libpotrace: 1.11

Installed packages to be UPGRADED:
	tracker: 1.2.6_1 -> 1.2.6_2
	totem-pl-parser: 3.10.3 -> 3.10.3_1
	texlive-texmf: 20140525_4 -> 20150523
	texlive-base: 20140525_7 -> 20150521_1
	tex-xmltex: 1.9_1 -> 1.9_2
	tex-web2c: 20140525_2 -> 20150521
	tex-ptexenc: 1.3.2 -> 1.3.3_1
	tex-kpathsea: 6.2.0_1 -> 6.2.1
	tex-jadetex: 3.13_2 -> 3.13_3
	tex-formats: 20140525_1 -> 20150521
	tex-dvipsk: 5.994_1 -> 5.995_1
	policykit: 0.9_8 -> 0.9_9
	pkgconf: 0.9.11 -> 0.9.12
	pcre: 8.37_1 -> 8.37_2
	p5-Params-Validate: 1.20 -> 1.21
	gvfs: 1.20.3_2 -> 1.20.3_3
	file-roller: 3.14.2,1 -> 3.14.2_1,1
	cmake: 3.2.3_1 -> 3.2.3_2

Installed packages to be REINSTALLED:
	libgxps-0.2.2_1 (direct dependency added: libarchive)

The process will require 126 MiB more space.
611 MiB to be downloaded.

Proceed with this action? [y/N]: y
Fetching tracker-1.2.6_2.txz: 100%    1 MiB   1.5MB/s    00:01    
Fetching totem-pl-parser-3.10.3_1.txz: 100%  194 KiB 199.3kB/s    00:01    
Fetching texlive-texmf-20150523.txz: 100%  588 MiB  11.9MB/s    00:52    
Fetching texlive-base-20150521_1.txz: 100%    2 MiB   2.8MB/s    00:01    
Fetching tex-xmltex-1.9_2.txz: 100%    1 MiB   1.0MB/s    00:01    
Fetching tex-web2c-20150521.txz: 100%    1 MiB   1.5MB/s    00:01    
Fetching tex-ptexenc-1.3.3_1.txz: 100%   29 KiB  30.2kB/s    00:01    
Fetching tex-kpathsea-6.2.1.txz: 100%  164 KiB 168.2kB/s    00:01    
Fetching tex-jadetex-3.13_3.txz: 100%    1 MiB   1.8MB/s    00:01    
Fetching tex-formats-20150521.txz: 100%    5 MiB   5.6MB/s    00:01    
Fetching tex-dvipsk-5.995_1.txz: 100%    2 MiB   2.2MB/s    00:01    
Fetching policykit-0.9_9.txz: 100%  138 KiB 141.9kB/s    00:01    
Fetching pkgconf-0.9.12.txz: 100%   20 KiB  21.1kB/s    00:01    
Fetching pcre-8.37_2.txz: 100%    1 MiB   1.1MB/s    00:01    
Fetching p5-Params-Validate-1.21.txz: 100%   44 KiB  45.4kB/s    00:01    
Fetching libgxps-0.2.2_1.txz: 100%   95 KiB  98.0kB/s    00:01    
Fetching gvfs-1.20.3_3.txz: 100%  900 KiB 922.3kB/s    00:01    
Fetching file-roller-3.14.2_1,1.txz: 100%  927 KiB 949.3kB/s    00:01    
Fetching cmake-3.2.3_2.txz: 100%    3 MiB   3.6MB/s    00:01    
Fetching texlive-tlmgr-20150523.txz: 100%   88 KiB  90.3kB/s    00:01    
Fetching libpotrace-1.11.txz: 100%   30 KiB  30.9kB/s    00:01    
Checking integrity... done (0 conflicting
[snip]

I do find the current situation painful because the list of ports that I
build contains a bunch that are large and take a long time to compile,
such as openoffice-4, openoffice-devel, webkit-gtk2, webkit-gtk3,
firefox, thunderbird, chromium, octave, and scilab.  These all have a
large number of dependencies and almost alway get rebuilt when I run
poudriere.  My last poudriere run took about 8 hours, multiplied by four
combinations of FreeBSD version and arch.