[RFC] SA/EN ABI/Lib flagging for package rebuilding

[Bryan Drewery]

Currently for any SA/EN we force rebuild all packages for
pkg.freebsd.org. This is a very time-consuming process at 24 hours for
each set and 10 (non-head) sets to build.

We really only need to rebuild packages if ABI/KBI is changed (which I
would think would happen almost never) or if a library is updated. The
library being updated needs a rebuild in case of packages linking
against the library statically. There's no quick way to determine what
package may be infected by a static linkage to a library so we just
rebuild them all.

There are 2 somewhat conflicting goals here. We need head packages to be
usable on all ABI/KBI changes. The simplest here was decided to just
rebuilding head packages always. This is a compromise I am willing to
accept. I had wanted to use __FreeBSD_version for this in the past but
it tends to be missed or bumped gratuitously.


My idea to cover both cases so that packages will only rebuild if the
public API or libraries change in the jail or anything else which
affects the resulting binaries.

I will take a checksum of the following and if anything differs then do
a rebuild:

/lib /usr/lib /libexec (.so included due to symbol versioning and
detecting added/removed libraries which can change packages)
/usr/include (all public API for the system including kernel API for
modules)
/usr/libdata

That is all I am aware of for the public API/KPI/ABI/KBI that is
relevant for packages.

I also realize I need to include some of the binutils files as they may
change how binaries are produced. This is all of 'ar as ld nm objcopy
objdump ranlib readelf strip' in /usr/bin/. Also all of GCC/Clang
binaries in /usr/bin: 'cc clang clang++ clang-cpp clang-tblgen cpp
c++filt CC c++ g++ gcc gnu-ar gnu-ranlib'.

I don't want to consider all of /bin /usr/bin /sbin and /usr/sbin as it
catches SA that do not really need a rebuild, such as
FreeBSD-SA-15:07.ntp which modified /usr/sbin/ntpd.

By having a whitelist of files from *bin* I risk not rebuilding in a
rare case where /bin/sh or /bin/cp (random example) had a vulnerability
fixed in them that could change resulting binaries. I consider that to
be very unlikely though. Perhaps for head packages it is more relevant
but I still consider it unlikely other changes will require rebuilding
packages.

If I forgot other critical pieces from *bin* please list them.

An alternative would be for there to be a reliable flag in head and
SA/EN noting whether binaries need to be rebuilt. There had been a
technical limitation to bumping __FreeBSD_version for SA as it would
modify all binaries. I think that was fixed. Portmgr fought hard to
always have __FreeBSD_version bumped but that's a lost cause. As I
mentioned I don't trust this flag in head either.

All feedback and other ideas welcome.

Thanks,
Bryan Drewery

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20150409/36852839/attachment.sig>