cvs commit: ports/Mk bsd.port.mk

[Kris Kennaway]

On Tue, Aug 07, 2007 at 03:37:44PM +0200, Alexander Leidinger wrote:
> Quoting Kris Kennaway <kris at obsecurity.org> (from Mon, 6 Aug 2007  
> 13:38:08 -0400):
> 
> >On Mon, Aug 06, 2007 at 01:34:06PM +0200, Michael Nottebrock wrote:
> >>Alexander Leidinger schrieb:
> >>> Kris, what technical reasons are against explicit dependencies, in
> >>> your opinion?
> >>Explicit dependencies would be great, if they can be guaranteed to be
> >>correct, which basically means we need a way auto-generate them. Maybe
> >>this could be done in a similar way to the security check target - run
> >>ldd/objdump over installed executables and libraries, record symbol
> >>names somewhere, determine dependencies by comparing records ...
> >>
> >>Explicit dependencies that need to be determined and maintained manually
> >>by port maintainers are useless, since they'll be almost guaranteed to
> >>be wrong most of the time for those ports that would profit the most
> >>(shave off the most implicit dependencies) from having them.
> >
> >Yes, this is the most serious problem.  Also there is no need to
> >introduce a new variable to handle it: if you want to record explicit
> 
> I fail to see where a new variable comes into play...

OK, I misunderstood how EXPLICIT_PACKAGE_DEPENDS is to be used.  I
think I understand now :)

> >dependencies a better way is to use LIB_ or RUN_DEPENDS and track the
> >direct vs inherited dependencies differently in the package database.
> 
> Why do you want to differentiate between implicit and explicit  
> dependencies? We don't need to list the implicit ones, they are  
> resolved implicitly already even without listing them in the package.

Having to compute them recursively is inefficient.  It's more
efficient to keep the full transitive list for each port and update
that information at the time when one of the ports changes.

> Let's step back for a moment...
> 
> ATM we record the dependencies and the dependencies of the  
> dependencies and so on in a package. I call this implicit dependency  
> recording.
> 
> With explicit dependency recording I associate to only record the  
> dependencies in the package, which are listed in LIB_ or RUN_DEPENDS  
> of the corresponding port for the package.
> 
> What I propose and what is implemented with the feature switch which  
> triggered this discussion is: to do the explicit dependency recording  
> instead of the implicit dependency recording. The ports collection  
> resolves the dependencies recursively so we don't need to record the  
> implicit dependencies. They are superflous in +CONTENTS (similar for  
> the corresponding entries in +REQUIRED_BY).
> 
> This changes
>  - the amount (less) of packages listed in INDEX for a given port.
>  - the amount (less) of dependencies listed in +CONTENTS for a given
>    port/package.
>  - the amount (less) of ports listed in +REQUIRED_BY.

This is part of the problem: you're trading space for time when we're
doing a lot of work to *reduce* the time complexity of ports.  At the
least we'd need to (separately) maintain both the explicit and the
full set of implicit dependencies for each port for efficiency
reasons.  This is an implementation detail though.

> When we _would_ switch _now_ it does _not_ change
>  - the dependency tree for a port.
>  - the amount of installed ports when installing x11/gnome2 or x11/kde
>    or whatever (as they will still be resolved and installed
>    implicitly).
> 
> It allows to have those nice development/update/build properties I  
> talked about in a previous post.

Well sort of.  Sometimes you can get away with only updating the ports
that explicitly depend on a changed port, but often you have to
transitively update everything that depends on it.  There is no way to
reliably tell which situation you're in, so the safe option remains to
always recurse.

> To get those nice properties, we have  
> to extend the LIB_ and RUN_DEPENDS of our ports with all those  
> dependencies, which the binaries of the port in question reference but  
> have not listed already in the Makefile. Not all ports in the ports  
> collection fail to have all dependencies, but those which fail them  
> are most of the time not easy (gnome/kde/... are a huge number of  
> ports and are the prominent ports which would need such a treatment).
> 
> When we switch to explicit dependency recording without adding the  
> explicit LIB_ and RUN_DEPENDS
>  - we gain the benefit of using less space (on disk/CD, via network
>    transfer).
>  - we will not break existing ports.
>  - we don't gain the build/update/development benefits.
> 
> 
> Can someone please point me to a strong technical reason not to switch  
> to explicit dependency recording? If not, are there strong technical  
> reasons to not record all the required dependencies in the ports which  
> are missing them after switching to explicit dependency recording  
> (note: please see my other mail where I inlined scripts to  
> automatically find suitable LIB_DEPENDS lines for the common case)?

I think I missed this.  It might help with another (unrelated) case,
where port builds autodetect optional components that happen to be
installed, and then link to them without the port registering a
dependency.

Probably these scripts should be added to Tools/.

> Kris, if you are concerned about the impact on the ports collection,  
> please make a exp-build run with this feature and compare it to a  
> normal run.

OK.

Kris