DPS Initial Ideas

[Kris Kennaway]

On Sat, May 12, 2007 at 11:44:22PM +0200, Michel Talon wrote:
> On Sat, May 12, 2007 at 03:33:02PM -0400, Kris Kennaway wrote:
> > On Sat, May 12, 2007 at 11:09:35AM +0200, Michel Talon wrote:
> > 
> > > Seriously, the FreeBSD package
> > > system is in great need of a profound overhaul, pretending it works well
> > > is complete denial of reality. I hope that young people working on 
> > > summer code projects will infuse *new* ideas, and not spend their
> > > vacations polishing inadequate tools.
> > 
> > I know that this is your belief, but please try to avoid grasping at
> > straws: there are elements in your argument that are along the lines
> > of "The FreeBSD package system is broken and needs to be fundamentally
> > changed.  Rewriting it to use SQLite is a fundamental change.
> > Therefore rewriting it to use SQLite will fix the problems."
> > 
> 
> Really i don't think at all this way. I think that *perhaps* SQLite
> may marginally better than a Berkeley database for solving part of the
> problem, not much more. What i reacted to, was the conservatism which 
> pervades the community as soon as someone emits the idea of using a new tool. 

It seems to me that you do not appreciate the reasons behind this
conservatism.  A very important one is that we have two students who
have committed to spending their summer working on improving the
existing pkg_tools in ways that will solve some of the real problems
we are facing, and the project we have agreed upon is that they will
be using existing tools rather than rewriting from scratch as part of
a not-yet-defined larger project.

To some extent it is the timing here that is most unfortunate.  If
SQLite has been raised as a viable alternative a few months ago it
would have made a great project idea, but instead we have committed to
improving our existing tools, and the barrier for throwing out these
plans is therefore very high.  The burden of proof is therefore set
much higher than "SQL is awesome and buzzword-compliant and might be
better".

> It may be that borrowing from Debian the idea of "abstract" dependencies
> which can be fulfilled by several concrete packages may also simplify
> the dependency problem. For example tomcat may depend on "java" and java
> my be fulfilled either by diablo-jdk15 or jdk15. This way when you change
> from diablo-jdk15 to jdk15 you don't need to change anything to tomcat.
> 
> Another feature that Debian has, and which may happily complete the previous
> one, is the specification of necessary dependencies with a version number
> in a certain range (this obviously requires a reasonable standardisation of
> version numbers, so that comparison of <some package>-0.99 to 
> <some package>-1.0-rc doesn't depend on arcane rules). This way you don't need
> to change dependencies which are in the correct range, even if a more recent
> version exists. This mechanism has been imported in NetBSD pkgsrc.

We actually have both of these features in ports, but they have not
been pushed down into packages.  I think it will be relatively simple
to do so, without requiring a rewrite from scratch.

> And a problem which has proven useful in Debian is keeping track of the
> packages which have been required by the end user and those which have been
> installed as dependencies. This is the difference between apt-get and
> aptitude. Apparently people are very happy to be able to remove not only
> a package they have required, but also all its dependencies (which are
> not required by another program) at one stroke. This also helps in case
> some big package requires dependency A, but after upgrade, they have changed
> their mind and require alternative dependency B. With this mechanism, after
> upgrade A disappears, while without it you will have both an upgraded version
> of A and B. I have observed on my machine this is an important cause 
> of time monotonic bloat of the package tree.

This one could also be added to the existing tools.

> To answer the slowness problem in registering installed packages, one may
> think about making use of the INDEX file. In fact all the information that
> is necessary to fill the dependency entries is contained in INDEX, and
> accessible here in milliseconds with any tool such as awk. It so happens that
> the ports system doesn't make any use of the INDEX file and systematically
> recomputes the dependencies through recursive make invocations which are very
> time consuming. Of course this requires up to date INDEX, or a mechanism to
> keep INDEX continually up to date.

The problem is that maintaining the INDEX is expensive and/or tricky.
p5-FreeBSD-Portindex comes close but seems to have some wrinkles.

> Part of the registration is also filling the +REQUIRED_BY files of the
> dependencies of a package when one installs a package.  If this package has a
> lot of dependencies this means opening, editing and closing a large number of
> files. This is expensive. One may imagine using a database containing the
> global dependency information, then +REQUIRED_BY files are no more necessary,
> since the information can be recomputed in very little time.

Yes, this is one thing that is addressed in the current plan.

> > Given that this work is happening (or at least will be happening, I am
> > not sure when the SoC officially starts), the best thing is for
> > interested people to work with Garrett to help him achieve the goals
> > of his project.
> 
> Sure. I am convinced this is the reason why several people, including myself
> present some ideas in the mailing list now, before Garrett begins working on
> his project.

I think this is a retcon; the original poster did not appear to have
this motivation and neither do many others who have posted.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20070512/af027db0/attachment.pgp