SoC2009: libpkg, pkg tools rewrite

[Garrett Cooper]

On Thu, Apr 30, 2009 at 2:45 PM, Jeremy Lea <reg at freebsd.org> wrote:
> Hi,
>
> On Sat, Apr 25, 2009 at 03:20:59PM -0400, David Forsythe wrote:
>> This summer I'll be working on creating a package library and using
>> that library to rewrite the pkg tools.  A package library has been
>> discussed and even started before, but FreeBSD still does not have
>> one.  This summer I'd like to get enough of the library done to atle
>> ast have a new set of pkg tools completed with the current features,
>> but ideally I'd like to get far enough to splice in some of the ideas
>> I have for new features.
>
> Since I've already done most of the work on this already, please,
> please, please, don't ignore what I have done.  I know that it is not
> perfect, and that it is now five plus years out of date.  But I have
> covered half of the bullet points on your to-do list already.
>
> The code is at http://sourceforge.net/projects/fpkg and some README
> stuff is at http://fpkg.sourceforge.net/
>
> Some things which I think are critical:
>
> 1. The library needs a global "package manager".  This needs to perform
> all of the tasks, and it should ideally do this through a task queue
> (which I didn't implement).  See the lib/lib.h header in FreePKG.
>
> 2. The package manager must be able to separate out a structure of
> variables which can be determined without opening the +CONTENTS file.
> This must also include a package state, and the package manager must
> move these package headers from state to state.
>
> 3. There needs to be proper depends handling in the packages, so that we
> can maintain the correct dependency graph.  This is vital for upgrading.
> This means adding @libdep and @filedep types to the package file.
>
> 4. bsd.port.mk should do everything through the tools.  It should have
> no knowledge of the contents of /var/db/pkg.
>
> These are all done, to some degree in the code.  The mistakes I made:
>
> 1. I made the file->pkg database to sensitive.  If there is a miss it
> rebuilds the database for scratch - it should do a search through the
> +CONTENTS files and only rebuild it if there was a hit (meaning the
> database was wrong).
>
> 2. There needs to be a pkgname and origin database, which can be loaded
> at startup to prime the package manager.  The dependency graph should
> also be stored in a database.  These should be rebuilt if any directory
> in /var/db/pkg has a mtime later than the database (so could the file
> database).
>
> 3. There needs to be a set of flags which indicate how a package got
> installed (as a dependency or by the user), and if it has been upgraded
> in-place and might have old leftover libraries.  These could go in
> +CONTENTS.
>
> In addition I also began the design of a new on disk package format.
> This was back when there was a group called openpackages which was
> trying to standardize ports/packages between the BSDs.  In particular it
> has some ideas on package signing, which is an often requested feature.
>
> http://people.freebsd.org/~reg/opdesign.html
>
> Regards,
>  -Jeremy

    There's also http://libpkg.berlios.de/ that you could use as a
starting point, and please, PLEASE, don't neglect the SoC work that
was done last year by Anders Nore. It was committed to p4, but hasn't
made it upstream to CVS yet...
    Also, don't fall into the trap I fell into 2 years ago by trying
to reinvent the wheel. Plenty of people have been down this road
already, and can help you in finding your way if you ask.
Best of luck,
-Garrett