BSD tar (was Re: Making pkg_XXX tools smarter about file types...)

[Wes Peters]

On Friday 28 March 2003 22:47, Tim Kientzle wrote:
>
> P.S.  It's galled me for a while that pkg_add has
> to fork 'tar' to extract the archive. 

Me too, me too.

> I've started
> piecing together a library that reads/writes tarfiles.

Excellent.  A general design goal in userland should be to implement 
functionality in libraries and then wrap small driver programs around 
them to export the basic functionality to userland.  I guess this partly 
violates the original UNIX tools philosophy, but all it really does is 
move it from the original pipes interfaces into the dynamic linker.

> With this, it should be possible to make pkg_add
> considerably more efficient.  In particular, rather
> than extracting to a temp directory, then parsing important
> information, then moving the files, it should be
> possible using this library to read the initial
> entries ("+CONTENTS", in particular) directly into
> memory, process the information there, then extract the
> remainder of the package files directly into their
> final locations. 

I'd much rather see the metadata moved outside the file archives, but 
that's a separate argument and in now way detracts from your proposed 
work.  ;^)

> So far, I have a library API
> outlined, and functional read support implemented.
> Next step is to hack up a minimal tar implementation
> that uses it to make sure everything's working
> correctly.
>
> So far, the library automatically detects compression
> formats (using techniques like those in my
> pkg_install patch) and has some rough support
> for detecting the archive format as well.
> (One goal of mine: support for 'pax extended
> archives', which I understand can handle ACLs.)

I have wondered outloud before if pax might be a suitable starting place 
for such a hacking expedition.  Others who've worked on pax assure me it 
is not.  ;^(  Sigh.  So much code, so few programmers.

> Of course, such a library could also form the
> basis for a BSD-licensed tar to replace GNU tar.
> I understand a few people have wanted such a thing.

Why yes, yes they have.

-- 

        Where am I, and what am I doing in this handbasket?

Wes Peters                                               wes at softweyr.com