Using shell commands versus C equivalents

[Garrett Cooper]

Andrew Turner wrote:
> On Sat, 16 Jun 2007 21:53:06 -0700
> Tim Kientzle <kientzle at freebsd.org> wrote:
>
>   
>>>    Also, were the bottlenecks seen in pkg_delete and pkg_add, or
>>> does it appear to be distributed across the board?
>>>       
>> The biggest time sink in pkg_add is writing each file to a temp
>> dir then copying it to its final location.  There are a couple
>> of strategies for avoiding this (by writing the files directly
>> to their final location), but it basically requires rewriting
>> pkg_add from scratch.  I prototyped this a long time ago and
>> found about a 3x speedup.  (Parts of that prototype eventually
>> became libarchive.)
>>     
> I've also seen a 3x speedup by using my reimplementation of pkg_add
> using my package management library, libpkg (http://libpkg.berlios.de).
> It is not production ready yet as if it fails partway through an
> installation it won't clean up and installed files.
>
>  
>   
>> I haven't looked closely at pkg_delete, but I doubt there's
>> much that can be done to speed it up; once you've examined the
>> dependency information to determine what can be deleted,
>> actually removing the files is a pretty straightforward
>> operation.
>>     
> I ran a quick test on installing and removing a single package with
> both the cvs and my own version of pkg_delete. I got a small but
> significant speed improvement with my implementation. The difference
> was too small to be noticeable by a human though (from 0.11s to 0.07s).
>
> Andrew
>
>   
Your source looks very nice, but there are a few comments:

1. How do you read BDB stuff without including the BDB headers/libs?
2. I can't go and graft your libs, or do something similar with the 
current source because I don't want to break production code 
(pkg_install) of this importance and muck up a lot of user's systems 
irrevocably. I'll be looking at your source though for good ideas and 
see if I can integrate them into the existing pkg_install set though, 
and give you credit where necessary.

Thanks,
-Garrett