pkgng default schedule... registering a few reasons for rethinking the final implementation...

[Matthew Seaman]

On 23/08/2012 20:50, Kris Moore wrote:
> Well, it was about time I got to doing a benchmark of this anyway :)
> 
> I did quick benchmark of how one of our utilities parses through a list
> of 1k packages on a newer i5 system:
> 
> First test, using /var/db/pkg/<pkg> check we have been doing:
> 
> 0.178s 0:00.31 54.8%
> 0.123s 0:00.26 61.5%
> 0.099s 0:00.15 60.0%
>  
> Second test, using "pkg info <pkg>":
> 
> 5.347s 0:11.41 91.7% 
> 5.444s 0:11.52 91.3%
> 5.878s 0:11.32 91.4%
> 
> The pkg info command is quite a bit slower in this case, but 5 seconds
> isn't horrible.
> 
> Now I ran the same benchmark on a slower 1.66gz Atom system, checking
> about 1200~ packages:
> 
> First test, using /var/db/pkg/<pkg> check we have been doing:
> 
> 0.604s 0:00.76 86.8%
> 0.622s 0:00.77 84.4%
> 0.614s 0:00.73 90.4%
>  
> Second test, using "pkg info <pkg>":
> 
> 28.507s 0:54.80 99.1%
> 28.282s 0:54.60 99.4%
> 28.302s 0:54.52 99.4%
> 
> Now this is what concerns me a bit. It took closer to 30 seconds, which
> is quite a while to wait, especially if a utility like ours has to run
> these checks when it starts up, to show the user whats installed / not
> installed on the system.
> 
> The only way around It I've found is to do a quick "pkg info" on the
> entire DB, dump that to a list, then begin to grep through that list for
> each item, but it still takes 10~ seconds on the atom. That may be what
> I end up having to do, but it still stinks to go from a half a second
> startup, to 10 seconds each time. Any other ideas on how to do this
> faster with the new pkgng?

Doing one big pkg query in bulk is going to be faster than sequentially
querying for one package at a time.  Sqlite startup time is significant,
and probably relatively more so on slower hardware.

What data are you trying to extract from pkg info?  Did you try using
pkg query instead?  (pkg query does much the same as pkg info, but is
more oriented towards scripting.)


-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 267 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20120823/4ae96528/signature.pgp