portsearch: approach to finding ports that provide a given file

[Vasil Dimov]

History:
--------

Some time ago I decided that
$ find /usr/ports -name pkg-plist |xargs grep something
is not the best way to search for ports that install `something'
Some obvious disadvantages of using this method are:
* hell slow
* misses PLIST_FILES variables in Makefiles
* %%VARIABLES%% are not expanded
* too long and complex to type

The tool:
---------

So I created a tool that addresses the above points. What it does is
to create a database with expanded PLIST_FILES and %%VARIABLES% of all
ports' plists for later adequate and fast searching.

Anyway tool's database is completely sufficient to satisfy search
requests not only based on plist contents - it is possible to do
everything that `make search' and /usr/ports/Tools/scripts/portsearch
can do. Currently it can search for ports by name (e.g. like
make search name=). I intend to add all functionality of `make search'
and scripts/portsearch.

Tool's database contains one index file and one plist file. The index
file is created from /usr/ports/INDEX with unique numbers prepended to
each line, giving every port (short) unique identifier. The plist file
contains number and plist-file where number identifies the port,
owning the plist-file, by its unique identifier.

Optimizations and time statistics:
----------------------------------

Since generating plists for all the ports is a slow operation (longer
than `make index') the tool can update its database, once created - not
generating the plists for ports that have not been changed.

Some statistics, can be used for rough comparison:
make index: 19 min
database creation from scratch: 23 min
database creation when no ports have been changed: 20 sec
database creation when ports tree is 1 day newer than the database: 1 min
time to find ports that provide some 2 (or more) letters files:
  immediate (less than a second)
time to find ports that provide .* (all ports match, all 1464060 files
  get output-ed): 4 sec

Flaws:
------
Some ports generate their plists after being installed, making it
impossible to record the plist without fetching the source tarballs,
compiling and installing the software. Currently for these ports empty
or flawed plists are recorded. One way to fix this problem is to
retrieve the plists for these ports from the precompiled packages
available at ftp.freebsd.org but the whole packages need to be fetched,
causing (large) amount of unefficient network traffic.
Any ideas for resolving the problem are welcome.

About the name:
---------------
I choose `portsearch' for obvious reasons.
It will not interfere with any existing software, particularly
/usr/ports/Tools/scripts/portsearch as the tool installs in
/usr/local/bin/ and /usr/ports/Tools/scripts/ is not in the PATH
by default. On 23 Dec 2005 I emailed the author of
scripts/portsearch Mark Ovens <marko at freebsd.org> and asked him
if he has something against a tool with the same name to exist
in /usr/local/bin/. I got no answer since then.

Where is it:
------------

The tool can be downloaded from
http://vdev.datamax.bg/portsearch/
I intend to submit it as a port in the next few weeks.

Please send any questions, comments and suggestions.

-- 
Vasil Dimov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 155 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20060117/0fa04770/attachment.bin