Importing netbsd cdb

[Baptiste Daroussin]

On Wed, Nov 06, 2013 at 10:45:50AM +0200, Atte Peltomaki wrote:
> On Tue, Nov 05, 2013 at 11:02:26PM +0100, Jeremie Le Hen wrote:
> > On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote:
> > > 
> > > Here is a code that imports the cdbrw from netbsd into a new lib/libcdbrw
> > > library, the read part is also added to libc but not exposed.
> > > 
> > > As an example of using that library I also got the service_mkdb patches from
> > > netbsd that makes it by default emit a cdb database and add a switch to allow it
> > > to still create the old .db database format.
> > > 
> > > in the libc, getservent has been modified to first try to read the .cdb files
> > > and fallback on reading the old .db file. (I'm not sure if it is worth keeping
> > > reading the old db format.)
> > > 
> > > http://people.freebsd.org/~bapt/cdbrw.diff
> > > 
> > > The plan after that is to get pw_util(3) directly generating a cdb file for
> > > pwd.db and spwd.db and modifiy pwd_mkdb(8) so that by default it uses the API
> > > from pw_util(3) and have a switch to fallback on creating in bdb format.
> > > 
> > > I also plan to do the same for cap_mkdb(1) and getcap(3).
> > > 
> > > With cdb querying is way faster (I don't have number but I can get some it
> > > needed)
> > > the size of the db is also way smaller:
> > >  64K   /var/db/services.cdb
> > >  2,1M    /var/db/services.db
> > > 
> > > Any objection?
> > 
> > What are the benefits of this, beside generating a much smaller
> > database?  Do you know about benchmarks that compare it against the
> > current database engine used?  It would be unfortunate to notice a
> > regression afterward :).
> 
> At least randomly googled third-party benchmarks suggest that CDB is
> vastly superior to BDB:
> http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/
> 

From the netbsd commit log, it says that in case of services it is at least as
slow, and at best faster, from my testing it is way faster, (sorry no numbers)

Concerning the migration, the way I did it for now, is that getservent is trying
to open the cdb database and fallsback on the bdb one, so migrating to
11.0-RELEASE should be ootb.

Running:
services_mkdb

Will generate the cdb version of the db

services_mkdb -V db

Will generate the old format db.

regards,
Bapt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20131106/efd62998/attachment.sig>