Switching /etc/*.db from bdb to tinycdb

[Peter Wemm]

On Wed, May 2, 2012 at 4:41 AM, Baptiste Daroussin <bapt at freebsd.org> wrote:
> Hi,
>
> I plan to work on switching all the /etc/*.db files from bdb 1.85 to cdb (using
> the tinycdb implementation)
>
> Reason for doing that:
> 1/ we use the *.db files as a constant database on recreating them using
> *_mkdb(8) commands
> 2/ bdb 1.85 database are endianness dependant which can cause problem if I want
> pkgng to be able to create users when cross installing packages (like adding
> packages in a mips eb chroot environment on an amd64 host)
> 3/ the tinycdb API is clean and easy to use (I find it easier and cleaner to
> deal with it)
> 4/ tinycdb is public domain so not problem for us to use it.
>
> On the side effects, I'm not sure this is relevant but tinycdb is also faster to
> read then bdb 1.85.
>
> How do I plan to do it.
>
> I plan to import the read part or tinycdb into libc and the write part into
> libutil (to avoid cluttering libc).
>
> Why importing to libc? to allow all the get*ent to parse the /etc/*.db files.
>
> How to implment it? slowly with backward compatibility:
> for each kind of files: try to open with cdb, if fail fall back to bdb.
>
> I plan to rewrite all the *_mkdb for make them use cdb.
>
> Link for tinycdb:
> http://www.corpit.ru/mjt/tinycdb.html
>
> Any opinion, ideas, advices on this?

Please keep in mind that old binaries need to keep working.

Old libc doesn't have a text file parser in it - the only thing it
knows is how to read the *.db files, so they can never go away or be
made incompatible.

eg: there are people "supposedly" running a libc.so.4 based oracle.
Even FreeBSD-2.1.5 a.out binaries still understand the *.db files that
we use (assuming you dig up the a.out stuff to make them run).

Please keep backwards compatability in mind.  Even if that means
moving db/* to a standalone shim/converter utility or have some other
way to generate/update the "old" *.db files.  This has far bigger
impact than things like the utmp/utx change.

Also keep in mind the symbol scope stuff may not necessarily have the
same visibility goals as everyything else.  eg: libc.so.7 based
binaries from an old machine should still "see" the new implementation
- things like getpwnam() ABI won't be changing.   You *do want* an
ancient libc.so.7 binary that was compiled on 7.0-release to correctly
use the new implementation when its run on a 10.0 machine.

Please take a lot of care with a change like this.
-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell