General purpose library for name/value pairs.

[Teske, Devin]

On Jul 16, 2013, at 3:06 PM, Steve Kiernan wrote:

On Mon, 8 Jul 2013 10:57:17 -0700
Jordan Hubbard <jkh at mail.turbofuzz.com<mailto:jkh at mail.turbofuzz.com>> wrote:


On Jul 8, 2013, at 8:03 AM, Pawel Jakub Dawidek <pjd at freebsd.org<mailto:pjd at freebsd.org>> wrote:

How about instead of supporting int8, uint8, int16, uint16, int32,
uint32, int64 and uint64 I'd just support 'number' type, which would be
uint64_t. This shouldn't break transporting signed values and because I
don't support basic types like int, long, etc. casting or conversion has
to be done anyway. This would reduce number of supported types to:

That's a good idea.  Since you're re-inventing Apple's XML property list API (but without serialization and quite a few other things), keeping the number of supported types down is much better than the converse - exposing the details of 16/32/64 bit numbers and their signedness will hang you over the long term, and you can always provide explicit conversion functions to/from Number for those who actually care.

String, Number, Boolean, Data, Date, Array and Dictionary are all plists support, and Apple developers have gotten along pretty well for many years with that set (not supporting Dictionaries, btw, is a pretty fundamental loss IMHO - it means you have to always iterate through lists to find your stuff, which is meh!).

When Apple extended the Plist metaphor for IPC (to create XPC), they also added out-of-band types like file and shared memory descriptors.  You have file descriptors, but not shared memory.  The latter is kind of important if you want to do larger IPCs with this mechanism someday and want to support "big payloads" transparently without passing the burden of the data management to the application programmer.

Before you also come back with "but but this is a kernel API!  For just one purpose!" let me just say that it's absolutely achievable to have ONE API for talking userland<->kernel and userland<->userland with the same types and the transport mechanism(s) largely abstracted away.  You don't need two - it's already been done with one, just look next door. :)

If you add file serialization of this format to your picture (and add the dictionary type) , bingo, now you have a preferences API that FreeBSD has lacked from day one ("just roll your own format and stick the file in /etc" being the canonical response to that need).

What about looking at NetBSD's libprop?

PROPLIB(3)             FreeBSD Library Functions Manual             PROPLIB(3)

NAME
    proplib -- property container object library

LIBRARY
    library ``libprop''

SYNOPSIS
    #include <prop/proplib.h>

DESCRIPTION
    The proplib library provides an abstract interface for creating and
    manipulating property lists.  Property lists have object types for
    boolean values, opaque data, numbers, and strings.  Structure is provided
    by the array and dictionary collection types.

    Property lists can be passed across protection boundaries by translating
    them to an external representation.  This external representation is an
    XML document whose format is described by the following DTD:

                http://www.apple.com/DTDs/PropertyList-1.0.dtd

    Property container objects are reference counted.  When an object is cre-
    ated, its reference count is set to 1.  Any code that keeps a reference
    to an object, including the collection types (arrays and dictionaries),
    must ``retain'' the object (increment its reference count).  When that
    reference is dropped, the object must be ``released'' (reference count
    decremented).  When an object's reference count drops to 0, it is auto-
    matically freed.

    The rules for managing reference counts are very simple:

    o   If you create an object and do not explicitly maintain a reference to
        it, you must release it.

    o   If you get a reference to an object from other code and wish to main-
        tain a reference to it, you must retain the object.  You are respon-
        sible for releasing the object once you drop that reference.

    o   You must never release an object unless you create it or retain it.

    Object collections may be iterated by creating a special iterator object.
    Iterator objects are special; they may not be retained, and they are
    released using an iterator-specific release function.


There's also my own "figpar" free for the taking...

http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/druidbsd/figpar/

>From figpar.c

dteske at scribe9.vicor.com<mailto:dteske at scribe9.vicor.com> figpar $ wc -l *.h *.c
     123 figpar.h
      40 string_m.h
     376 figpar.c
     292 string_m.c
     831 total

I recently had to resurrect string_m.{c,h} for another project, and made them suitable for the FreeBSD tree:

http://druidbsd.cvs.sourceforge.net/viewvc/druidbsd/fdpv/
(if you're interested; really just pointing at string_m.{c,h})

I wrote figpar back in 2002 and it's pretty simple.

It parses any POSIX style file and lets *you* (as the user of figpar) decide how different matches are handled. It indeed has what the OP was interested in...

Do X when you get NUM (instead of all separate types, etc.).

>From figpar.h:

===

/* cfgvalue_t is a 4 byte union that allows you to store
   varied types of data in a single common container. */
typedef union {

   char *str;            /* a pointer to a null terminated string */
   int32_t num;          /* a signed 32-bit integer value */
   u_int32_t boolean:1;  /* boolean integer value (0 or 1) */
   u_int32_t u_num;      /* an unsigned 32-bit integer value */
   char **strarray;      /* pointer to an array of strings */

} cfgvalue_t;

===

The OP would simply bump that union to 8 bytes and make the default "num" be a u_int64_t (not how there is also "u_num" etc.).

The rest is pretty simple. Of the 499 lines that comprise figpar.h and figpar.c, there's only two typedefs to learn and two funtions to utilize. The rest you bring to the table yourself (you add functions for handling directives by "match" -- uses fnmatch(3) to match a configuration directive and fire your function).

I've used this to parse pretty much any config file ever seen (from apache httpd.conf to jail.conf-style configuration files). HINT: You can match on "{" and implement a state/identifier-machine to implement groupings.

What it does is remove all the nasty details of how to parse a file and let you worry about what gets fired when it deems it's found a match to one of your definitions.

It also tries to keep things nice, as you pass it a struct-array of definitions (which I find helpful because I can have a nice block-definition that's centrally located for continuous munging).

Sure, it'd need some style(9) application (I wrote it in 2002), but it's also extremely portable (tested successfully on over 4-dozen platforms).

As you can see from the union above, it supports most types and you can extend it to support more. But in my experience, I've never needed more than that basic set.

Of course, as it is under 500 lines, it may not be as flexible as bison+yacc, but it's never done me wrong (and pretty easy for any person on a distributed project to grasp and run-with).

My 2 cents.
--
Devin

_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.