General purpose library for name/value pairs.

[Pawel Jakub Dawidek]

Hi.

During my last project I took time to implement general purpose
name/value pair library similar to what exists in Solaris/IllumOS.

Currently we have plenty of random methods to pass data around:
- nmount(2) implements its own name/value pair approach,
- GEOM uses XML to pass data to userland,
- various sysctls and ioctls use binary structures to export data to
  userland.

The libnv library I implemented operates around two types:
- nvpair_t that describes single name/value pair,
- nvlist_t that describes a list of name/value pairs.

In most cases you can do without nvpair_t.

Let's try an example:

	nvlist_t *nvl;

	nvl = nvlist_create(0);
	nvlist_add_string(nvl, "filename", "/tmp/foo");
	nvlist_add_int32(nvl, "flags", O_CREAT | O_WRONLY);
	nvlist_add_uint16(nvl, "mode", 0600);
	if (nvlist_send(sock, nvl) < 0) {
		nvlist_destroy(nvl);
		warn(1, "nvlist_send() failed");
		return (-1);
	}
	nvlist_destroy(nvl);
	return (0);

As you can see we first allocate nvlist and add three elements to it.
Note that we neither check if allocation succeeded nor individual
additions. The library is designed so that write-like operations can
gracefully handle previous failures. For example if nvlist_create()
fails, it returns NULL, but nvlist_add_string() detects if nvl is NULL
and does nothing. If nvlist_create() succeeded, but nvlist_add_string()
failed, nvlist_add_string() will set an error within nvlist structure.
Once we build out nvlist we can either call nvlist_send(), which will
fail (if nvl is NULL or has an error set) or we can call nvlist_error()
to check for error (if nvl is NULL, ENOMEM will be returned).
nvlist_destroy() also accepts NULL nvl and it never modifies global
errno value for convenience (that's why warn() above will show correct
error message).

The API doesn't support types like 'int', 'long' or any other type with
arch-dependent size. This is on purpose. I want this API to be used for
communication between userland and kernel, even if userland is 32bit and
kernel is 64bit, as well as for network communication. nvlist_send()
internally uses nvlist_pack() function that produce binary blob to send
over the network or to the kernel. It implements adaptive endianess
concept known from ZFS - it doesn't convert all values to the network
byte order, but it records machine's order in the header, so if receiver
uses different byte order, only then swapping will be done. If both
machines are either little or big endian no extra swapping will occur.

The library allows to send and receive descriptors, of course only over
UNIX domain sockets:

	nvlist_t *nvl;
	int fd;

	fd = open("/etc/passwd", O_RDONLY);
	if (fd < 0)
		err(1, "open(/etc/passwd) failed");

	nvl = nvlist_create(0);
	nvlist_add_string(nvl, "filename", "/etc/passwd");
	nvlist_move_descriptor(nvl, "fd", fd);
	if (nvlist_send(sock, nvl) < 0)
		err(1, "nvlist_send() failed");
	nvlist_destroy(nvl);

Also note that I used nvlist_move_descriptor() function and not
nvlist_add_descriptor(). The former will allow nvlist to consume the
given descriptor, so we don't have to close it, the latter will dup(2)
the given descriptor and then add it to the nvlist.

So, to add elements to the list one can use either nvlist_add_<type>()
or nvlist_move_<type>() functions. To get the values one also have two
choices: nvlist_get_<type>() and nvlist_take_<type>() - the former just
returns the value (but the value is still part of the nvlist) and the
latter removes associated nvpair from nvlist and returns its value.
For example:

	nvlist_t *nvl;
	const char *command;
	char *filename;
	int fd;

	nvl = nvlist_recv(sock);
	if (nvl == NULL)
		err(1, "nvlist_recv() failed");

	command = nvlist_get_string(nvl, "command");
	filename = nvlist_take_string(nvl, "filename");
	fd = nvlist_take_descriptor(nvl, "fd");

	printf("command=%s filename=%s fd=%d\n", command, filename, fd);

	nvlist_destroy(nvl);
	free(filename);
	close(fd);
	/* command was freed by nvlist_destroy() */

Of course nvlist_move_<type>() and nvlist_take_<type>() functions are
only available for types that require some resources (memory or
descriptor), so there is nvlist_move_string(), nvlist_move_int8_array(),
but there is no nvlist_move_int8(), as it doesn't make sense.

Also note that if there is no element matching the given name and type,
the function will fail on internal assertion. This is caller's
responsibility to verify if the given element is on the list before
trying to get it using nvlist_exists_<type>() API:

	if (nvlist_exists_string(nvl, "command"))
		command = nvlist_get_string(nvl, "command");
	else
		command = NULL;

To iterate over all elements one can do the following:

	nvlist_t *nvl;
	nvpair_t *nvp;

	nvl = nvlist_recv(sock);
	if (nvl == NULL)
		err(1, "nvlist_recv() failed");

	for (nvp = nvlist_first_nvpair(nvl); nvp != NULL;
	    nvp = nvlist_next_nvpair(nvl, nvp)) {
		printf("name=%s type=%d\n", nvpair_name(nvp),
		    nvpair_type(nvp));
	}

For every function that takes name as an argument there are two more
functions that allow to provide name as format string and arguments or
as va_list, for example:

	int16_t nvlist_get_int16(const nvlist_t *nvl, const char *name);
	int16_t nvlist_getf_int16(const nvlist_t *nvl, const char *namefmt, ...) __printflike(2, 3);
	int16_t nvlist_getv_int16(const nvlist_t *nvl, const char *namefmt, va_list nameap) __printflike(2, 0);

The whole implementation can be found here:

	http://people.freebsd.org/~pjd/libnv.tgz

Only the nv.h header file is public, so yes, nvlist and nvpair
structures are not exposed to callers.

I'd be grateful for opinions.

This work was sponsored by the FreeBSD Foundation.

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://mobter.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20130704/e53ac913/attachment.sig>