[Machine readable output from userland utilities] report

[Zaro Korchev]

Hi Jonathan
I just saw your email. I must have missed it earlier. Sorry about that.

The functions were just some example names. I forgot to put the sol_ prefix in all of them.

I understand your concerns about multi-threading. The idea is to have functions that serialize the object in an allocated buffer as it is constructed. Here is a more detailed example of what I mean:

char *ucldump_object_start(const struct ucldump *restrict ctx, char *buf, size_t buf_len)
{
	if (ctx->format == UCL_EMIT_JSON_COMPACT)
	{
		if (buf_len < 1) return 0;
		buf[0] = '{';
		return buf + 1;
	}
	// etc.
}

char *ucldump_object_key(const struct ucldump *restrict ctx, char *restrict buf, size_t buf_len, const char *restrict key, size_t key_len)
{
	if (ctx->format == UCL_EMIT_JSON_COMPACT)
	{
		// Make sure there is enough space for
		// "key":
		if (buf_len < (1 + key_len + 1 + 1)) return 0;

		*buf++ = '"';
		*buf++ = '"';
		memcpy(buf, key, key_len);
		buf += key_len;
		*buf++ = ':';

		return buf;
	}
	// etc.
}

This is still just an example. I don't know what will be the most appropriate way to implement this.

The idea is that for JSON and YAML sol_object_start() can call ucldump_object_start(), sol_object_key can call ucldump_object_key() and so on. This way UCL can take care of the exact output format while SOL provides a uniform API for all formats (including other formats like XML that can use a different backend).

Since the serialized data is written to an allocated buffer, such functions will not be an issue for multi-threading. Multi-threaded applications just need to use separate buffers and separate files/sockets/pipes/whatever (which they must do anyway).


Zaro


Le 28 May 2014 à 15:16, Jonathan Anderson a écrit :

> On 28 May 2014, at 9:36, Zaro Korchev <zkorchev at mail.bg> wrote:
>> The idea is to serialize the object as it is constructed. That way it is not necessary to keep all the data in memory at any given moment. This also allows to create pipeline when the second application starts consuming output before the first one finished producing it.
>> 
>> for example:
>> object_start(buf, ...) writes to buf:
>> {
>> object_key(buf, ..., "foo") writes to buf:
>> "foo"
>> sol_object_integer(buf, ..., 42) writes to buf:
>> : 42
>> sol_object_end(buf, ...) writes to buf:
>> }
> 
> I notice that you have a mix of "object_" and "sol_object_" calls: are these all purely serialization functions (and should they all be prefixed with "sol_")? It would be unfortunate to commit to a stateful object *creation* API, as that would make things difficult for multithreaded work (e.g. each thread creates an object, each of which is added to the top-level object as it is completed rather than in lexical order).
> 
> 
> Jon
> --
> Jonathan Anderson
> 
> jonathan at FreeBSD.org
> http://freebsd.org/~jonathan/
>