[GSoC] Machine readable output from userland utilities

[Chris H]

>
> On 5/20/14, 1:33 PM, Erik Cederstrand wrote:
>> Den 20/05/2014 kl. 18.59 skrev Zaro Korchev <zkorchev at mail.bg>:
>>
>>> I'm working on the project "Machine readable output from userland utilities" and I want
>>> to share my ideas and thoughts.
>>>
>>> [...]
>>> Do you have any suggestions or ideas?
>> Microsoft PowerShell does something similar. They print human-readable output to the
>> shell, but pass C# (I think) object instances along when piping data to other utilities.
>> The idea of separating formatting from data is good, but it also leads to insanity because
>> what you see is never what is passed on to the next utility. The utilities need to support
>> the filtering and processing you would otherwise do with awk, grep, sed etc., and you need
>> to look up two manual pages - one for the human output, and one for the C# models. Also,
>> you can only combine utilities that were meant to be combined. This is a huge loss of
>> flexibility compared to what I'm used to from UNIX.
>>
>> Has anyone given general thought to where the "Machine readable output from userland
>> utilities" idea is headed? In particular, what should happen if I stick a pipe at the end
>> of a command? Do we want utilities to understand XML/JSON for input, too? So for example
>> 'rm' understands JSON from 'ls'. Unless this was really smart thought out, only tools that
>> were prepared to work together would be able to do so, and you would need to look up which
>> utilities work together.
>>
>> Even if we leave input alone and focus on output, we might want to let sed, grep, awk and
>> friends understand JSON/XML input so we can e.g. use XPath syntax to grep for certain
>> elements in XML output, or filter certain JSON elements from the output. At least I think
>> FreeBSD should gain command-line tools that offer at a subset of the JSON and XML
>> functionality that e.g. Python offers when working with these data formats. Using simple
>> awk and grep with these formats would be regex madness.
>>
>> I've had long hours fighting with grep, awk and others to get simple scripts to behave
>> properly due to the machine-unfriendly format of some tools, so I really like the idea of
>> machine-readable output. sysctl offers some good examples of do's and dont's in this
>> regard, and I feel that the concept of machine readability needs to be thought through
>> very carefully to be exactly as awesome as when I first started grep'ing, awk'ing, sed'ing
>> and piping my way through the UNIX shell.
>>
>
> Basically the idea would be to write a simple tool that is able to
> extract using an xpath or json selector.
>
> Example (very rough code):
>
>   ifconfig --output xml | selector --format xml --path /name --path
> /name/etheraddr  | \
>      while read name ether ; do
>        echo "Interface $name has hardware address $ether" ;
>      done
>
> In all seriousness though, the real target is people writing higher
> level languages (than shell) on top of FreeBSD.  Perhaps python or ruby
> spawning a utility and then that utility making the output easy to read.
>
> One thing to note is that the output should not just be formatted but
> normalized as well.  The fact that "uptime" can emit 15 different
> formats for the uptime string is terrible for people coding on top of
> the base utils, the json/xml/other output should be decided on some form
> of normalized data likely in seconds + microseconds or something, but
> anything truly machine readable is better than the current output when
> popen'd by a webapp.
>
> -Alfred
Greetings, all.
I may be getting into this thread a bit late in the game. But if I
understand the gist of this correctly; isn't all this pretty much what
Perl was intended for?

All the best.

--Chris
>
>
>
>
>
>
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
>