CBOR (Was: My experiences with Rust)

On Mon, 25 Aug 2025 09:03:51 -0400
"Isaac (.ike) Levy" <ike@blackskyresearch.net> wrote:

> If your parser pipeline has an absolute need to not miss things like "fatal:
> dropping all", one could simply look for the string "drop" not "dropped",
> 
> Given this output,
> 
> --
> foo dropped 1/2 foo
> bar sortof double! dropped bar
> baz dropped 2/2 baz
> fatal: dropping all
> bang sortof last bang
> --
> 
> $ myprogram | grep drop
> foo dropped 1/2 foo
> bar sortof double! dropped bar
> baz dropped 2/2 baz
> fatal: dropping all
> $ 
> 
> Caught it!
> 
> 
> Now, if the original program doesn't give enough output, (messages like
> "fatal: dropping all"), that could be worth investigating in the upstream
> program.
> 
> Next steps?
> Putting on my sysadmin hat, as a user, I may investigate/replicate the
> event, perhaps read the man page, read the source code, and try to
> understand why this is happening.  Perhaps the program doing something
> performance constrained and can't possibly output more?  Perhaps the program
> simply being lazy in output, perhaps begging for improvement? Is this output
> a wrapper shell/other buffer issue, not even the program at all? Is this
> case something which happens so rarely, and is so expensive to improve- or
> so difficult to maintain long-term, that it's cleaner to just anticipate the
> output?
> 
> If the output is not in my power to change, yet my need still exists to
> catch it reliably, we all have an infinitude of tools and languages to solve
> output structure problems to satisfaction.
> 
> Again, in any sufficiently complex system, ideal coordination between
> systems and subsystems is not possible.  Good software starts with trusting
> programs to do what they say they'll do, great software starts with a
> healthy mistrust programs will fail, and anticipating common failures which
> affect the important parts of your using them.
> 
> --
> One last thought back to PHK's original point:
> 
> If the output is in CBOR format, the entire problem above still exists, but
> now *requires* specific programs to read and understand, before a user or
> program can begin to understand it.

It's not harder than adding "| bunzip2" to you pipeline, and then
understanding structured output is much simpler than yours
> bar sortof double!
> dropped bar
> baz dropped 2/2 baz

..which is yet something not understandable == still to be translated to JSON
by you.

-- 
WBR, @nuclight