sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al.

[Garrett Cooper]

On Sun, Oct 10, 2010 at 3:49 PM, Devin Teske <dteske at vicor.com> wrote:
> Trimming further context...
> On Oct 9, 2010, at 7:30 PM, Garrett Cooper wrote:

...

> Perhaps you meant env FAILURE=0 sysrc foo && reboot ?
>
> $ cat failure.sh
> #!/bin/sh
> echo "FAILURE: $FAILURE"
> $ FAILURE=0 && sh failure.sh
> FAILURE:
> $ env FAILURE=0 sh failure.sh
> FAILURE: 0
>
> Ah, beautiful. I'd been searching for a way to set an environment variable
> prior to running a command in one-swift blow. I see env(1) is handy for
> that.

And the cool thing is that it works from other shells:

$ export FOO=0; csh -c "env FOO=1 csh -c 'env | grep FOO'"
FOO=1

That's why I prefer it in command line examples (it's deterministic).

> Though honestly, the reason it's not working for you is because FAILURE has
> not been exported...

I didn't state it explicitly that way, but yes... that's what I was implying.

...

> Hence why when adding environment-variable based tunables in ~/.bashrc, it's
> best to use export (either after initial assignment or with assignment
> specified on export line in-one-go) else the assignment won't survive the
> script...

Which makes sense because you're the developer, but does it make sense
for production quality code, especially when users are better than
developers at finding code flaws, i.e. doing the following:

$ export FAILURE=a
$ exit $FAILURE
exit: Illegal number: a
$ echo $?
2

Implementation in the shell may vary (again, this was /bin/sh).

> Safe for four exceptions...
> 1. When the script itself executes the set(1) built-in with either `-a' flag
> or `-o allexport' arguments
> 2. The parent shell does the above and does not execute the child as a
> sub-shell but rather sources the child using the `.' built-in
> 3. The script itself has an invocation line resembling any of the following:
> #!/bin/sh -a
> #!/bin/sh -o allexport
> #!/usr/bin/env sh -a
> #!/usr/bin/env sh -o allexport

Hmm... forgot about that :D.

> 4. The parent shell does the above and does not execute the child as a
> sub-shell but rather sources the child using the `.' built-in.
> Which, in any of the above cases, simple assignment to a variable name will
> cause it to be exported to all children/sub-shell environments.
> Works for example in a live-shell too...
> $ unset FAILURE
> # start from a clean slate
> $ FAILURE=0 && sh failure.sh
> FAILURE:
> # Success, we're not exporting on bare assignment
> $ set -a
> # Turn on allexport in the interactive shell
> $ FAILURE=0 && sh failure.sh
> FAILURE: 0
> # Success, we're exporting on bare-assignment due to allexport
> $ set +a
> # Turn off allexport in the interactive shell
> $ FAILURE=0 && sh failure.sh
> FAILURE: 0
> # It was still exported
> $ unset FAILURE
> # Let's unexport it
> $ FAILURE=0 && sh failure.sh
> FAILURE:
> # success, no longer exported automatically via allexport feature

Part of the reason why I follow the set global once with empty values
or defaults and declare locals given the chance and if I care.
Otherwise there's too much wiggle room for surprises from the user's
environment :).

> Understood. There really isn't any degree of shell style in FreeBSD,
> but it would be nice if there was..
>
> I find that to be the case quite often when dealing with shell scripting.
> I've been trying to bring a little style to the shell scripting world these
> days ^_^
>
>
> Ah, coolness. command(1) is new to me just now ^_^
>
> Yeah.. I was looking for something 100% portable after I ran into
> issues with writing scripts for Solaris :).
>
> I went back to our legacy systems just now (FreeBSD-4.11) and tried this...
> $ uname -spr
> FreeBSD 4.11-STABLE i386
> $ /bin/sh -c "command -v '['"
> command: unknown option: -v
> Meanwhile:
> $ uname -spr
> FreeBSD 8.1-RELEASE-p1 amd64
> $ /bin/sh -c "command -v '['"
> [
> So it would appear that on FreeBSD at least, type(1) built-in is more
> portable (this perhaps traces back to it's 4.4BSDLite roots).
> I was thinking ... perhaps another flag. But alas, -p was the only valid
> option back then, which only causes a default PATH to be used rather than an
> inherited one.

Potentially. command(1) is just POSIX compatible, whereas type may be
BSD compatible *shrugs*.

...

> On legacy system:
> $ uname -spr
> FreeBSD 4.11-STABLE i386
> $ /bin/sh -c '[ "-n" ] && echo true'
> true
> $ /bin/sh -c '[ "-e" ] && echo true'
> true
> $ /bin/sh -c '[ "-z" ] && echo true'
> true
> $ /bin/sh -c '[ "-r" ] && echo true'
> true
> $ /bin/sh -c '[ "-f" ] && echo true'
> true
> $ /bin/sh -c '[ "-s" ] && echo true'
> true
> Up-to-date is the same:
> $ uname -spr
> FreeBSD 8.1-RELEASE-p1 amd64
> $ /bin/sh -c '[ "-n" ] && echo true'
> true
> $ /bin/sh -c '[ "-e" ] && echo true'
> true
> $ /bin/sh -c '[ "-z" ] && echo true'
> true
> $ /bin/sh -c '[ "-r" ] && echo true'
> true
> $ /bin/sh -c '[ "-f" ] && echo true'
> true
> $ /bin/sh -c '[ "-s" ] && echo true'
> true

Fair enough. My supposed knowledge and/or memory mislead me here :/.

...

> and the `our' keyword too ^_^

Yeah, each with their own nuances... but my is closer to local than
our is closer to local IIRC.

> Indeed. Is it weird to have "code that is itself considerate and/or kind" in
> that respect? lol

Well... code is code. The developer should understand what's going on
in the code before they copy-paste things at will, because they might
omit certain bits and introduce bugs into their code if they have no
clue what they're doing or don't understand the context that is
required to use your code.

...

> Oops! I intended to copy the message verbatim, but typo'd in the translation
> from one terminal to the next -- one of the cases where copy/paste could
> have been my friend. (realized this as I tried the same commands over again
> and got capital 'P' -- thanks)

NP.

...

> Hmmm, sysctl(9) is lock-free, which might imply that both sysctl(8) and
> sysctl(3) are also lock-free, and proposed sysrc(8) is lock-free, so might
> that imply that the atomicity tests would fare the same for all of the
> above?

.../sys/kern/kern_sysctl.c says otherwise (look for the comment above
the sysctllock declaration). The locking is just hidden from the
developer/end-user.

> Here's what I'm thinking we should do to solve this...
> Since the atomicity of the write operation is anything-but singular
> (meaning, it takes incrementally larger blocks of time to write larger
> amounts of data, increasing the risk-curve for failure to occur by two
> operations coinciding at the same time -- I'm truly sorry, my wife has me
> helping her with her business statistics II course, forgive me, I'll
> clarify).

...

I think I said it before, but yes.. I completely agree with the
atomicity approach. I prefer `mktemp /tmp/XXXXXX' because it would do
nothing more than potentially clutter up /tmp if the operation fails
for whatever reason (instead of /etc), and it's less of a security
concern. I suppose that's less of a problem though, because if someone
has the ability to write to /etc, then all bets are off, but the
clutter part is a bit annoying..

...

> Shouldn't be that hard.

Your approach seems reasonable.

...

> Last, but not least...
> Not sure why, but...
> $ uname -spr
> FreeBSD 4.11-STABLE i386
> $ /bin/sh -c 'echo OPTIND: $OPTIND; unset OPTIND; echo failed'
> OPTIND: 1
> unset: Illegal number:
> $ uname -spr
> FreeBSD 8.1-RELEASE-p1 amd64
> $ /bin/sh -c 'echo OPTIND: $OPTIND; unset OPTIND; echo failed'
> OPTIND: 1
> unset: Illegal number:

Hmmm... that seems like a shell bug to me, in particular that it would
always declare that value (esp when POSIX makes no mention of the
variable being that way, and the manpage for sh doesn't mention that
either). I'd have to ask jilles@ about that.

$ sh -c 'echo OPTIND: $OPTIND; unset OPTIND'
OPTIND: 1
unset: Illegal number:
$ sh -c 'echo "OPTIND: $OPTIND"; unset OPTIND'
OPTIND: 1
unset: Illegal number:
$ bash -c 'echo "OPTIND: $OPTIND"; unset OPTIND'
OPTIND: 1
$ env optind=1 bash -c 'echo "optind: $optind"; unset optind'
optind: 1

...

> Nah... the purpose of sysrc(8) would be to print, munge, check, verify, etc.
> what source_rc_confs sees at boot time -- so in-turn being a tool to
> administer how those very rc.d scripts you speak of act at boot-time...
> Because most all of the rc.d scripts I've seen (in `/etc/rc.d',
> `/usr/local/etc/rc.d', and any other directories that are configured in the
> `local_startup' option within rc.conf(5)).
> Generally speaking though, things in these rc.d directories have two
> formats:
> They either end in `.sh' or they don't.
> The ones that end in `.sh' are old-style and tend to ignore `/etc/rc.subr'.
> The old-style scripts are passed in `start' at boot-time and they often use
> case/esac to switch on the first positional argument ($1).
> The ones that don't end in `.sh' are new-style and tend to source
> `/etc/rc.subr'. New-style scripts are expected to adhere to rcorder(8)
> markups placed in the script as strategic comments (preceded with `#') (see
> rc(8) for even more info).
> The new-style functions are not passed a parameter at startup, rather the
> ${start_cmd}() function is called by /etc/rc at boot-time when [
> "${name}_enable" = YES ] in rc.conf(5).
> These scripts are more sophisticated and more modular in that they rely on
> shared-code in /etc/rc.subr
> But, in the end-run, the new-style scripts tend to all run load_rc_config
> from /etc/rc.subr which ends up calling source_rc_confs anyway (among other
> things -- such as sourcing /etc/rc.conf.d/service_name, but we talked about
> sysrc(8) not having context of the service_name that load_rc_conf() does,
> though we could add some flag to cause sysrc(8) to emulate the startup of an
> rc.d script by-name which would follow the same actions performed by
> load_rc_conf, allowing the user of sysrc(8) to see additional values
> configured in /etc/rc.conf.d/service_name given the additional context).

Part of the reason why I brought it up was to serve an additional
purpose as a means to configure a system for newbies (something randi@
asked me to look into a while back that kind of fizzled out due to
lack of interest).

> I think in a previous e-mail we talked about re-mapping `-d' from dependency
> to be more like sysctl(8)'s `-d' which is akin to "describe".
> All we'd have to do is find the line in /etc/defaults/rc.conf and return
> "cat /etc/defaults/rc.conf | grep "[[:space:]]*$var=" | tail -1 | sed -e
> 's/.*#[[:space:]]\{1,\}//'" ... (finds last line that matches a
> bare-assignment of the variable by-name, trimming up-to first '#' including
> all additional white-space following first '#', retaining what's left up to
> the end of the line).
> Admittedly, that's a poor-excuse of a parser, since it will miss multi-line
> comments. A full-on sub-shell would be much better (but I'll leave that for
> the final script which will hit the e-mail in another thread).

Sounds reasonable (and I didn't realize that sysctl's -d option did
that -- thanks!).

...

I would just hold to this statement in /etc/defaults/rc.conf:

# All arguments must be in double or single quotes.

and also state:

"this tool functions based on the idea that the rc.conf files are
simply written, and can be evaluated as standalone configuration data.
Anything that doesn't conform to these requirements isn't guaranteed
to work with the tool in all cases"

Thanks!
-Garrett