sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf
et. al.
Devin Teske
devin.teske at fisglobal.com
Mon Oct 25 18:02:14 UTC 2010
On Sat, 2010-10-23 at 15:20 +0400, Anthony Pankov wrote:
> Greetings.
>
> Just for note.
>
> Some time ago i thought about the same problem.
> I started something then had delayed it forever in favour of fast wrong
> way.
>
> So, the aim was:
>
> ---
> NAME
> modcfg - modify configuration
>
> SYNOPSIS
> modcfg -f config_file -t config_type {-s param=val | -u param}
> modcfg -l
> modcfg -i plugin
>
> DESCRIPTION
> The modcfg utility modifies configuration file in
> accordance to parameters.
>
> The following options are available:
> -f config_file - the file itself which to be modified
> -t config_type - type of config_file. Type specifies the internal
> structure of config_file, or, more roughly,
> program which this file belongs to. To
> list available types see -l option.
> -s param=val - set configuration parameter 'param' to value
> 'val'.
> -u param - unset configuration parameter 'param' (or set it to default).
>
> -l - list all supported types of config files.
>
> -i plugin - install plugin 'plugin' for modcfg utility.
> Plugin is used to support additional configuration file type.
>
> EXAMPLES
> The command
> modcfg -f /etc/rc.conf -t rc -s keymap=ru.cp1251
> sets parameter 'keymap' to value 'ru.cp1251'in file rc.conf.
>
> The command
> modcfg -f /etc/rc.conf -t rc -u keymap
> resets parameter 'keymap' to default value.
I think it's great that you've taken on the task of finding a solution
that works best for you, but this doesn't really work that well for the
case of rc.conf(5) files because it neither sources
`/etc/defaults/rc.conf' nor calls source_rc_confs() from said-file.
So yes, one could modify a variable in a given file with your utility,
but it doesn't take into account that rc.conf(5) is not a single file,
it's a collection of files. Least of which contains:
/etc/defaults/rc.conf
/etc/rc.conf
/etc/rc.conf.local
/etc/rc.conf.d/*
NOTE: The last one is not sourced by source_rc_confs() but is sourced
by /etc/rc.d/* (the local-service boot-scripts), most [if not all] of
which call load_rc_config(). In example, /etc/rc.d/foo would first
source /etc/rc.subr then call `load_rc_config foo' which will both call
source_rc_confs() from /etc/defaults/rc.conf AND
source /etc/rc.conf.d/foo [if it exists].
This heirarchical structure forces a level of complexity onto the user
which _can_ be confusing at times to less experienced users, and
therefore providing a utility that ignores said-heirarchy forces the
user to either navigate the complexity themselves or to write a wrapper
script around your utility to manage it for them.
Whereas, I started my utility (sysrc) from the ground-up with the
express purpose of wrangling this complexity. I wanted a tool that would
-- overall -- help me in my daily routines of managing thousands of
FreeBSD workstations/pedestals/servers (fyi: a pedestal is server-class
hardware in workstation-tower-like chassis).
> To install obtained plugin 'samba.mcfg' use
> modcfg -i samba.mcfg
> After that configuration type 'samba' will be supported.
Out of curiosity, why must the user be forced to "install the plugin"
manually? Why not have the code automatically probe for the plugin on
first-use? For example, when 'samba'-type is used for the first time,
take that as a queue to load the module and if the module can't be
loaded, die with an error. I think this streamlines the process (but
wouldn't go as far as to remove the `-i' parameter -- keep it, it could
be useful to some).
>
> ---
>
> INTERNALLY
>
> modcfg itself is very simple. It parse command line options, then load
> plugin (module) for specified 'config_type', then call _set(file,param,
> value) or _unset(file,param) function from this module.
> So, plugin (module) should have functions such as:
> _set(file, param, value) or, better name, {$type}_set. rc_set, for
> example.
> _unset(file, param)
> _description - for display in module list.
Your utility I think loosely resembles Apple's Core Data Programming
API.
I recently have been making my way through this Mac/iOS API known as
"Core Data Programming" which uses relationship mapping (similar to your
`config_type') to teach the Core Data API how modify different
underlying data containers (but extends far beyond BSD-style
configuration files).
For example, Core Data -- in an object-oriented fashion -- provides a
set of high-level access routines for accessing your data meanwhile the
specifics of how that data was obtained or how said data is persistently
stored back to the container are hidden from your application (much in
the way that your utility masks from the user how the data is stored in
the specific configuration-file type).
Much like your utility, Core Data already supports some well-known data
containers -- SQLite, Boulder/IO (simple key=value pairs), etc. -- and
can easily be extended through OOP's multiple-inheritance, class-method
overrides, and abstract sub-classing.
However, Core Data is not limited to working on just files (or
databases). Core Data is useful for literally anything where there is
data in one format that needs to be parsed into another format and then
optionally modified and then stored back in the original format.
For example, you could write a set of Core Data classes that can read
information from the Internet via a socket, present the objects (after
first being transparently fetched and parsed) to the calling-code, allow
the calling-code to modify the data, and then have the same underlying
classes transparently submit the modified data back to the Internet. The
end-result is that the code interfacing to the Core Data API has no idea
whether the objects presented to it via the ``Core Data Object Model''
came from the local disk, the Internet, a database, an Apache-style
config, a BSD-style config, NFS, or anything else. This leaves your
application free to change the underlying data structure of how you
store/distribute/retrieve the data without having to modify the code
that ultimately presents that data to the user (for either viewing or
modifying).
Your utility loosely reminds me of this in that your utility leaves the
end-user blissfully unaware of the underlying configuration-file
structure (nice).
> FUTURE
> Of course, better way is to run modcfg as:
> modcfg rc {-s ... | -u ...}
> i.e. reduce "-f /etc/rc.conf -t rc" to the key "rc",
> i.e. add another call scheme
> modcfg subsystem {-s ... | -u ...}
>
> For rc it is trivial. But in general, for all installed programs and
> they instances, it is not possible now.
Does your program take into consideration that the rc.conf(5) files are
actually shell scripts?
They are not Boulder/IO (key=value pairs -- one per-line), or any other
similar format. Rather, they are sourced (executed within the same
contextual namespace as the parent) shell scripts.
When I rewrite my sysrc(8) utility in C, I'll likely be using a parser
that I wrote which utilizes the same parsing logic as the POSIX bourne-
shell itself. That way, I'm guaranteed to treat these files like what
they really are -- shell scripts without an invocation-line -- so that I
can catch things like this:
hostname=foo; sshd_enable=YES; nfs_client_enable=YES
(and that's only the beginning)
The logical parser will need to know things like:
a. When the state-machine is in the "bare-word space"
b. How to treat leading bare-words containing an assignment differently
than trailing bare-words containing an assignment
c. How to differentiate between different read-states when:
1. within single-quotes
2. within double-quotes
3. within a sub-shell block ($(...), `...`, (...), etc)
d. How to interpret nested blocks
A suitably-advanced parser capable of doing the right things in all
circumstances invariably will need to make use of a LIFO queue (a
"stack") to accurately handle nested elements.
Thankfully however, we don't need to implement 100% of the POSIX bourne-
shell interpreter because in the case of modifying rc.conf(5) files, we
really only need to modify assignment operations happening within the
bare-word namespace (the top-level namespace opposed to assignments
taking place within sub-shells; assignments within sub-shells cannot
filter back-up the chain to the calling parent, so we won't care about
most all nesting cases).
In example, we care about this assignment:
hostname=foo
but not this one:
(
hostname=foo
)
nor this one:
ignored=$( hostname=foo )
nor this one:
: ${hostname:=foo}
NOTE: OK, yes that one is in the proper namespace that we care about,
but I'm deeming this one as inconsequential because if someone so-
desires such a line in their rc.conf(5) file, I'm willing to bet that
they also don't want some program blowing away such a line --
considering that it's not really a proper assignment since the
assignment only occurs in the event that hostname is null or unset
(which depends on the state of /etc/defaults/rc.conf).
nor would we care about further obscure assignments, such as:
(
ignored="`/bin/sh -c \"hostname=foobar\"`"
)
NOTE: That's a sub-shell within a sub-shell within a sub-shell (3
namespaces away from parent/bare-word namespace).
Although, maybe we should care about some of the few odd-ball cases
where assignments don't look like normal assignments... such as:
env hostname=foo
However... this is easy because the state-machine will be able to track
these, both via our LIFO/stack and by way of checking for when:
a. the leading bare-word expands to `env'
NOTE: Here, `expands' means not that I intend to perform all the same
expansions that the bourne-shell performs, such as parameter expansion,
but at least quote expansion
b. while being in the top-level namespace
And even with all that, we won't catch ALL the possible cases...
For example, we'll still fail to catch the following assignment:
FOO=env
$FOO hostname=bar
Which is perfectly valid, since when source_rc_confs() is called at
boot-time (by way of /etc/rc called by /sbin/init invoked by the kernel)
out of /etc/defaults/rc.conf, the /etc/rc.conf file is essentially
executed by /bin/sh, and the above two lines will produce the
environment variable:
hostname=bar
in-turn causing the later stages to read said environment variable and
set the ensuing hostname.
My sysrc(8) script -- as you can see -- has quite the goal... handle all
but the most esoteric cases (such as bare-word abstraction demostrated
above ala "FOO=env").
>
>
>
> Wednesday, October 13, 2010, 3:06:25 AM, you wrote:
>
> DT> On Tue, 2010-10-12 at 12:10 -0700, Doug Barton wrote:
> >> On 2010-10-11 at 10:40 -0700, Doug Barton wrote:
> >> | So to summarize, the general idea is a good one and needed, and an area
> >> | that I'd like to see more work in. Perhaps it might be a good idea to
> >> | move the discussion about that to freebsd-rc@?
> >> |
> >> |On 2010-10-11 at 12:22 -0700, Devin Teske wrote:
> >> |> I'll look into signing up for the rc mailing list (didn't see that when
> >> |> I checked last -- I'll have to look again). Maybe I'll post v2.0 to
> >> |> there (but will cc back hackers cause I know folks may not be part of
> >> |> both).
> >>
> >> The canonical way to deal with that is to post the message to the proper
> >> list (-rc@), then post a brief note to the other list (-hackers@) saying
> >> where the discussion is being continued. We discourage people from
> >> cc'ing multiple FreeBSD lists.
>
>
> DT> This thread is moving over to the -rc@ list.
> DT> New thread: sysrc(8) -- a sysctl(8)-like utility for managing rc.conf(5)
>
> DT> The first post to the -rc@ list will be version 2.0 of the script which
> DT> attempts to address the following (which were discussed in this thread
> DT> here on -hackers@):
>
> DT> 1. Style -- remove some personal styles in favor of standardized styles.
> DT> (the FreeBSD environment doesn't need all the extra things that are
> DT> required to run in an embedded environment -- which the first version
> DT> was coded for)
>
> DT> 2. Remove a disgusting-amount of comments (the first release of the
> DT> script had a hurdle to climb in that it had to establish rapport with
> DT> the targeted audience -- y'all).
>
> DT> 3. Remove shell inheritance of SUCCESS/FAILURE (this was silly for a
> DT> finished product).
>
> DT> 4. Remove unnecessary code-sense (some things just don't need to be
> DT> tested for in a known environment -- such as FreeBSD vs. embedded).
>
> DT> 5. Remove dependency checks (have(), depend(), and show_deps() are
> DT> gone).
>
> DT> 6. Remove fake "function" keywords (public objections win)
>
> DT> 7. Rename sysrc() function to sysrc_get() to:
> DT> a. prevent confusion between the script and the internal function
> DT> b. to coincide with the remainder of related functions (sysrc_get,
> DT> sysrc_set, sysrc_find, and sysrc_desc).
>
> DT> 8. Fix sysrc_get() function to mask positional parameters (don't expand
> DT> "1", "2", etc.)
>
> DT> 9. Fix sysrc_get() function to clean the environment prior to sourcing
> DT> rc.conf(5) files (preventing expansion of normals such as PS1, TERM,
> DT> etc.)
>
> DT> 10. New function: `sysrc_find $varname'
> DT> Find which file holds the effective last-assignment to a given variable
> DT> within the rc.conf(5) file(s). If the variable is found in any of the
> DT> rc.conf(5) files, the function prints the filename it was found in and
> DT> then returns success. Otherwise output is NULL and the function returns
> DT> with error status.
>
> DT> 11. Fix sysrc_set() function to use mktemp(1) (prevent race-conditions
> DT> where sysrc(8) could be executing in concurrence, possibly whacking the
> DT> output-file in an unexpected manner).
>
> DT> 12. New function: `sysrc_desc $varname'
> DT> Attempts to return the comments associated with varname from the rc.conf
> DT> (5) defaults file `/etc/defaults/rc.conf' (or whatever RC_DEFAULTS
> DT> points to). Multi-line comments are joined together. Results are NULL if
> DT> no description could be found.
>
> DT> 13. Use getopts(1) to parse command-line options rather than manually
> DT> parsing (now we can support grouping of flags -- i.e. "-avN").
>
> DT> 14. Remove `--help' option (using getopts(1) now ... that was the only
> DT> long-option we had, and we don't need it).
>
> DT> 15. Remove `-d' as we know it. No longer dump internal dependency list,
> DT> but mimick `-d' from sysctl(8) -- Print a description of the given
> DT> variable.
>
> DT> 16. Remove `SYSRC_SHOW_DEPS' environment variable.
>
> DT> 17. Add `SYSRC_VERBOSE' environment variable (inheritable from the
> DT> shell, so that folks whom don't want to always pass `-v' can plop
> DT> `SYSRC_VERBOSE=1' into their shell startup scripts, `~/.bash_profile'
> DT> and `~/.profile' for example).
>
> DT> 18. Add `-f file' option.
> DT> Operate on the specified file(s) instead of rc_conf_files.
>
> DT> 19. Add `-a' option.
> DT> Dump a list of non-default configuration variables.
>
> DT> 20. Add `-A' option.
> DT> Dump a list of all configuration variables (incl. defaults).
>
> DT> 21. Add `-v' option.
> DT> Verbose. Print the pathname of the specific rc.conf(5) file where the
> DT> directive was found.
>
> DT> 22. Add `-i' option.
> DT> Ignore unknown variables.
>
> DT> 23. Add `-N' option.
> DT> Show only variable names, not their values.
>
>
>
> DT> And, here's the new usage:
>
> DT> Usage: sysrc [OPTIONS] name[=value] ...
> DT> OPTIONS:
> DT> -h Print this message to stderr and exit.
> DT> -f file Operate on the specified file(s) instead of rc_conf_files.
> DT> -a Dump a list of non-default configuration variables.
> DT> -A Dump a list of all configuration variables (incl. defaults).
> DT> -d Print a description of the given variable.
> DT> -e Print query results as `var=value' (useful for producing
> DT> output to be fed back in). Ignored if -n is specified.
> DT> -v Verbose. Print the pathname of the specific rc.conf(5)
> DT> file where the directive was found.
> DT> -i Ignore unknown variables.
> DT> -n Show only variable values, not their names.
> DT> -N Show only variable names, not their values.
>
> DT> ENVIRONMENT:
> DT> RC_DEFAULTS Location of `/etc/defaults/rc.conf' file.
> DT> SYSRC_VERBOSE Default verbosity. Set to non-NULL to enable.
>
>
> DT> See you all on the -rc@ list.
>
>
>
> --
> Best regards,
> Anthony mailto:ap00 at mail.ru
>
>
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
More information about the freebsd-hackers
mailing list