sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf
et. al.
Devin Teske
dteske at vicor.com
Fri Oct 8 00:53:47 UTC 2010
On Thu, 2010-10-07 at 17:36 +0200, Stefan Esser wrote:
> Am 07.10.2010 05:29, schrieb Devin Teske:
> >
> > On Oct 6, 2010, at 4:09 PM, Brandon Gooch wrote:
> >
> >> On Wed, Oct 6, 2010 at 3:45 PM, Devin Teske <dteske at vicor.com> wrote:
> >>> Hello fellow freebsd-hackers,
> >>>
> >>> Long-time hacker, first-time poster.
> >>>
> >>> I'd like to share a shell script that I wrote for FreeBSD system
> >>> administration.
> >>>
> >>
> >> It seems the list ate the attachment :(
> >
> >
> > Here she is ^_^ Comments welcome.
>
> Hi Devin,
>
> I think your script is quite useful in specific situations, but getting
> it into FreeBSD may prove harder than you might have expected.
>
> But it should be easy to get it into the ports tree, if it proves
> useful it may later make it into a new release (and be merged back) ...
I'm willing to see that through to fruition. Should be fun.
> Since I'm travelling with no access to my FreeBSD systems, I can not
> verify all details in my reply, but I hope it is still useful for you.
>
> Let me start with a few remarks:
>
> Your script is really useful for querying of rc.conf (et al.) settings,
> less so for modifications to the files (IMHO). The reasons are many:
>
> 1) Changes to rc.conf do not take effect, immediately (different from
> sysctl(1) settings).
Funny that you would mention that.
How this script came-to-be is that I first developed the internal
functions (sysrc() and sysrc_set()) for a larger script called "host-
setup" which I'm programming for our company to allow field-engineers to
setup their system using /usr/bin/dialog (dialog(1)).
This larger script uses the internal sysrc() function to query values
before presenting them to the user.
Conversely, when the user invokes the editor to change the value, the
script does the following -- say, for example, in changing the hostname:
1. It uses sysrc_set() to change the value in the rc.conf (or
rc.conf.local -- or any other file mentioned in $rc_conf_files
within /etc/defaults/rc.conf)
2. It then re-reads the value from the system configuration files.
3. Calls hostname(1) with the new value to make it effective without a
reboot.
So, in reality, this script (sysrc(8)) could be used by other
scripts/programs in much the same way I use it -- as a way to munge the
rc.conf(5) files *in addition to using standard utilities such as route
(8), hostname(1), ifconfig(8) and many others.
I agree that teaching this tool how to make changes effective without a
reboot is neither warranted nor desired. Since each directive that can
be munged by this utility indeed pertains to a different subsystem (some
modify the behavior of /etc/rc, some modify the behavior
or /etc/rc.sendmail, some /etc/rc.firewall, some /etc/rc. some modify
the behaviour of /etc/rc.d/*), it's beyond the scope of this utility to
know what to do with the given setting once changed to make it effective
without a reboot.
However, I will say that in a large-scale deployment, simply having an
ability to do the following is still beneficial:
for host in host1 host2 host3 host4 host5 ...; do
ssh $host /bin/sh -c "'sysrc ntpdate_flags="..." && reboot'"
done
or, if you really know your shell-foo:
for host in host1 host2 host3 host4 host5 ...; do
ssh $host /bin/sh -c "'sysrc ntpdate_flags="..." && service ntpdate
stop && service ntpdate start'"
done
> 2) Changes to rc.conf are persistent and need not be applied on each
> reboot (similar to changes to sysctl.conf, but different from
> sysctl(1))
I'm not sure what you mean by "changes ... are persistent", but yes, I
do agree that the average lay person is convinced that a reboot is
required to make any change to the rc.conf files effective (however, you
and I perhaps know better).
It should also be noted that sysctl.conf(5) does not require a reboot to
re-apply the settings contained within. These days, one can run the
following command to reload sysctl.conf(5):
service sysctl reload
Conversely, for a specific example of rc.conf(5) being read continuously
after boot, we often have to restart apache, and we do-so like this:
/usr/local/etc/rc.d/httpd restart
The httpd script in /usr/local/etc/rc.d utilizes all the usual suspects
in rc(8) rcorder(8) and `/etc/rc.subr'.
In fact, just about every script in either /etc/rc.d
or /usr/local/etc/rc.d will re-read /etc/rc.conf when run. No reboot
required, just munge the file and run your rc(8) script either directly
or via service(8).
> 3) While there is a specified order of evaluation when reading from
> rc.conf files, it is not obvious that writing to the last file in
> $rc_conf_files that defined some parameter (or the first one, if
> the parameter is not currently defined) is the right thing to do.
> (E.g. if there is a rc.conf and rc.conf.local, it may well be the
> latter that should receive new parameter definitions.)
I've have previous incarnations of these utilities that DID go through
each and every file listed in rc_conf_files within /etc/defaults/rc.conf
and clean-up ALL instances of the variable, however having been bitten
by that, I think that it's better to replace only the last occurrence
(the one that takes effect when source_rc_confs is called).
We have at certain sites set up SUP (the Software Update Protocol -- the
precursor to cvsup and csup in many ways) and configured cron-jobs to
SUP certain files off of a central server. One of those files
is /etc/rc.conf
So what we would do, is to have all the configuration settings that
apply to a given site put into /etc/rc.conf and all the settings that
pertain to the local machine into /etc/rc.conf.local
What this achieves is that -- since /etc/rc.conf is automatically shared
among all machines in a given LAN -- we can plunk-down settings that we
feel should apply to all machines in the site (like ntpdate_flags --
pointing ntpdate to a given server, allowing all machines in the site to
sync up to a single time server), meanwhile, if a single machine needs
to sync up with a different time-server, we can have ntpdate_flags in
the local machines /etc/rc.conf.local override the value provided
in /etc/rc.conf
If we had sysrc munge ALL files rather than the last one, then we'd be
breaking the setup scheme where /etc/rc.conf is to not be modified by-
hand.
>
> 4) I seem to remember that there was support for a rc.conf.d directory,
> but I'm not sure and cannot easily check this, right now.
Yes, /etc/rc.subr in the load_rc_config() function will source a file
according to the rc(8) script name (passed in as the first argument of
load_rc_config()).
Unfortunately, I think that the lack of context in the case of sysrc
makes it impractical to go searching through all /etc/rc.conf.d/* files
for some configuration value. sysrc isn't passed a name of a service the
way that load_rc_config is.
Though I agree that the rc.conf.d scripts shouldn't be neglected. For
example, if /etc/rc.conf/httpd help the following text:
httpd_enable="YES"
And a user wanted to either query or change this setting, currently the
sysrc script will say:
$ sysrc httpd_enable
sysrc: unknown variable 'httpd_enable'
Because /etc/rc.conf.d isn't checked (as should be expected because
we're not given a context of a service name).
Though, perhaps via some "-f file" option to sysrc, we can bridge the
gap...
$ sysrc -f /etc/rc.conf.d httpd_enable
httpd_enable: YES
Or, prehaps:
$ sysrc -n httpd httpd_enable
httpd_enable: YES
(envisioning "-n name" to cause the script to source /etc/rc.conf.d/name
after sourcing the gammut of $rc_conf_files via source_rc_confs()
in /etc/defaults/rc.conf).
What do you think? I like both ideas.
> Some suggestions in random order:
>
> a) Allow the user to specify files to operate on (with the defaults
> set as in your current script). This would allow to edit sysctl.conf
> (which has the same syntax as rc.conf) with your script, among other
> things (e.g. to specify the rc.conf file to write to).
I like that idea.
I'm also thinking perhaps of adding a chroot(8) mechanism which would be
useful for working in jails, however that may not be necessary since we
now have jexec(8) these days. One could simply do:
jexec JID sysrc hostname=foo
jexec JID /bin/sh -c 'hostname "$( sysrc hostname )"'
The first command performs the sysrc command within the jail, causing
the rc.conf(5) files to be munged.
The second command makes the change effective.
> b) Allow the user to dump multiple parameters selected by a pattern or
> all parameters (support for a wildcard and "-a" option to sysctl).
> I understand that patterns and the shell variable expansion options
> that can be used in parameter names (e.g. 'hostname%%.*' given as
> example in your script) may conflict, but it would still be very
> useful, for example to print all "ifconfig_*" values).
I really thought long and hard about this one. I started out wanting it,
but then changed my mind.
In truth, because I was mirroring sysctl(8), I thought it would be
rather pleasant to have a "-a" flag that dumped all the non-default
values.
This is very doable. I just had to sleep on it awhile to determine what
I wanted to do (show values for each file? or show overall end-result
values perloined by sourcing all of the files in a row, dumping only the
effective values of each/every variable, thus producing a comprehensive
list of directives that the system will see upon boot-up (in-turn
providing the system administrator with a quick snapshot of how the
system will boot -- both valuable for problem trouble shooting and also
for double-checking your work before you reboot).
What do you think?
> c) Let user print comments for parameters (possibly selected by pattern)
> as present in the defaults/rc.conf file. This should for example
> allow to print the descriptions for "*ip6*" to help identify the
> parameter to modify. (I think the "-d" option of sysctl dumps the
> descriptions instead of or together with the values selected.)
I really really like that idea! This brings us even closer to the
operation of sysctl(8).
This alone could make this script double it's current value (by giving
us all a faster way to research possible directives that can be thrown
into the rc_conf_files).
I envision people using -d to quickly find out of a value is valid and
to get a quick description of what it does and also show the default
value.
Simply displaying the default value is useful to even the most seasoned
veterans because it answers the question: "do I need to customize the
default value?" as quickly as possible.
Of course, `-d' is currently used for remediating external dependencies
out of the script (passing `-d' will cause the script to dump a list of
internal dependencies upon exit-for-any-reason, producing a table of
barewords that are used within the script and their associative meanings
-- such as "shell keyword", "shell built-in", "shell function", "tracked
alias' to external executables" etc.).
> d) Add an option to report the file(s) where parameters are defined.
> In fact, there are three cases of interest: 1) all files that set
> the parameter, 2) the file from which the value will be used (the
> last one from the list returned by 1) I guess), and 3) the file
> that would be modified if the value was to be written.
I really wanted to option too, but didn't want to delay the release of
this script to the public domain any further.
I'm envisioning a `-v' parameter for verbosity as well as an inheritable
environment setting SYSRC_VERBOSE to do the same -- enable the logging
of which file was ultimately queried for the answer or which file was
ultimately munged in a set-request (via `name=value' syntax).
> For a script imported as a port, there is no need for "have" and
> "depend", since dependencies will be resolved when installing the
> port (and you only use base functionality, anyway). I understand that
> there helper functions make maintenance and porting to other systems
> easier, but I'm quite sure they will not be accepted if the script is
> ever to be imported into FreeBSD itself.
I agree fully, 100%. The have(), depend(), and show_deps() functions in
conjunction with the _depend, SYSRC_SHOW_DEPS, and other related
variables have no home if this script makes it into the FreeBSD
distribution.
However, what these scripts provide right now is the following:
a. I find it very useful both to myself and to less-experienced shell
developers to have a call to depend() listing all dependencies for the
following block of code prior to the actual execution of said block
BECAUSE it helps readers know what is required should they want to rip
out that block for their own use in their own code in their own shell
script. For example, many times when I write shell scripts that
accomplish complex tasks using low-level shell features, there is often
people that desire to re-use a handy shell function but often forget to
grab some other dependent function defined further up or farther below.
The depend() syntax helps remind folks what is required for each block
of code.
b. Help determine whether a given environment is suitable for the script
by first and foremost providing human-readable errors that give the user
a succinct and disambiguous error message -- as we all know that
sometimes a simple error of "grepfoo: command not found" is ultimately
helpful, but when you have a multi-hundred line shell script it's handy
to know who was trying to use `grepfoo' (the first argument of the
depend function gives the user that clue by issuing instead error
statements like "Missing dependency 'grepfoo' required by mygrep" --
developer knows that the `mygrep' shell function either needs to use
have() -- aka type -- to work around cases where grepfoo may not be a
valid command). This is invaluable when programming for embedded
environments such as the crunchgen(1)'d mfsroot boot ``floppy image''
generated by the release(7) process documented in the FreeBSD Release
Engineering process detailed at
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/release-
build.html ).
c. These functions actually allow this script to run on foreign
operating systems. For example, one can run this script on Linux. It
will likely *ALWAYS* say "unknown variable '...'"
since /etc/defaults/rc.conf doesn't exist on any OS except BSD
derivatives of a specific flavor.
Naturally... all of the above-mentioned benefits are nullified if/when
sysrc gets slurped into the the base distribution.
I have no qualms with judiciously removing all the depend(), have() and
show_deps() stuff if it seems like this script has a shot at making it
into the base distribution.
Though, ultimately, I envisioned the sysrc() and sysrc_set() functions
being slurped into /etc/rc.subr where they can be used by rc(8) scripts
that naturally source /etc/rc.subr to get access to beauties such as
load_rc_config(). After-which the sysrc(8) script would be reprogrammed
to (in a FreeBSD fashion) source /etc/rc.subr and then use the functions
as defined in there.
> In either case, the comments relevant for usage of the script (e.g.
> those below "Command Usage", "usage" and "sysrc $setting") should be
> extracted into a man page. The rest of the commands and the portability
> support would probably removed after an import into FreeBSD (keeping
> them in the "vendor import" files, but none of the versions that are
> extracted from the repository and used in FreeBSD).
I do agree that the comments are ... excessive =)
My desire was to get out the initial version of the script to the public
domain (in 1.0-version fashion) in it's primordial state. I left all
that stuff in there (the copious comments, the helper functions -- have
(), depend(), and show_deps() -- etc.) to be a beacon of shining glory
to budding shell-scripters.
Though, in contrast to the agreed immediate removal of the helper
functions, it would indeed be a shame to carte-blanche remove the
copious documentation embedded within.
You are right that the inline documentation really belongs in a *roff-
formatted man-page, but I hadn't considered "vendor import" files as
another possible destination.
> I could perform the import of "sysrc" as a port, but I guess there
> will be quite some discussion during the next few days and there
> might be others interested in being maintainer of such port or there
> might even be consensus to import it into the source tree ...
I'm grateful for the offer. I've not yet been blessed (or should I say
-- cursed) with the direct commit privileges, but am working on it (by
first submitting things -- I'm told by many friends whom _do_ have
commit privs that the process is essentially thusly stated, "find
someone willing to commit code on your behalf and when they grow tired
of managing your code, they'll curse you with direct commit privileges"
^_^, do you remember telling me that Julian? lol).
> Best regards, STefan
>
> PS: This reply is not sent to the list, but you are free to reply
> to the list and quote my message.
I think there's enough conversation-worthy notes here that it's worth
CC'ing the list back-in. Thanks for giving me the heads up that it's ok
(I would have asked first if you hadn't).
P.S. I think it's safe to say I chose the right mailing-list to post
this stuff to (I scoured the list and read all the charters before
chosing which list to burden with this script). So far so good ^_^
--
Cheers,
Devin Teske
-> CONTACT INFORMATION <-
Business Solutions Consultant II
FIS - fisglobal.com
510-735-5650 Mobile
510-621-2038 Office
510-621-2020 Office Fax
909-477-4578 Home/Fax
devin.teske at fisglobal.com
-> LEGAL DISCLAIMER <-
This message contains confidential and proprietary information
of the sender, and is intended only for the person(s) to whom it
is addressed. Any use, distribution, copying or disclosure by any
other person is strictly prohibited. If you have received this
message in error, please notify the e-mail sender immediately,
and delete the original message without making a copy.
-> FUN STUFF <-
-----BEGIN GEEK CODE BLOCK-----
Version 3.1
GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O
M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h
r>++ y+
------END GEEK CODE BLOCK------
http://www.geekcode.com/
-> END TRANSMISSION <-
More information about the freebsd-hackers
mailing list