rc(8) parallel tasks
Mark Martinec
Mark.Martinec+freebsd at ijs.si
Fri Oct 16 22:14:10 UTC 2015
Tried a today's git checkout on one of our small virtual 10.2 boxes
with not many services. The installation is almost trivial and a
reboot went without any problems. Time for a reboot was shrunk by
about 10 seconds (which is understandable with a small number of
services). Very good, almost too good to be true for such fairly
small and non-intrusive piece of shell/awk scripting - thanks!
Mark
2015-10-16 20:37, Cyril Vechera wrote:
> On 10/16/2015 07:06 PM, Mark Felder wrote:
>> On Fri, Oct 2, 2015, at 11:08, Cyril Vechera wrote:
>>> Hi there.
>>>
>>> We've got a small launcher script (~250 loc) for parallel services
>>> start/stop etc. It is used on our embedded systems and our users
>>> containers. And I've done a proof of concept for implanting it to the
>>> FreeBSD's standard /etc/rc for execution starting scripts in
>>> parallel.
>>> It gave me a boot time reduction of rc part from 27 to 7 seconds,
>>> mostly
>>> on eliminating jams for network or other long-latency resources
>>> waiting.
>>>
>>> The launcher is written in pure POSIX shell and uses FIFOs (named
>>> pipes)
>>> as a mutexes for synchronization. So it is embedded into /etc/rc and
>>> /etc/rc.d preserving rc.subr preloading. As a primary requirement, it
>>> guarantees topological order (strict partial order) defined by
>>> dependencies. It requires only POSIX shell, FreeBSD or Linux kernel,
>>> mkfifo and a writeable file system. Due to last requirement, it can
>>> be
>>> run on the late stage or should be supplied by some kinf of writtable
>>> fs, ie tmpfs. The FreeBSD-integrated version uses standard rcorder
>>> annotations (REQUIRE, BEFORE and PROVIDE) and there's no need to
>>> change
>>> rc.d scripts
>>>
>>> It's not a full init replacement or a kind of services supervision
>>> tool.
>>> It only starts or invokes a group of scripts in parallel with
>>> resolving
>>> and assuring execution in dependencies order.
>>>
>>> Please take a look at the script and patch set for FreeBSD:
>>>
>>> https://github.com/cvss/jet9-multitask-init/blob/master/jet9-multitask-init
>>> https://github.com/cvss/jet9-multitask-init/tree/master/examples/freebsd
>>>
>>>
>> Your first link is a 404, but this looks really nice!
>
> In the last commit (v1.3.0) I've renamed 'jet9-multitask-init' to
> 'jet9-multitask-flow' to avoid confusion with naming, because it's not
> as it seems, from name, an init replacement, but just a parallel task
> launcher. And now the actual repository URL is
> https://github.com/cvss/jet9-multitask-flow
>
> In this commit I've complete the FreeBSD compatibility. Now a script
> or dependency name can include minuses `-` and dots `.` (the first
> stone I stumbled over was ftp-proxy). And I've cleaned up the main
> script code
> https://github.com/cvss/jet9-multitask-flow/jet9-multitask-flow and
> have split it to more functions that can be redefined. So it's now
> easier to rewrite dependency extraction for FreeBSD rc-scripts -
> current implementation is too rough and takes three 'awk' runs for
> each rc-script. The last is not only time loss, but as DMarck
> mentioned before, using awk restricts parallel rc to be run only after
> FILESYSTEMS stage is done. Maybe it would be better to add to the
> rcorder(8) some new option to dump the gathered dependencies in
> tsort-compatible listing and insert them directly to flow execution
> plan.
>
> I've also added rc.conf variable `rc_parallel` to turn on and off
> parallel execution. There's a risk to discover an incomplete
> dependency annotations in some rc-scripts that earlier were masked by
> serial execution. I've done some checks by enabling as much rc.conf
> variables as possible to start more rc-scripts, and didn't found any
> error. But it looks too good to be true and I'm afraid that it's just
> to poor testing. I think if some ordering conflict will be found, it
> could be worked-around with introducing a white-list for script names
> that must be run only in sequentially.
>
> So it remained first to check if it really works in different
> conditions.
More information about the freebsd-hackers
mailing list