Bug 217055 - Consolidate random sleeps in periodic scripts
Alan Somers
asomers at freebsd.org
Mon Feb 13 22:07:19 UTC 2017
On Mon, Feb 13, 2017 at 2:25 PM, Cy Schubert <Cy.Schubert at komquats.com> wrote:
> In message <CAOtMX2jj_GCKjWW8CpapHutwH7OY00WnSWQS5VOuruv6i9Avqw at mail.gmail.c
> om>
> , Alan Somers writes:
>> On Mon, Feb 13, 2017 at 12:01 AM, Cy Schubert <Cy.Schubert at komquats.com> wrot
>> e:
>> > In message <CAOtMX2gJRuKKwwcHW5ZxTTZAm5Tmb7cVQ1SZEjwnuingYnO-Zg at mail.gmail.
>> c
>> > om>
>> > , Alan Somers writes:
>> >> I propose that we remove the various anti-congestion sleeps from
>> >> different periodic scripts, and add a single anti-congestion sleep to
>> >> the very beginning. Does this sound like a good idea to all of you?
>> >>
>> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217055
>> >
>> > I think the problem with the sleeps is simply the sleeps. My original plan
>> > to put my sleep/fetch in the background was shot down by some who thought
>> > it wasn't simple enough.
>> >
>> > Secondly, we don't need sleeps every boot. Ntpd for example only needs a
>> > sleep twice a year max to fetch a new leapfile so, to have a sleep every
>> > boot would be annoying.
>> >
>> > The best solution to replace sleeps would be to put a list of files:URLs
>> > into a queue to be fetched by fetcher script which would fetch only needed
>> > files that boot (or in the case of ntp via periodic.conf twice a year).
>> >
>> > A single script with a queue of files to fetch with one anti-congestion
>> > sleep, preferably in the background.
>> >
>> > NTP, btw can (will) use the leapfile in /etc/ntp until a fresher copy is
>> > fetched.
>> >
>> > Let's remove all fetching functions from the various rc scripts and queue
>> > them up early in a fetcher rc script, preferably in the background if at
>> > all possible.
>> >
>> >
>> > --
>> > Cheers,
>> > Cy Schubert <Cy.Schubert at cschubert.com>
>> > FreeBSD UNIX: <cy at FreeBSD.org> Web: http://www.FreeBSD.org
>> >
>> > The need of the many outweighs the greed of the few.
>>
>> Unfortunately that won't work, Cy. Some scripts may need to
>> dynamically determine what files to fetch, in a way that we can't do
>> in a single separate fetcher script. Worse, some scripts, like
>> 300.statistics from sysutils/bsdstats, need to _post_ a URL, not get
>> one.
>
> Diverse requirements cannot be addressed by one knob. To assume that
> various applications all have the same sleep requirement won't work.
Can you think any any periodic script whose sleep needs couldn't be
satisified by a single sleep at the beginning of the periodic run? I
can't. All sleeps I know of in /etc/periodic and
/usr/local/etc/periodic are for the purposes of reducing congestion
spikes on a server somewhere. The only way a single sleep could be
insufficient is if the random time interval is too small.
>
> I suppose we could have an optional single sleep script but we can't
> summarily remove all sleeps and assume all rc and periodic scripts sleep
> for some, one or possibly no applications requiring a sleep at any given
> time.
What? I can't make sense of that sentence.
> We can have a general sleep but removing the option of others would
> be counter productive. It doesn't make sense to have an arbitrary sleep
> just in case a subsequent script might need it.
Nothing in /etc/periodic needs to be run at a precise time, so adding
a sleep won't hurt anything. And if the sleep is configurable, a
sysadmin can always disable it. Also, from an anticongestion
standpoint it's objectively less good to chain multiple sleeps
together instead of using a single longer sleep. The reason is
because when you add several uniformly distributed random variables,
the result approaches a normal distribution with a peak in the middle.
But for anticongestion purposes, a uniform distribution is really what
you want.
> If we have to, let's either
> reduce the length of the sleeps or put better yet background them.
I don't like the idea of backgrounding parts of the periodic scripts,
for three reasons. One, it's complicated. Two, it prevents
periodic(8) from sending a single status email. Three, periodic(8)
might start the next day's run before the previous day's is complete.
>
> What's motivating this? Server? Laptop?
Servers mostly. A confounding issue is Bug 210188 - periodic daily
sleeps even when invoked from a terminal . When I run periodic by
hand, it still sleeps. It would be easier to fix that bug if only one
sleep were involved instead of several.
-Alan
More information about the freebsd-pkg
mailing list