running newsyslog fiveminly

[Eugene Grosbein]

01.08.2011 00:31, Jeremy Chadwick writes:

>> For second kind of logs we have lines in newsyslog.conf such as following:
>>
>> /var/log/mpd.log 640 16 * @T0000  JC
>>
>> This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only.
>> Note, that compressing the file takes 8 minutes.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> I have three things to say on the matter, all of which are somewhat

Five things really :-)

> independent of one another so please keep that in mind.  I imagine #1
> below is your problem.
> 
> 1) The newsyslog.conf(5) man page has this clause in it, for the "when"
> field (in your case, @T0000):
> 
>      when    ...  If the when field contains an asterisk (`*'), log rotation
>              will solely depend on the contents of the size field.  Otherwise,
>              the when field consists of an optional interval in hours, usually
>              followed by an `@'-sign and a time in restricted ISO 8601 format.
> 
>              If a time is specified, the log file will only be trimmed if
>              newsyslog(8) is run within one hour of the specified time.  If an
>              interval is specified, the log file will be trimmed if that many
>              hours have passed since the last rotation. ...
> 
> You might think that "one hour of the specified time" value/clause
> correlates with the interval that newsyslog is run at via cron, but that
> would be wrong.  newsyslog REALLY DOES have hard-coded values for 3600
> seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog).  I
> have not looked at the code, but the fact of the matter is, 1 hour
> appears to be a "special" value.  I would heed that as a warning.
> 
> 2) Are you absolutely sure mpd.log is being rotated AND compressed within
> the 5 minute window?  If mpd.log is extremely large and your disks are
> slow, this could take a long time.  If possible, try (temporarily)
> removing bzip2 from the picture (remove J flag).

I've noted (see above) that compression takes 8 minutes.
I just think newsyslog should not deal with the file at 00:05.

> 3) mpd(8) logs via syslog(3).  When newsyslog(8), are you aware that it
> sends a SIGHUP to syslogd(8)?  As such, are you absolutely certain when
> this happen (every 5 minutes!) that the new log files are getting
> created correctly and promptly?

I see no other problems.

> 4) To debug this, you're probably going to need to run some cronjobs or
> daemons that keep a very close eye on /var/log/mpd.log* when the log
> rotation runs, in combination with running syslogd(8) in debug mode
> and/or verbose mode.

syslogd or newsyslo needs debug mode?

> 5) Why do you need to rotate logs every 5 minutes?  Why do you need such
> extreme levels of granularity in your rotated logs?  Just how much data
> are you logging via syslog?  If a lot, why so much?  It might be more
> effective to consider expanding your logging infrastructure to multiple
> machines if this the case.

Most of my boxes are diskless NanoBSD installations having /var in memory
and I need very detailed debug logs that grow quickly. These logs
can easily overflow /var partition in case of network problems (storms etc.)
so newsyslog have to check them often.

And I have another router that has an HDD to keep daily log and I'd like
to have their crontabs unified.

Eugene Grosbein