About newsyslog behavior

[Jim Xochellis]

Hi Matthew, hi list,

On Wednesday, July 2, 2003, at 02:01 PM, Matthew Seaman wrote:

> On Wed, Jul 02, 2003 at 12:08:31PM +0300, Jim Xochellis wrote:
>
>> I am using the newsyslog utility to turn over my log files
>> automatically. I have noticed that some processes have problem to
>> continue using their log file after newsyslog has turned it over and
>> need to receive the SINGHUP signal to re-start logging correctly.
>> However sending SINGHUP  has uncomfortable consequences in some cases
>> (for instance when sending signal to netatalk, or other file servers
>> perhaps).
>
> That's 'SIGHUP', although I do like the idea of the kernel singing to
> it's processes to keep them in line...
>

Sorry about the typo...

> The use of the SIGHUP signal to tell a long-running process to
> reconfigure itself, reopen all of it's file descriptors etc. is a very
> useful convention, but unfortunately it is only a convention -- as
> you've found out many processes don't follow it.
>
>> I suspected  that some processes are confused because a *new* log file
>> is created and these processes are making the assumption that their 
>> log
>> file will be always the same and perhaps they open it once and then
>> work with the FILE pointer. (just a simple theory that explains some
>> facts) I have confirmed that newsyslog actually creates a new log file
>> (instead of copying it and then disposing its contents) by reading the
>> source of the newsyslog.c file and particularly the dotrim() function.
>> I have also tested that changing the contents of the log files by hand
>> does not affect the logging of most processes (surely not the logging
>> of netatalk).
>
> Yes, this is certainly true.  You can confirm that the process in
> question is holding an open file decriptor on the log file by use of
> the fstat(1) command.
>
> Now, it's entirely possible to cycle log files by essentially:
>
>     # cp log log.old
>     # cat /dev/null > log
>
> (instead of what newsyslog does which is effectively:
>
>     # mv log log.old
>     # touch log
> )
>
> but this still has problems.  Somehow, you'ld need to tell the process
> with the open file descriptor that the underlying file had shrunk in
> size, and that it should do an fseek(stream, 0, SEEK_END) --- other
> wise, it will write any new data at the same file offset as the former
> end-of-file, leading to all sorts of weirdnesses.
>

I understand, but I also suspect that there is a very rare possibility 
to face this kind of problems. (please correct me if I am wrong at this)

1) Many servers (netatalk, samba, apache, etc) are creating many 
processes which are all using the same log files. Hence, these servers 
have already solved the above problems somehow.
2) When implementing logging we are always flushing the log file *very* 
frequently and I believe that this fact is reducing the possibility of 
facing the above problems.
3) I have done some tests my self and had no problems at all. (Of 
course that is not a proof, just an indication...)


>> Having the above in mind, isn't it worthwhile to add an option in
>> newsyslog in order to avoid the creation of a new log file when it is
>> inconvenient?
>> Isn't it feasible to dispose the contents of the old log file instead
>> of creating a new one?
>> Anything that I am missing here? (giving the fact that I am not a unix
>> guru, only a C programmer)
>
> One thing that might be of use if you're stuck with a program that
> won't play ball with newsyslog is to use something like cronolog(1)
> which is in ports as sysutils/cronolog --- see
> http://www.ford-mason.co.uk/resources/cronolog/ However, this requires
> that your program can write it's log output to a pipe rather than

Interesting if applicable (not in my case I am afraid)

> directly to a file.  cronolog will manage switching to a new log file
> according to the time and date.  Or you could code up a very simple
> equivalent that reads log messages from it's stdin and writes to the
> named log file, but that can handle a SIGHUP telling it to reopen the
> log file.
>

I am thinking about changing the newsyslog instead (adding an option) 
but I am such a newbie in Unix, makefiles and all this stuff (except c) 
;-)


Regards,
Jim Xochellis