Archiving a log file
frank2 at fjl.co.uk
Sun Aug 4 00:05:18 UTC 2013
On 04/08/2013 00:20, kpneal at pobox.com wrote:
> On Sun, Aug 04, 2013 at 12:11:21AM +0100, Frank Leonhardt wrote:
>> The answer isn't (AFAIK) newsyslog
>> As a one-off, I need to archive an old log file - say httpd-access.log -
>> while its still open. I don't want this to happen automatically and I
>> don't want to set up newsyslog or anything like that. And I really don't
>> want to mess about with signals to whatever is writing to the file, even
>> assuming the writer could respond to them. I can't just rename the file
>> as it's open for writing, and there would also be a good chance that
>> something will be added to the file while it's being compressed.
>> What I actually do is:
>> cp httpd-access.log httpd-access.log-03-Aug-13 && :> httpd-access.log &&
>> bzip2 httpd-access.log-03-Aug-13
>> Data might be lost here as something may be added between the cp being
>> completed and the file being truncated. It's not the end of the world if
>> this happens, but is there a better way? I could always shut down Apache
>> for the duration, but I don't want to do that either, so in this case
>> I'm happy to take the risk (it's not like I'm likely to miss anything
>> that important).
>> I don't know if this can be relied on as a POSIX thing, but the cp
>> command simply(!) issues read() and write() calls until read() fails to
>> get any more bytes, so if data is being appended to the file after cp is
>> started it'll still be copied. Therefore the window where stuff could be
>> written after the copy but before the truncation is shortened, but extant.
>> So what's the magic utility I don't know about?
> How about cronolog? I use it with Apache where Apache logs to cronolog
> and cronolog handles the rotating of the logs. No signals. No races.
> It even makes a symlink pointing at the newest log file.
> It doesn't seem to have a way to compress logs, but you could probably
> script up something that wakes up every so often and compresses files
> if a newer file exists.
Thanks for the suggestion - I wasn't aware of cronolog. Unfortunately it
doesn't work on existing log files, only stuff piped to it from the
start, so it won't help here. I can see it being very handy in other
situations though. httpd-access.log was just an example of such a file,
but I'm looking for a general solution.
More information about the freebsd-questions