Apache Rotate Logs and Log Rotate.

[Peter Pluta]

I made a cron entry to compress the logs. We keep them for 10 days. Each 
log on this server is 940mb for the access and 430mb for the error. This 
site is very busy and I like having logs for various reasons (dump users 
uploading illegal content is one reason). The memory buffer clears out 
in around 10 minutes because the new log started 10 minutes after the 
old ended. I added 30    6   *   *   bzip2 -1q /var/log/http-*.log.* to 
cron. I think this is a pretty good setup. Our log error level is set to 
info as well. The web application (php driven) is not spitting up that 
many errors. Most of them are missing images that were deleted and just 
broken old links. Thanks for all you're replies David. I really learned 
a lot and i know understand much more. I started carbon copying the 
emails to the mailing list in case anyone is interested in this as well.

David Robillard wrote:
> On 3/5/07, Peter Pluta <peter at placidpublishing.net> wrote:
>> Gotcha, do you use a script to compress the logs after the SIGUSR1 and
>> after waiting for a bit for apache to clear it's logging buffer (to not
>> have missing logs)?
>
> No I don't. I don't even see why one would want to do this?
>
> Newsyslog deletes extra logs. So if our disk space is enough to hold
> the amount of logs we require (see math below), then there's no need
> to compress any Apache logs at all. Right!?!!
>
> If we come back to my example of 100 VirtualHost with log files of 2Mb
> each and we keep only 10 of them. Using USR1 as the kill signal, For
> an httpd children to miss any log entry would mean that this children
> writes more than 10 times 2Mb of logs in a very short period of time.
> Check your VirtualHost load and determine the average response time
> for each httpd children. If it's 2min (which is HUGE for an httpd
> children) That would mean that you'd need to have more than 20Mb of
> logs generated in less than 2min. In ASCII, that's a whole lot of
> logs. I'd say your best bet would be to switch your LogLevel from
> "debug" to "info" in your httpd.conf and restart Apache... ;)
>
> Or you run a really busy website.
> Or your web application code/architecture may need a revision.
>
> Have fun!
>
> David
>
>> > Well, if you do use newsyslog to rotate Apache log files, then it's
>> > just a matter of setting the number of files you whish to keep. From
>> > newsyslog.conf(5)
>> >
>> >  count   Specify the maximum number of archive files which may exist.
>> >          This does not consider the current log file.
>> >
>> > Let's say you rotate your files once they reach 2Mb for example and
>> > that you've configured 10 in your newsyslog,conf <count> field. Then
>> > that means a maximum of 10 x 2Mb = 20Mb will be kept for one
>> > VirtualHost. Now if you have 100 virtual hosts all configured this
>> > way, then you will need 100 x 20Mb = 2000Mb or 2Gb for all your Apache
>> > logs.
>> >
>> > Considering today's disk drive sizes are well beyond the 300Gb, I
>> > don't think this is a problem at all.
>> >
>> > Of course, YMMV so check your own needs and do the math.
>> >
>> > Cheers,
>> >
>> > David
>