analyzing httpd-error.log

[Charles Howse]

On Mar 30, 2009, at 3:11 PM, Mel Flynn wrote:

> On Sunday 29 March 2009 22:25:55 Charles Howse wrote:
>> On Mar 29, 2009, at 1:54 PM, Glen Barber wrote:
>>> On Sun, Mar 29, 2009 at 1:33 PM, Charles Howse <chowse at charter.net>
>>>
>>> wrote:
>>>> On Mar 28, 2009, at 11:51 PM, Olivier Nicole wrote:
>>>>> Hi,
>>>>>
>>>>>> Webalizer is doing what it's supposed to with httpd-access.log,  
>>>>>> but
>>>>>> when I give it the error log to process is coughs, spits and  
>>>>>> spills
>>>>>> out errors with no data processed.  My research hasn't turned  
>>>>>> up a
>>>>>> good solution for webalizer and -error.log.
>>>
>>> What are the errors?
>>
>> Intrusion attempts, (a few) bad links in my website, also I use the
>> error.log to troubleshoot cgi scripts.
>
> Nothing beats tail -f for debugging.
>
> 404's can be gathered from access log, by webalizer (or awstats  
> or ...) and
> are summarized when enabled. If I remember correctly, there's also a  
> referer
> top list, that specifies which pages link to invalid pages, but it  
> may have
> been awstats that does this.
>
> Can't think of anything specific for apache error log, I roll my own  
> grok
> rules if I'm really interested in a specific vulnerability. See  
> sysutils/grok.

Yes, awstats does a fair job of summarizing 404's.
I have a hardware router, not extremely interested in a packet filter  
for my webserver.
I guess I could do some deny from rules in apache.conf if necessary...?

*****
#!/bin/sh

date=`date "+%b %e"`

cd /var/log

grep "$date" httpd-error.log | grep -v 192.168.254.254 | grep -v  
192.168.254.3 > /root/err.log

mail -s "httpd-error.log" charles < /root/err.log
*****
I run this from cron @ 11:59 PM every night, and that gives me the  
error log for that day.
Maybe I won't miss anything serious between 11:59 and midnight.

Can anyone help with a grep command that will filter out all addresses  
beginning with 192.168.254?
Something to replace the 2 piped commands with only 1.
I tried: grep -v -regexp[^192\.168\.254\.] but that didn't match any  
records...?