awk help

[Andreas Perstinger]

I think awk is the better tool for your task but you could still 
simplify your shell script a little bit:

>      hits_rpt="/etc/ipf_pool_hits_rpt" >      rm $hits_rpt>      touch $hits_rpt> > 
hits_new="/etc/ipf_pool.hits.yes">      rm $hits_new>      touch 
$hits_new> >      hits_no="/etc/ipf_pool.hits.no">      rm $hits_no> 
  touch $hits_no
Using the "truncate" command you could reduce these lines to

hits_rpt="/etc/ipf_pool_hits_rpt"
hits_new="/etc/ipf_pool.hits.yes"
hits_no="/etc/ipf_pool.hits.no"
truncate -s 0 $hits_rpt $hits_yes $hits_no

>      ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null > >      for line in `cat $hits_rpt`; do> >        # drop the first 3 
rpt lines>        poollist_line="">        poollist_line=`echo -n $line 
| grep poollist`>        [ -n "${poollist_line}" ] && continue> > 
role_line="">        role_line=`echo -n $line | grep Role`>        [ -n 
"${role_line}" ] && continue> >        nodes_line=""> 
nodes_line=`echo -n $line | grep Nodes`>        [ -n "${nodes_line}" ] 
&& continue
Assuming that your file always contains the three unneeded lines at the 
beginning you could use the "tail" command to start at line 4. And I 
would also use a while-loop in combination with the "read" built-in 
function. Then you are able to process pairs of lines:

tail -n +4 $hits_rpt | while read line
do

>        in_line1=`echo -n $line | grep Address:`
>        [ -n "${in_line1}" ] && save_in_line1="${in_line1}"
> 
>        in_line2=`echo -n $line | grep Hits`
>        [ -n "${in_line2}" ] && save_in_line2="${in_line2}"
> 
>        if [ "${save_in_line1}" -a "${save_in_line2}" ]; then
>          build_line1=${save_in_line1##*:}
>          build_line1=${build_line1%%/*}
>          build_line1="${build_line1};"
>          build_line2=${save_in_line2##*Hits }
>          # So remove everything to the right of the word Bytes.
>          build_line2=${build_line2%%Bytes*}

Using sed instead of grep would simplify the extraction of the IP 
address and the number of hits:

     ip=$(echo $line | sed -e 's/ *Address: *\([^/]*\).*/\1/')
     read line
     hits=$(echo $line | sed -e 's/ *Hits *\([[:digit:]*]\).*/\1/')

>          if [ ${build_line2} -gt 0 ]; then
>            db_rec="$added_date  ${build_line1}"
>            echo "${db_rec}" >> $hits_new
>          fi
>          build_line="${build_line2}  ${build_line1}"
>          echo "${build_line}" >> $hits_no
>          in_line1=""
>          in_line2=""
>          save_in_line1=""
>          save_in_line2=""
>        else
>          continue
>        fi
>      done
>      exit 0

So the complete shell script would be:

#!/bin/sh

added_date="`date +%Y%m%d`"
hits_rpt="/etc/ipf_pool_hits_rpt"
hits_new="/etc/ipf_pool.hits.yes"
hits_no="/etc/ipf_pool.hits.no"
truncate -s 0 $hits_rpt $hits_yes $hits_no

ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null

tail -n +4 $hits_rpt | while read line
do
     ip=$(echo $line | sed -e 's/ *Address: *\([^/]*\).*/\1/')
     read line
     hits=$(echo $line | sed -e 's/ *Hits *\([[:digit:]*]\).*/\1/')
     if [ "$hits" -gt 0 ]; then
         echo "$added_date  ${ip};" >> $hits_new
     fi
     echo "$hits  ${ip};" >> $hits_no
done
exit 0

Does this still take minutes to process your data?

Bye, Andreas