Notification for gmirror failures

[Miroslav Lachman]

Andrea Brancatelli wrote on 2018/05/08 09:28:
> Hi,
> 
> out of curiosity, does any kind of GMirror-failure notification tools
> exist?
> 
> Either native... SNMP... external ports... whatever?

I don't know any daemon like solution so I wrote some simple shell 
script. Its used on all our machines. It should be run be cron in some 
sane interval (like each 5 or 10 minutes)

#!/bin/sh

## $Id: check_gmirror.sh 8c1f418ba7f5 2017-06-15 12:36 +0200 lachman $

export PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin"

mailto="you at example.com"
subject_pref="$(hostname) gmirror"
notify_state="/var/run/${0##*/}.notified"

cmd_out=$(gmirror status -s)

components="$(echo "$cmd_out" | grep -F -v COMPLETE)"

if [ -n "$components" ]; then
     if [ ! -f "$notify_state" ]; then
         subject="$subject_pref FAILED - $(date '+%Y-%m-%d %H:%M:%S')"
         echo "$cmd_out" | sed -E 's/(SYNCHRONIZING,) [0-9]+%/\1 x%/' > 
"$notify_state"
         echo "$cmd_out" | mail -s "$subject" "$mailto"
         echo "$subject" | logger
     else
         subject="$subject_pref FAILED - $(date '+%Y-%m-%d %H:%M:%S')"
         echo "$cmd_out" | sed -E 's/(SYNCHRONIZING,) [0-9]+%/\1 x%/' > 
"$notify_state.now"
         diff -u -b -B "$notify_state" "$notify_state.now" > 
"$notify_state.diff"

         if [ -s "$notify_state.diff" ]; then    ## if diff is not empty
             cat "$notify_state.diff" | mail -s "$subject" "$mailto"
             echo "$subject additional failure occurred" | logger
             cp "$notify_state.now" "$notify_state"
         fi
         rm "$notify_state.now" "$notify_state.diff"
     fi
else
     if [ -f "$notify_state" ]; then
         rm "$notify_state"
         subject="$subject_pref now OK - $(date '+%Y-%m-%d %H:%M:%S')"
         echo "$cmd_out" | mail -s "$subject" "$mailto"
         echo "$subject" | logger
     fi
fi


We have similar script for checking ZFS too.

Miroslav Lachman