Do we want a periodic script for a zfs scrub?

Fri Jun 11 08:42:29 UTC 2010

Quoting jhell <jhell at dataix.net> (from Thu, 10 Jun 2010 15:11:12 -0400):

> On 06/10/2010 12:34, Artem Belevich wrote:
>> You can do something like this:
>>
>> #SCRUB_TS="2010-06-08.20:51:12"
>> SCRUB_TS=$1
>> # parse timestamp, move it forward by 1 month and print in seconds  
>> since Epoch
>> NEXT_SCRUB_DATE_S=`date -j -f "%Y-%m-%d.%H:%M:%S" -v+1m +"%s" $SCRUB_TS`
>> # for debugging purposes convert epoch time into something human-readable
>> NEXT_SCRUB_DATE=`date -r $NEXT_SCRUB_DATE`
>> # surrent time in secs since Epoch.
>> NOW_S=`date +"%s"`
>> # Compare two times to figure out if next scrub time is still in the future
>> if [ $NOW_S -gt $NEXT_SCRUB_DATE_S ]; then
>>     echo yup.
>> else
>>     echo nope.
>> fi
>>
>> --Artem
>
> #!/bin/sh
>
> lastscrub=$(zpool history exports |grep scrub |tail -1 |cut -f1 -d.)
> todaypoch=$(date -j -f "%Y-%m-%d" "+%s" $(date "+%Y-%m-%d"))
> scrubpoch=$(date -j -f "%Y-%m-%d" "+%s" $lastscrub)
>
> echo $lastscrub Last Scrub From zpool history
> echo $todaypoch Today converted to seconds since epoch
> echo $scrubpoch Last scrub converted to seconds since epoch
>
> expired=$((((($todaypoch-$scrubpoch)/60)/60)/24))

Apart from the fact that we can do this with one $(( ))... what  
happens if/when time_t is extended to 64 bits on 32 bit platforms? Can  
we get into trouble with the shell-arithmetic or not? It depends upon  
the bit-size of the shell integers, and the signedness of them. Jilles  
(our shell maintainer) suggested also to use the seconds since epoch  
and I asked him the same question. I'm waiting for an answer from him.

The same concerns apply to test(1) (or the corresponding buildin) in  
the solution of Artem.

By calculating with days everywhere (like in my solution), I'm sure  
that it takes longer to hit a wall than by calculating with seconds  
since epoch (which can cause a problem in 2038 or during a transition  
when this problem is tackled in time_t but not here, which is not that  
far away). The off-by-one day once every 4 years shouldn't be a  
problem. If someone can assure with some nice facts, that using the  
seconds since epoch will not cause problems in the described cases, I  
have no problem to switch to use them.

Bye,
Alexander.

> if [ ${expired:=30} -ge ${daily_scrub_zfs_threshold:=30} ]; then
>         echo "Performing Scrub...."
>         else
>         echo "SORRY its only been $expired days since your last scrub."
> fi
>
>
> My reasoning for setting expired to have a default value of 30 depended
> on whether a pool may have just been created in which a scrub would have
> never been performed thus with this value being equal to that of the
> default threshold would allow that pool to be scrubbed on the first day
> it was created.
>
> I considered just doing ${expired:=${daily_scrub_zfs_threshold:=30}}
> which would also allow it to be set to whatever a user set their value
> to before the pool was created and adds another layer of redundancy on
> that variable in a fail-safe sort of way.
>
> Regards & nice work on this. I just noticed the CFT just after writing
> this. but still have a look at the above it may simplify the testing
> while providing some fallback for what I stated above.
>
>>
>>
>>
>> On Thu, Jun 10, 2010 at 8:38 AM, Alexander Leidinger
>> <Alexander at leidinger.net> wrote:
>>> Quoting Artem Belevich <fbsdlist at src.cx> (from Thu, 10 Jun 2010 07:59:46
>>> -0700):
>>>
>>>>> Good idea! I even found a command line which does the calculation for the
>>>>> number of days between "now" and the last run (not taking a leap year
>>>>> into
>>>>> account, but an off-by-one day error here does not matter).
>>>>
>>>> You can get exactly one month difference by using -v option of 'date'
>>>> command to figure out the time/date offset by arbitrary amount.
>>>> Combined with +"%s" format to print number of seconds since Epoch and
>>>> -r to specify the reference point in time it makes 'date' pretty
>>>> useful in scripts.
>>>
>>> What we have is the date of the last scrub (e.g. 2010-06-08.20:51:12), and
>>> what we want to know is if between the last scrub and now we passed a
>>> specific amount of days or not.
>>>
>>> What I do is taking the year multiplied with 365 plus the day of the year.
>>> Both of this for the last date of the scrub and "now". The  
>>> difference is the
>>> number of days between those two dates. This value I can use with  
>>> -le or -ge
>>> for the test command.
>>>
>>> This is only off by one once in a leap year when the leap-day is in-between
>>> the two dates (those people which want to scrub every 4 years are  
>>> off by two
>>> when both leap-days are in-between, but a scrub of every 4 years or more
>>> looks unreasonable to me, so I do not care much about this).
>>>
>>> This is done in one line with two calls to date (once for the last scrub,
>>> once for "now") and a little bit of shell-buildin-arithmetic. If you have a
>>> more correct version which is not significantly more complex, feel free to
>>> share it here.
>>>
>>> Bye,
>>> Alexander.
>>>
>>> --
>>>  "Who would have though hell would really exist? And that it would  
>>> be in New
>>> Jersey?" -Leela
>>> "Actually..." - Fry
>>>
>>> http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
>>> http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137
>>>
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>
> --
>
>  jhell
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>

-- 
Before marriage the three little words are "I love you," after marriage
they are "Let's eat out."

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137