dangers of delaying an fsck on busy fileserver ?

[Scott Long]

Gore Jarold wrote:
> --- Scott Long <scottl at samsco.org> wrote:
> 
>> Gore Jarold wrote:
>>> --- Scott Long <scottl at samsco.org> wrote:
>>>
>>>
>>>> In an ideal world, the only consequence of
>> delaying
>>>> bgfsck is that
>>>> not all filesystem blocks will be marked free
>> that
>>>> should be.  So
>>>> if you deleted a large tree of files before the
>>>> crash, those blocks
>>>> might still show up in use until bgfsck
>> completes.
>>>
>>> Thank you.  Would _you_ do this with valuable data
>> ?
>> Very good question =-)  If you're using softupdates
>> then any
>> damage will have been done when the hard shutdown
>> happens; bgfsck
>> won't create any new damage.  The biggest problem of
>> bgfsck beyond
>> the i/o slowness and near deadlocks that it can
>> create (modulo the
>> fixes that the Kostik is working on) is that if it
>> does encounter
>> damage that it can't fix automatically, it exits and
>> leaves the 
>> filesystem inconsistent.  So you need to keep a very
>> close eye on
>> your logs and check for this, then schedule downtime
>> when it happens
>> so you can babysit a full fsck.
> 
> 
> Ahhh... I think you may have misunderstood my original
> question.  What I am saying is, I don't _ever_ want to
> do a background fsck.  My systems are too busy (and
> have too large of disks) to deal with the (current)
> baggage of making a 4 TB snapshot and then
> bg_fsck'ing.
> 
> What I am saying is the following:
> 
> - I set background_fsck_delay="86400"
> 
> - I tell datacenter techs NOT to call me when the
> system crashes - just to hit reset.
> 
> - users bang on the system, as normal, for X hours -
> all the while the filesystems are _dirty_ and nothing
> is being done about it
> 
> - I wake up hours later, unmount the filesystems, and
> foreground fsck them
> 
> My goal in all of this is to keep from being woken up
> in the middle of the night.  I don't care about the
> downtime to the system when I eventually do foreground
> fsck them, I just don't want to do it in the middle of
> the night _and_ I don't want my users to have to sit
> around waiting for me to do the fsck _on top of_ the
> fsck downtime itself.
> 
> So ... comments ?  I _suspect_ the conclusions are
> about the same - running on a dirty FS is the same as
> running on a dirty FS while being bg_fsck'd ... but I
> want to make sure...
> 

So yeah, the only consequence of doing the preen but not the
bgfsck is that you might loose some free space after a crash.
How much depends on your workload.  If you're deleting large files
or large trees of files, the loss might be significant.  It
sounds like you understand how to manage that, though, so it'll
probably be a fine solution for you.

Scott