dangers of delaying an fsck on busy fileserver ?

[Eric Anderson]

On 05/20/07 15:28, Gore Jarold wrote:
> --- Scott Long <scottl at samsco.org> wrote:
> 
>> Gore Jarold wrote:
>>> --- Scott Long <scottl at samsco.org> wrote:
>>>
>>>
>>>> In an ideal world, the only consequence of
>> delaying
>>>> bgfsck is that
>>>> not all filesystem blocks will be marked free
>> that
>>>> should be.  So
>>>> if you deleted a large tree of files before the
>>>> crash, those blocks
>>>> might still show up in use until bgfsck
>> completes.
>>>
>>> Thank you.  Would _you_ do this with valuable data
>> ?
>> Very good question =-)  If you're using softupdates
>> then any
>> damage will have been done when the hard shutdown
>> happens; bgfsck
>> won't create any new damage.  The biggest problem of
>> bgfsck beyond
>> the i/o slowness and near deadlocks that it can
>> create (modulo the
>> fixes that the Kostik is working on) is that if it
>> does encounter
>> damage that it can't fix automatically, it exits and
>> leaves the 
>> filesystem inconsistent.  So you need to keep a very
>> close eye on
>> your logs and check for this, then schedule downtime
>> when it happens
>> so you can babysit a full fsck.
> 
> 
> Ahhh... I think you may have misunderstood my original
> question.  What I am saying is, I don't _ever_ want to
> do a background fsck.  My systems are too busy (and
> have too large of disks) to deal with the (current)
> baggage of making a 4 TB snapshot and then
> bg_fsck'ing.
> 
> What I am saying is the following:
> 
> - I set background_fsck_delay="86400"
> 
> - I tell datacenter techs NOT to call me when the
> system crashes - just to hit reset.
> 
> - users bang on the system, as normal, for X hours -
> all the while the filesystems are _dirty_ and nothing
> is being done about it
> 
> - I wake up hours later, unmount the filesystems, and
> foreground fsck them
> 
> My goal in all of this is to keep from being woken up
> in the middle of the night.  I don't care about the
> downtime to the system when I eventually do foreground
> fsck them, I just don't want to do it in the middle of
> the night _and_ I don't want my users to have to sit
> around waiting for me to do the fsck _on top of_ the
> fsck downtime itself.
> 
> So ... comments ?  I _suspect_ the conclusions are
> about the same - running on a dirty FS is the same as
> running on a dirty FS while being bg_fsck'd ... but I
> want to make sure...

So can't you turn off background fsck, and set fsck_y_enable="YES"? 
That would allow your NOC to hit reset, and it'll come back and fsck in 
the foreground while you sleep.

Eric