UFS Crash and directories now missing

[Jerome Herman]

On 28/04/2012 19:52, Alejandro Imass wrote:
> On Sat, Apr 28, 2012 at 1:31 PM, Robert Bonomi<bonomi at mail.r-bonomi.com>  wrote:
>> Alejandro Imass<aimass at yabarana.com>  wrote:
>>> On Sat, Apr 28, 2012 at 11:39 AM, Robert Bonomi
>>> <bonomi at mail.r-bonomi.com>  wrote:
>>>>   Alejandro Imass<aimass at yabarana.com>  wrote:
>>>>> After a little more research, ___it it NOT unlikely at all___ that
>>>>> under high distress and a hard boot, UFS could have somehow corrupted
>>>>> the directory structure, whilst maintaining the data intact.
>>>> This is techically accurate, *BUT* the specifics of the quote "corruption"
>>>> unquote in the case under discussion make it *EXTREMELY* unlikely that this
>>>> is what happened.
>>>>
>>>> 99.99+++% of all UFS filesystem "corruption' issues are the result of a
>>>> system crash _between_ the time cached 'meta-data' is updated in memory
>>>> and that data is flushed to disk (a deferred write).
>>>>
>>>> The second most common (and vanishingly rare) failure mode is a powerfail
>>>> _as_ a sector of disk is being written -- resulting in 'garbage data'
>>>> being written to disk.
>>>>
>>>> The next possibility is 'cosmic rays'.  If running on 'cheap' hardware
>>>> (i.e., without 'ECC' memory), this can cause a *SINGLE-BIT* error in
>>>> data being output.
>>>>
>>>> The fact that the 'corrupted' filesystem passed fsck -without- any reported
>>>> errors shows that everything in the filesystem meta-data was consistent
>>>>
>>> [...]
>>>
>>>> I think it is safe to conclude that the probabilities -greatly- favor
>>>> alternative #1.
>>>>
>>> OK. So after your comments and further research I concur with you on
>>> the mv but if it wasn't a human, then this might be exposing a serious
>>> security flaw in the jail system or the way EzJail implements it.
>> BOGON ALERT!!!
>>
> I admit my ignorance on how the filesystem works but I don't think
> your condescending remarks add a lot of value. The issue here is this
> actually happened and there is a flaw somewhere other than "the stupid
> administrator did it".
Ok,

Not wanting to take any side in what could end up in personal attacks 
and nasty things being said about any poster genitors but :

- Jails are very widely used, in fact it is probably one of the most 
used functionnality of FreeBSD. Far beyond ZFS, MAC or any of the other 
nice thingies FreeBSD has.
- Jails are very often misused. Though not overly complex, creating a 
proper jail and upgrading it can sometime be a bit tricky.
- Though not entirely devoid of bug and perfect, FreeBSD 8.2 is probably 
the best thing there is out there when it comes to system stability. It 
might be lacking some little nooks and cranies when it comes to perfect 
compliance with obscure standards, it might not behave as expected in 
some very few situation, but these are extremely rare. FreeBSD 8.2 is 
very widely used and this is one of the first time I heard of such a 
problem in jails. Nothing even remotely rings a bell.

Take all these information into account and put yourself in our shoes. 
When reading your problem description, most of us will be inclined to 
think that you did something wrong.

My personnal guess would be that you probably abused  "ln" a bit too 
much when creating the jails (total shot in the dark here, but it could 
explain what happened).  I don't see how journaling could impact your 
jails in anyway except if your jails were all extremely new when the 
crash happened or that the I/O was such that FreeBSD could never sync 
and commit journal from the time you created your jails to the time 
where the system crashed.
Extremely unlikely.

So my question is : where all the jail created properly ? Did you cpdup 
each and every one of them or were you lazy at some point ? Are all the 
jails properly declared in rc.conf ? My guess would be that the first 
jail was created in the right way, but that others were created using cp 
and ln, resulting in unexpected behaviour in the end. If I am right then 
the "surviving" jail would be either the first or the last you created.

Jerome Herman

> _______________________________________________
> freebsd-questions at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"