Reading a corrupted file on ZFS

[Karl Denninger]

On 2/13/2021 11:51, Stefan Esser wrote:
> Am 12.02.21 um 20:49 schrieb Karl Denninger:
>> On 2/12/2021 13:26, Artem Kuchin wrote:
>>> 12.02.2021 19:37, Karl Denninger пишет:
>>>> On 2/12/2021 11:22, Artem Kuchin wrote:
>>>>>
>>>>> This is frustrating. why..why..
>>>>
>>>> You created a synthetic situation that in the real world 
>>>> almost-never exists (ONE byte modified in all copies in the same 
>>>> allocation block but all other data in that block is intact and 
>>>> recoverable.)
>>>>
>>> I could be 1 GB file with ZFS wisth block size of 1MB and with 
>>> rotten bits within the same 1MB of block on different disks. How i 
>>> did it is not important, life is unpredictable, i'm not trying to 
>>> avoid everything. The question is what to do when it happens. And 
>>> currently the answer is - nothing.
>>>
>> The answer to a problem that does not present itself statistically 
>> speaking in the real world is the last one you worry about.  Worry 
>> about all the other ones and cover them first.
>
> True for some kinds of real world, false for others.
>
> I have a number of older 2 TB drives and use 2 of them to store
> DVB transport stream data, for later processing. They are not
> mirrored and the data format assumes that the transmission is not
> perfect. I have noticed that some of these files have been lost
> due to bad sectors, and while this would not be a problem on UFS,
> ZFS thinks I should not trust the file at all, despite 99,9999%
> of it being perfectly usable. (A fraction of the TS data is
> extracted from these files and stored as PS data on a ZFS RAID,
> where it will be safe and I never lost a file.)
>
> There are other applications that do not care for all data to be
> perfectly conserved (I know of some) and where using a mirrored
> setup configuration would just be a waste of money.
>
> Not everybody operates a data center with a large drive farm that
> holds data worth magnitudes more than the hardware cost (in which
> case the cost of redundancy is well justified).
>
>
> A solution could be to use UFS for files that should be readable
> (or recoverable with a tool) after single sector failures.
>
> But why should I have to mix UFS for such data with ZFS for the
> rest of the system (and I know you very well that these two do
> not mix too well if both are put under simultaneous high load).
>
>
> Therefore: There are valid reasons to not have redundancy and
> still not loose a large file of data for single bad block of a
> non-redundant (e.g striped for performance) ZPOOL/ZFS.
>
> Regards, STefan
>
That's what the patch does -- it allows you to read the file but the 
known-bad blocks will be "blanked" (marked that it's no good.)

Perhaps that sysctl should be part of the system generally, but you CAN 
do it the hard way with zdb even without it although it's a SERIOUS pain 
in the neck.

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4897 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20210213/5e2d2bb1/attachment.bin>