Reading a corrupted file on ZFS

[Stefan Esser]

Am 12.02.21 um 20:49 schrieb Karl Denninger:
> On 2/12/2021 13:26, Artem Kuchin wrote:
>> 12.02.2021 19:37, Karl Denninger пишет:
>>> On 2/12/2021 11:22, Artem Kuchin wrote:
>>>>
>>>> This is frustrating. why..why..
>>>
>>> You created a synthetic situation that in the real world almost-never 
>>> exists (ONE byte modified in all copies in the same allocation block 
>>> but all other data in that block is intact and recoverable.)
>>>
>> I could be 1 GB file with ZFS wisth block size of 1MB and with rotten 
>> bits within the same 1MB of block on different disks. How i did it is 
>> not important, life is unpredictable, i'm not trying to avoid 
>> everything. The question is what to do when it happens. And currently 
>> the answer is - nothing.
>>
> The answer to a problem that does not present itself statistically 
> speaking in the real world is the last one you worry about.  Worry about 
> all the other ones and cover them first.

True for some kinds of real world, false for others.

I have a number of older 2 TB drives and use 2 of them to store
DVB transport stream data, for later processing. They are not
mirrored and the data format assumes that the transmission is not
perfect. I have noticed that some of these files have been lost
due to bad sectors, and while this would not be a problem on UFS,
ZFS thinks I should not trust the file at all, despite 99,9999%
of it being perfectly usable. (A fraction of the TS data is
extracted from these files and stored as PS data on a ZFS RAID,
where it will be safe and I never lost a file.)

There are other applications that do not care for all data to be
perfectly conserved (I know of some) and where using a mirrored
setup configuration would just be a waste of money.

Not everybody operates a data center with a large drive farm that
holds data worth magnitudes more than the hardware cost (in which
case the cost of redundancy is well justified).


A solution could be to use UFS for files that should be readable
(or recoverable with a tool) after single sector failures.

But why should I have to mix UFS for such data with ZFS for the
rest of the system (and I know you very well that these two do
not mix too well if both are put under simultaneous high load).


Therefore: There are valid reasons to not have redundancy and
still not loose a large file of data for single bad block of a
non-redundant (e.g striped for performance) ZPOOL/ZFS.

Regards, STefan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20210213/c71ee2dc/attachment.sig>