zfs i/o error, no driver error

[Andriy Gapon]

on 07/06/2010 11:34 Jeremy Chadwick said the following:
> On Mon, Jun 07, 2010 at 11:15:54AM +0300, Andriy Gapon wrote:
>> During recent zpool scrub one read error was detected and "128K repaired".
>>
>> In system log I see the following message:
>> ZFS: vdev I/O failure, zpool=tank
>> path=/dev/gptid/536c6f78-e4f3-11de-b9f8-001cc08221ff offset=284456910848
>> size=131072 error=5
>>
>> On the other hand, there are no other errors, nothing from geom, ahci, etc.
>> Why would that happen? What kind of error could this be?
> 
> I believe this indicates silent data corruption[1], which ZFS can
> auto-correct if the pool is a mirror or raidz (otherwise it can detect
> the problem but not fix it).

This pool is a mirror.

> This can happen for a lot of reasons, but
> tracking down the source is often difficult.  Usually it indicates the
> disk itself has some kind of problem (cache going bad, some sector
> remaps which didn't happen or failed, etc.).

Please note that this is not a CKSUM error, but READ error.

> What I'd need to determine the cause:
> 
> - Full "zpool status tank" output before the scrub

This was "all clear".

> - Full "zpool status tank" output after the scrub

zpool status -v
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub completed after 5h0m with 0 errors on Sat Jun  5 05:05:43 2010
config:

        NAME                                            STATE     READ WRITE CKSUM
        tank                                            ONLINE       0     0     0
          mirror                                        ONLINE       0     0     0
            ada0p4                                      ONLINE       0     0     0
            gptid/536c6f78-e4f3-11de-b9f8-001cc08221ff  ONLINE       1     0
 0  128K repaired

> - Full "smartctl -a /dev/XXX" for all disk members of zpool "tank"

Those output for both disks are "perfect".
I monitor them regularly, also smartd is running and complaints from it.

> Furthermore, what made you decide to scrub the pool on a whim?

Why on a whim? It was a regularly scheduled scrub (bi-weekly).

-- 
Andriy Gapon