[Bug 195746] zfs L2ARC wrong alloc/free size

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Fri Feb 13 16:04:45 UTC 2015


John K. Gates <john at infoassurancepro.com> changed:

           What    |Removed                     |Added
                 CC|                            |john at infoassurancepro.com

--- Comment #1 from John K. Gates <john at infoassurancepro.com> ---
I have been discussing this issue here:


Checksum and IO errors appear after an L2ARC device fills completely with cache
data on any release of FreeBSD after L2ARC compression
(http://wiki.illumos.org/display/illumos/L2ARC+Compression) was enabled
(9.3-RELEASE and later, and 10.0-RELEASE and later).

The following is from an 10.1-RELEASE-p5 (svn rev 278678) running a generic
kernel.  /etc/make.conf contains only "CPUTYPE?=core2".  ZFS pools are v28
(created under 9.2-RELEASE) and have NOT been updated for feature flags.

"zfs-stats -L" shows the L2ARC "DEGRADED" with numerous I/O and checksum

root at cadence:/ # zfs-stats -L

ZFS Subsystem Report                            Fri Feb 13 10:36:49 2015

L2 ARC Summary: (DEGRADED)
        Passed Headroom:                        30.28m
        Tried Lock Failures:                    24.83m
        IO In Progress:                         247
        Low Memory Aborts:                      103
        Free on Write:                          54.97k
        Writes While Full:                      10.62k
        R/W Clashes:                            562
        Bad Checksums:                          1.29m
        IO Errors:                              128.28k
        SPA Mismatch:                           48.53b

L2 ARC Size: (Adaptive)                         33.51   GiB
        Header Size:                    2.24%   768.85  MiB

L2 ARC Evicts:
        Lock Retries:                           18
        Upon Reading:                           0

L2 ARC Breakdown:                               35.47m
        Hit Ratio:                      26.64%  9.45m
        Miss Ratio:                     73.36%  26.02m
        Feeds:                                  568.79k

L2 ARC Buffer:
        Bytes Scanned:                          530.43  TiB
        Buffer Iterations:                      568.79k
        List Iterations:                        36.17m
        NULL List Iterations:                   974.71k

L2 ARC Writes:
        Writes Sent:                    100.00% 136.10k


Kernel variables showing compression is working on L2ARC, and there are I/O and
checksum errors:

root at cadence:/ # sysctl kstat.zfs.misc.arcstats.l2_compress_successes
kstat.zfs.misc.arcstats.l2_cksum_bad kstat.zfs.misc.arcstats.l2_io_error
kstat.zfs.misc.arcstats.l2_compress_successes: 1353514
kstat.zfs.misc.arcstats.l2_compress_zeros: 29
kstat.zfs.misc.arcstats.l2_compress_failures: 4985
kstat.zfs.misc.arcstats.l2_cksum_bad: 1290021
kstat.zfs.misc.arcstats.l2_io_error: 128275

Slight variations of this problem have been reported in numerous instances on
FreeBSD, FreeNAS and PC-BSD related forums and mailing lists, and is usually
dismissed as a hardware problem:


However, I've been able to easily duplicate this problem on two different sets
of high quality, reliable hardware (Dell PowerEdge, Intel SSD) that otherwise
tests perfectly.

To duplicate, simply create a zfs pool with a small L2ARC device and exercise
the pool with random I/O until the L2ARC fills.

You are receiving this mail because:
You are the assignee for the bug.

More information about the freebsd-bugs mailing list