ZFS honesty
Kris Kennaway
kris at FreeBSD.org
Sun Jan 6 13:32:48 PST 2008
Scott Long wrote:
> Kris Kennaway wrote:
>> Ivan Voras wrote:
>>> Kris Kennaway wrote:
>>>> Ivan Voras wrote:
>>>>> Robert Watson wrote:
>>>>>
>>>>>> I'm not sure if anyone has mentioned this yet in the thread, but
>>>>>> another thing worth taking into account in considering the
>>>>>> stability of ZFS is whether or not Sun considers it a production
>>>>>> feature in Solaris. Last I heard, it was still considered an
>>>>>> experimental feature there as well.
>>>>>
>>>>> Last I heard, rsync didn't crash Solaris on ZFS :)
>>>>
>>>> [Citation needed]
>>>
>>> I can't provide citation about a thing that doesn't happen - you
>>> don't hear things like "oh and yesterday I ran rsync on my Solaris
>>> with ZFS and *it didn't crash*!" often.
>>>
>>> But, with some grains of salt taken, consider this Google results:
>>>
>>> * searching for "rsync crash solaris zfs": 790 results, most of them
>>> obviously irrelevant
>>> * searching for "rsync crash freebsd zfs": 10,800 results; a small
>>> number of the results is from this thread, some are duplicates, but
>>> it's a large number in any case.
>>>
>>> I feel that the number of Solaris+ZFS installations worldwide is
>>> larger than that of FreeBSD+ZFS and they've had ZFS longer.
>>
>> Almost all Solaris systems are 64 bit.
>>
>> Kris
>
> So, let's be honest here. ZFS is simply unreliable on FreeBSD/i386.
> There are things that you can do mitigate the problems, and in certain
> well controlled environments you might be able to make it work well
> enough for your needs. But as a general rule, don't expect it to work
> reliably, period. This is backed up by Sun's own recommendation to not
> run it on 32-bit Solaris.
>
> But let's also be honest about ZFS in the 64-bit world. There is ample
> evidence that ZFS basically wants to grow unbounded in proportion to the
> workload that you give it. Indeed, even Sun recommends basically
> throwing more RAM at most problems. Again, tuning is often needed, and
> I think it's fair to say that it can't be expected to work on arbitrary
> workloads out of the box.
>
> Now, what about the other problems that have been reported in this
> thread by Ivan and others? I don't think that it can be said that the
> only problem that ZFS has is with memory. Unfortunately, it looks like
> these "other" problems aren't well quantified, so I think that they are
> being unfairly dismissed. But at the same time, maybe these other
> problems are rare and unique enough that they represent very special
> cases that won't be encountered by most people. But it also tells me
> that ZFS is still immature, at least in FreeBSD.
>
> The universal need for tuning combined with the poorly understood
> problem reports tells me that administrators considering ZFS should
> expect to spend a fair amount of timing testing and tuning. Don't
> expect it to work out of the box for your situation. That's not to
> say that it's useless; there are certainly many people who can attest to
> it working well for them. Just be prepared to spend time and possibly
> money making it work, and be willing to provide good problem reports for
> any non-memory related problems that you encounter.
To be clear, in this thread I have been mostly restricting myself to
discussion of kmem problems only, although I have also noted that there
are known ZFS bugs including bugs that are unfixed even in solaris (the
ZIL low memory deadlock is one of them). Indeed, pjd has a long list of
bug reports from me :)
I agree with the rest of this summary.
Kris
More information about the freebsd-current
mailing list