UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY

[Zaphod Beeblebrox]

On Mon, Sep 29, 2008 at 11:09 AM, Zaphod Beeblebrox <zbeeble at gmail.com>wrote:

>
> I certainly can't agree with this.  I don't think you're measuring the
> performance of the machine --- only measuring the performance of the
> filesystem.  When ZFS is active, memory is committed in the kernel to ZFS.
> That memory is then no longer available for paging.  Also, there exists data
> within the ARC (I'm always tempted to say the ARC Cache, but that is
> redundant) that is also then in paging memory.  You end up having to tune
> the size of the ARC to your workload, which is how UN*X did it upto 1992 or
> so.  If you choose the wrong size for the ARC, you end up with very poor
> performance.
>
> In particular, the ARC fights for space with the nvidia binary driver
> (which really does need _real_ memory).  To have the driver work at all, I
> have to keep the ARC from growing too large --- which is at odds with
> filesystem perofrmance.
>

I thought about this statement a bit, and I believe it might be good to give
a real world example.

I use spamprobe.  It's a dual-word basian spam filter.  It runs from
procmail as part of the local delivery process on my laptop for mail.  It
uses one of the berkley DB variants (I forget which at the moment and my
binary is statically linked so it will run on AMD64 as well as I386, so I
can't easily check).

I strongly suspect that the basic operation of spamprobe involves mmap()ing
the file (80 to 100 meg), performing a flurry of lookups (words in the
message) followed by a flurry of updates, followed by closing the file.  Now
my current laptop has ZFS for my home directories and that ZFS is backed by
two 320G laptop drives (RAID 1).  My old laptop has UFS on a single 160G
laptop drive.  The old laptop would deliver 10 to 20 email messages per
second through spamprobe.  The new laptop deilvers a mail message every 2 to
5 seconds.

In the UFS case, there would be little disk activity.  Running spamprobe
would only involved cached items and the disk activity would be the
occaisional write to sync the new data and writes to deposit each email
message.

Now there's some things I don't know.  Is the whole file rewritten to sync
the changes?  It seems like it is.  Whatever the difference between UFS and
ZFS here, it's triggering some pathological behavior.

The needs of NAS are not the needs of a local filesystem.  They are
different beasts.  If ZFS is optimized for being a NAS store, it does this
well.  Maybe UFS can become more ZFS like or vice-versa.  ZFS is a
comparatively young filesystem (and UFS is an equally old one).