7.2 dies in zfs

[Svein Skogen (listmail account)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Adam McDougall wrote:
> On Sat, Nov 21, 2009 at 11:36:43AM -0800, Jeremy Chadwick wrote:
> 
>   
>   On Sat, Nov 21, 2009 at 08:07:40PM +0100, Johan Hendriks wrote:
>   > Randy Bush <randy at psg.com> wrote:
>   > > imiho, zfs can not be called production ready if it crashes if you
>   > > do not stand on your left leg, put your right hand in the air, and
>   > > burn some eye of newt.
>   > 
>   > This is not a rant, but where do you read that on FreeBSD 7.2 ZFS has
>   > been marked as production ready.
>   > As far as i know, on FreeBSD 8.0 ZFS is called production ready.
>   > 
>   > If you boot your system it probably tell you it is still experimental.
>   > 
>   > Try running FreeBSD 7-Stable to get the latest ZFS version which on
>   > FreeBSD is 13
>   > On 7.2 it is still at 6 (if I remember it right).
>   
>   RELENG_7 uses ZFS v13, RELENG_8 uses ZFS v18.
>   
>   RELENG_7 and RELENG_8 both, more or less, behave the same way with
>   regards to ZFS.  Both panic on kmem exhaustion.  No one has answered my
>   question as far as what's needed to stabilise ZFS on either 7.x or 8.x.
> 
> I have a stable public ftp/http/rsync/cvsupd mirror that runs ZFS v13.
> It has been stable since mid may.  I have not had a kmem panic on any
> of my ZFS systems for a long time, its a matter of making sure there is
> enough kmem at boot (not depending on kmem_size_max) and that it is big enough
> that fragmentation does not cause a premature allocation failure due to lack
> of large-enough contiguous chunk.  This requires the platform to support a
> kmem size that is "big enough"... i386 can barely muster 1.6G and sometimes
> that might not be enough.  I'm pretty sure all of my currently existing ZFS
> systems are amd64 where the kmem can now be huge.  On the busy fileserver with
> 20 gigs of ram running FreeBSD 8.0-RC2 #21: Tue Oct 27 21:45:41 EDT 2009,
> I currently have:
> vfs.zfs.arc_max=16384M
> vfs.zfs.arc_min=4096M
> vm.kmem_size=18G
> The arc settings here are to try to encourage it to favor the arc cache
> instead of whatever else Inactive memory in 'top' contains.

Very interesting. For my iscsi backend (running istgt from ports), I had
to change the arc_max below 128M to stop iSCSI initiators generating
timeouts when the cache flushed. (This is on a system with a megaraid
8308ELP handling the disk back end, with the disks in two RAID5 arrays
of four disks each, zpooled as one big pool).

When I had more than 128M arc_max, zfs on regular times ate all
available resources to flush to disk, leaving the istgt waiting, and
iSCSI initiators timed out and had to reconnect. The iSCSI initiators
are the built-in software initator in VMWare ESX 4i.

//Svein

- --
- --------+-------------------+-------------------------------
  /"\   |Svein Skogen       | svein at d80.iso100.no
  \ /   |Solberg Østli 9    | PGP Key:  0xE5E76831
   X    |2020 Skedsmokorset | svein at jernhuset.no
  / \   |Norway             | PGP Key:  0xCE96CE13
        |                   | svein at stillbilde.net
 ascii  |                   | PGP Key:  0x58CD33B6
 ribbon |System Admin       | svein-listmail at stillbilde.net
Campaign|stillbilde.net     | PGP Key:  0x22D494A4
        +-------------------+-------------------------------
        |msn messenger:     | Mobile Phone: +47 907 03 575
        |svein at jernhuset.no | RIPE handle:    SS16503-RIPE
- --------+-------------------+-------------------------------
         If you really are in a hurry, mail me at
               svein-mobile at stillbilde.net
 This mailbox goes directly to my cellphone and is checked
        even when I'm not in front of my computer.
- ------------------------------------------------------------
                     Picture Gallery:
          https://gallery.stillbilde.net/v/svein/
- ------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksI/ZMACgkQODUnwSLUlKRr6gCfeq5dybIfp5RLOzjL04guLV25
+qgAn04SjnGG3lBRExQaMjxyKcd9Jcct
=ubYi
-----END PGP SIGNATURE-----