mysql on ZFS hangs

[Gergely CZUCZY]

Hello,

I'm trying to test a MySQL on ZFS setup, and I'm having some
troubles. At first I've copied all the mysql data files from the
UFS filesystem to the ZFS one, started mysqld and launched sysbench
at it.

On the UFS mysql was using around 100-180% CPU and did the benchmark
tasks pretty quick (in 60-120 seconds), and now it just stands there
for at least 10 minutes.

In top I see mysql in zfs state:
  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 6244 mysql       3  45    0  2071M   202M zfs:(& 1   0:47  0.98% mysqld

In gstat I see the ZFS is reading all both disks in the zmirror like
a madman, but mysql pretty does nothing it seems. mysqladmin says it
has around 23qps while on UFS it used to report 6-800qps.

vmstats shows if the system weren't on load entirely:
 r b w     avm    fre   flt  re  pi  po    fr  sr ad2 da0   in   sy   cs us sy id
 0 0 0 2242452 3224164   180   0   0   0  2862   0   0   0  407 1097 3299  0  5 95

FreeBSD sees 6G memory in this box (i've got 14G in it, I haven't had
the time to check yet why it doesn't see the rest. it used to see all the
memory).

FreeBSD sqltest.in.publishing.hu 7.0-BETA2 FreeBSD 7.0-BETA2 #3: Mon Nov  5 10:49:48 CET 2007     toor at sqltest.in.publishing.hu:/usr/obj/usr/src/sys/SQLTEST  amd64
mysql-server-5.0.45_1 Multithreaded SQL database (server)

  pool: zm
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        zm          ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da2     ONLINE       0     0     0

errors: No known data errors

Filesystem     Size    Used   Avail Capacity  Mounted on
zm              40G    128K     40G     0%    /zm
zm/qemu         40G    128K     40G     0%    /zm/qemu
zm/mysql        42G    2.0G     40G     5%    /zm/mysql
/zm/mysql       42G    2.0G     40G     5%    /var/db/mysql

(yes, i'm using it through a nullmount)

In mysql I'm using myisam (no chance for innodb). I think this behaviour
is quite abnormal.

Did anybody met this issue, is this known? Where should I start digging
around to solve this (tried google, not much help)?
Where can be the bottleneck that gives this whole thing? What have I done
wrong during the setup?

I need a bit of help to make this scenario work...

Sincerely,

Gergely Czuczy
mailto: gergely.czuczy at harmless.hu

-- 
Weenies test. Geniuses solve problems that arise.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 2234 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20071126/176fad6b/attachment.pgp