Is swap over ZFS safe on FreeBSD 9?

[Borja Marcos]

Hi

I decided to give another try to swap over ZFS. I've created a volume, added it as swap, and I've tried something crazy:

make -j 128 buildworld

The system is hung, it still responds to ping and establishes TCP connections, but nothing else. 

last pid: 70437;  load averages: 16.19, 12.86,  8.27    up 0+04:57:12  16:54:32
203 processes: 1 running, 202 sleeping
CPU:  0.0% user,  0.0% nice,  0.1% system,  0.1% interrupt, 99.9% idle
Mem: 6409M Active, 83M Inact, 1421M Wired
Swap: 32G Total, 45M Used, 32G Free
Timeout, server not responding.
HR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
 1790 borjam      1  47    0 16700K  1448K CPU2    2   1:44  9.81% top
 1729 borjam      1  48    0 35604K  1064K pfault 10   1:25  8.59% zpool
70436 root        1  63    0   178M   168M pfault  1   0:06  4.78% cc1plus
70316 root        1  63    0   195M   186M pfault  0   0:06  4.45% cc1plus
70370 root        1  63    0   173M   162M pfault  7   0:05  4.28% cc1plus
70300 root        1  63    0   207M   196M pfault  7   0:05  4.17% cc1plus
70389 root        1  63    0   179M   168M pfault  2   0:05  4.12% cc1plus
70390 root        1  63    0   185M   174M pfault 11   0:05  4.01% cc1plus
70297 root        1  62    0   194M   184M pfault  2   0:05  4.01% cc1plus
70421 root        1  62    0   168M   159M pfault  9   0:05  3.84% cc1plus
70379 root        1  62    0   195M   184M pfault  9   0:04  3.84% cc1plus
70372 root        1  62    0   189M   175M pfault  4   0:04  3.79% cc1plus
70384 root        1  62    0   155M   146M pfault  2   0:04  3.62% cc1plus
70288 root        1  62    0   179M   162M pfault  3   0:04  3.51% cc1plus
70382 root        1  62    0   160M   149M pfault 11   0:04  3.40% cc1plus
70430 root        1  62    0   158M   148M pfault  8   0:04  3.35% cc1plus
70313 root        1  62    0   193M   183M pfault 10   0:04  3.29% cc1plus


Lots of processes are in "pfault" state. I tried to disable "sync", primarycache and secondarycache for the swap volume, but without success.

Testing again,
last pid: 88772;  load averages:  0.00,  0.79,  2.73    up 0+00:28:14  17:34:15
186 processes: 1 running, 184 sleeping, 1 waiting
CPU:  0.0% user,  0.0% nice,  0.1% system,  0.0% interrupt, 99.9% idle
Mem: 6711M Active, 683M Inact, 519M Wired
Swap: 32G Total, 116M Used, 32G Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
 3839 root        1  40    0 16700K  1412K CPU0    0   0:10  0.00% top
88692 root        1  53    0   259M   247M pfault  2   0:07  0.00% cc1plus
88729 root        1 -20    0   199M   182M swread  2   0:07  0.00% cc1plus
88750 root        1  52    0   166M   154M pfault  2   0:06  0.00% cc1plus
88763 root        1  52    0   226M   211M pfault  8   0:06  0.00% cc1plus
88768 root        1 -20    0   206M   191M swread  6   0:06  0.00% cc1plus
88637 root        1 -20    0   218M   202M swread 11   0:06  0.00% cc1plus
88643 root        1  52    0   219M   204M pfault  7   0:06  0.00% cc1plus
88657 root        1 -20    0   225M   210M swread  4   0:06  0.00% cc1plus
88748 root        1 -20    0   232M   217M swread  3   0:06  0.00% cc1plus
88771 root        1 -20    0   235M   220M swread  6   0:06  0.00% cc1plus
88701 root        1  52    0   218M   203M pfault  3   0:06  0.00% cc1plus
88749 root        1  52    0   219M   205M pfault  1   0:05  0.00% cc1plus
88682 root        1 -20    0   201M   189M swread  0   0:05  0.00% cc1plus
88711 root        1 -20    0   240M   227M swread  2   0:05  0.00% cc1plus
88696 root        1  52    0   232M   218M pfault  5   0:05  0.00% cc1plus
88664 root        1 -20    0   222M   207M swread  5   0:05  0.00% cc1plus






Any ideas? I guess I better stay away from swap on ZFS for now.





Borja.