lightly loaded system eats swap space

tech-lists tech-lists at
Sun Jun 17 22:19:05 UTC 2018

Hello list,

context is (server)
freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, 
E5-2630 @2.3GHz, generic kernel.

There's one bhyve guest on this server (using 4x cpu and 16GB RAM, also 

There have been no special options for zfs configuration on the server, 
apart from several datasets having the compressed property set (lz4).

The server runs nothing else really apart from sshd and it uses ntpd to 
sync local time.

How come such a lightly loaded server with plenty of resources is eating 
up swap? If I run two bhyve instances, i.e. two of the same size as 
indicated above, so 32GB used for the bhyves, I'll get out-of-swapspace 
errors in the daily logs:

+swap_pager_getswapspace(24): failed
+swap_pager_getswapspace(24): failed
+swap_pager_getswapspace(24): failed

Here's top, with one bhyve instance running:

last pid: 49494;  load averages:  0.12,  0.13,  0.88 
                              up 29+11:36:06  22:52:45
54 processes:  1 running, 53 sleeping
CPU:  0.4% user,  0.0% nice,  0.4% system,  0.3% interrupt, 98.9% idle
Mem: 8664K Active, 52M Inact, 4797M Laundry, 116G Wired, 1391M Buf, 
4123M Free
ARC: 108G Total, 1653M MFU, 105G MRU, 32K Anon, 382M Header, 632M Other
      103G Compressed, 104G Uncompressed, 1.00:1 Ratio
Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse

49491 root          1   4    0 16444K 12024K select  9   0:12   6.49% ssh
32868 root         12  20    0  9241M  4038M kqread  2  23.2H   1.30% bhyve
49490 root          1  20    0 10812K  6192K sbwait  5   0:02   0.88% sftp

 From the looks of it, a huge amount of ram is wired. Why is that, and 
how would I debug it?

A server of similar spec which is running freebsd-current with seven 
bhyve instances doesn't have this issue:

last pid: 41904;  load averages:  0.26,  0.19,  0.15 
                              up 17+01:06:11  23:14:13
27 processes:  1 running, 26 sleeping
CPU:  0.1% user,  0.0% nice,  0.3% system,  0.0% interrupt, 99.6% idle
Mem: 17G Active, 6951M Inact, 41G Laundry, 59G Wired, 1573M Buf, 1315M Free
ARC: 53G Total, 700M MFU, 52G MRU, 512K Anon, 182M Header, 958K Other
      53G Compressed, 69G Uncompressed, 1.30:1 Ratio, 122M Overhead
Swap: 35G Total, 2163M Used, 33G Free, 6% Inuse


More information about the freebsd-stable mailing list