11.2-STABLE kernel wired memory leak

[Eugene Grosbein]

Hi!

Long story short: 11.2-STABLE/amd64 r335757 leaked over 4600MB kernel wired memory over 81 days uptime
out of 8GB total RAM.

Details follow.

I have a workstation running Xorg, Firefox, Thunderbird, LibreOffice and occasionally VirtualBox for single VM.

It has two identical 320GB HDDs combined with single graid-based array with "Intel"
on-disk format having 3 volumes:
- one "RAID1" volume /dev/raid/r0 occupies first 10GB or each HDD;
- two "SINGLE" volumes /dev/raid/r1 and /dev/raid/r2 that utilize "tails" of HDDs (310GB each).

/dev/raid/r0 (10GB) has MBR partitioning and two slices:
- /dev/raid/r0s1 (8GB) is used for swap;
- /dev/raid/r0s2 (2GB) is used by non-redundant ZFS pool named "os" that contains only
root file system (177M used) and /usr file system (340M used).

There is also second pool (ZMIRROR) named "z" built directly on top of /dev/raid/r[12] volumes,
this pool contains all other file systems including /var, /home, /usr/ports, /usr/local, /usr/{src|obj} etc.

# zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
os    1,98G   520M  1,48G        -         -    55%    25%  1.00x  ONLINE  -
z      288G  79,5G   209G        -         -    34%    27%  1.00x  ONLINE  -

This way I have swap outside of ZFS, boot blocks and partitioning mirrored by means of GEOM_RAID and
can use local console to break to single user mode to unmount all file system other than root and /usr
and can even export bigger ZFS pool "z". And I did that to see that ARC usage
(limited with vfs.zfs.arc_max="3G" in /boot/loader.conf) dropped from over 2500MB 
down to 44MB but "Wired" stays high. Now after I imported "z" back and booted to multiuser mode
top(1) shows:

last pid: 51242;  load averages:  0.24,  0.16,  0.13  up 81+02:38:38  22:59:18
104 processes: 1 running, 103 sleeping
CPU:  0.0% user,  0.0% nice,  0.4% system,  0.2% interrupt, 99.4% idle
Mem: 84M Active, 550M Inact, 4K Laundry, 4689M Wired, 2595M Free
ARC: 273M Total, 86M MFU, 172M MRU, 64K Anon, 1817K Header, 12M Other
     117M Compressed, 333M Uncompressed, 2.83:1 Ratio
Swap: 8192M Total, 940K Used, 8191M Free

I have KDB and DDB in my custom kernel also. How do I debug the leak further?

I use nvidia-driver-340-340.107 driver for GK208 [GeForce GT 710B] video card.
Here are outputs of "vmstat -m": http://www.grosbein.net/freebsd/leak/vmstat-m.txt
and "vmstat -z": http://www.grosbein.net/freebsd/leak/vmstat-z.txt
as well as "sysctl hw": http://www.grosbein.net/freebsd/leak/sysctl-hw.txt
and "sysctl vm": http://www.grosbein.net/freebsd/leak/sysctl-vm.txt