xenstore memory issue

[Andrew Thompson]

Hi,


I have three VMs with Rackspace and one is behaving oddly with xenstore
memory consumption. Here are the kernel versions and vmstat -m results.

FreeBSD us.e.com 10.0-RELEASE-p9 FreeBSD 10.0-RELEASE-p9 #0: Mon Sep 15
14:35:52 UTC 2014
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
 amd64

       xenbus    16     2K       -       86  16,32,64,256
     xenstore   409  4837K       - 38424052  16,32,64,128,256
      xen_hvm     2     8K       -        2  4096
     xen_intr    25     4K       -       25  128


FreeBSD uk.e.com 10.0-RELEASE-p12 FreeBSD 10.0-RELEASE-p12 #0: Tue Nov  4
05:07:17 UTC 2014
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
 amd64

       xenbus    11     2K       -       83  16,32,64,256
     xenstore   198  2317K       - 43428137  16,32,64,128,256,512
      xen_hvm     2     8K       -        2  4096
     xen_intr    24     3K       -       24  128


FreeBSD au.e.com 10.0-RELEASE-p12 FreeBSD 10.0-RELEASE-p12 #0: Tue Nov  4
05:07:17 UTC 2014
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
 amd64

       xenbus    11     2K       -       83  16,32,64,256
     xenstore  8477 101653K       -    55249  16,32,64,128,256,512
      xen_hvm     2     8K       -        2  4096
     xen_intr    14     2K       -       14  128


As you can see the third VM is using 100MB in xenstore memory and it seems
to be climbing by 1-2MB per hour. Eventually all the processes go in to
pfault state and it grinds to a halt.

How should I be debugging this? Is it either a local leak or the Xen host
is to blame?


cheers,
Andew