xenstore memory issue

[Andrew Thompson]

On 11 February 2015 at 05:22, Roger Pau Monné <roger.pau at citrix.com> wrote:

> Hello Andrew,
>
> El 10/02/15 a les 10.22, Andrew Thompson ha escrit:
> > On 10 February 2015 at 16:25, Andrew Thompson <thompsa at freebsd.org>
> wrote:
> >
> >> On 9 February 2015 at 23:28, Roger Pau Monné <roger.pau at citrix.com>
> wrote:
> >> A bit of trial and error with dtrace has narrowed this down. I can cause
> >> the leak by just opening /dev/xen/xenstore
>
> Thanks for doing all this legwork! IMHO the best solution is to switch
> xenstore dev to use cdevpriv in order to store each client data. What we
> are doing right now (storing client data in dev->si_dvr1) is plain
> wrong. I've uploaded two patches (one for HEAD and one for stable/10) so
> that you can try it also, please report back whether this fixes your
> problem or not:
>
> https://people.freebsd.org/~royger/xenstore_fix/
>
>
I have tested this on 10.0 and it does fix the issue. After 36000
allocations there are just 6 active at 13K

xenstore     6    13K       -    36699  16,32,64,128,256,512


I looked in to why this only affected one of my many VMs as they run
xe-update-guest-attrs and nova-agent which should have triggered the leak.
They are all stuck on xsdread when reading from the xenstore so the update
loop has stopped.

  UID   PID  PPID CPU PRI NI    VSZ   RSS MWCHAN   STAT TT          TIME
COMMAND
    0 13250 13249   0  20  0  16552  1968 xsdread  S     -       5:57.60
/usr/local/bin/xenstore-read domid (xenstore)

Is there an issue with the xs_dev_read and xs_queue_reply logic or would
the Xen host be timing out?


regards,
Andrew