virtualbox-4.1.10: Xorg CPU usage is 100% after Ubuntu guest is hanging idle for a while

[Yuri]

I observed this few times after guest was totally idle for hours. Other 
X programs become very sluggish.
Closing the guest lowers xorg CPU to <10%. 3D acceleration option is "off".

It doesn't look like this happens every time, something triggers this.

virtualbox-ose-4.1.10
NVidia driver, amd64
9.0

Yuri