Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Fri, 30 Mar 2012 11:53:10 -0500, Joe Greco <jgreco at ns.sol.net> wrote:

> On the same vmdk files?  "Deleting the VM" makes it sound like not.

Fresh new VMDK files every time, and always thick provisioned.

> None of the other VM's, even the VM's that had been abused in this
> horribly insensitive manner of being placed on intolerably slow iSCSI,
> developed this condition.

We've seen similar results. Baffling how VMs you know are worse off never  
develop this condition.

> There are dozens of other VM's running on these hosts, alongside the
> one that was exhibiting this behaviour.
> The VM continued to exhibit this behaviour even after having been moved
> onto a different ESXi platform and architecture (Opteron->Xeon).
> For the problem to "follow" the VM in this manner, and afflict *only*
> the one VM, strongly suggests that it is something that is contained
> within the VM files that constitute this VM.  That is consistent with
> the observation that the problem arose at a point where the VM is
> known to have had all those files moved from one location to a dodgy
> location.

We were hoping that was the explanation as well, but rebuilding the VM  
entirely from scratch on a new host and seeing the crash come back was a  
big blow to morale. :(

> That's why I believe the evidence points to corruption of some sort.
> Of course, your case makes this all interesting.

For the last year I've been convinced it's something hidden inside ESXi's  
I/O virtualization layer that happens to trigger on only certain VMs.