Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

[Mark Felder]

On Thu, 29 Mar 2012 19:27:31 -0500, Joe Greco <jgreco at ns.sol.net> wrote:

> It also doesn't explain the experience here, where one VM basically
> crapped out but only after a migration - and then stayed crapped out.
> It would be interesting to hear about your datastore, how busy it is,
> what technology, whether you're using thin, etc.  I just have this real
> strong feeling that it's some sort of corruption with the vmfs3 and thin
> provisioned disk format, but it'd be interesting to know if that's
> totally off-track.

We've ruled out SAN, but we haven't ruled out VMFS. Even FreeBSD Guests on  
standalone ESXi servers with no SAN exhibit this crash.

For the record, we only use thick provisioning and if it was corruption  
I'm not sure what layer the corruption could be at. The crashy servers  
show no abnormalities when I run either `freebsd-update IPS` or  
`pkg_libchk` to confirm checksums of all installed programs. Now the other  
data on there... it's not exactly verified, but our backups via rsnapshot  
seem to prove there is no issue there or we'd have lots of new files each  
run.