ACPI/power implementation causing performance loss with i7/Nehalem turbo boost

[Kevin Day]

Recently I bumped into something very weird. In some CPU heavy workloads, FreeBSD ran faster inside VMware's ESX hypervisor than it did running natively on bare metal. Simple pure CPU applications (such as "openssl speed") would run 10-30% faster on VMware. This seemed very counterintuitive, until I discovered what I believe to be the cause. 

Intel Nehalem and i5/i7 processors have a feature called "Turbo Boost", where the more cores that are inactive (ACPI states C2 or C3) the higher the clock rate of the active cores. In some processors increasing the clock speed by more than 1ghz. On a hunch, I disabled turbo boost (through the BIOS) on our ESX system, and this brought the speeds back on par with the bare metal FreeBSD box.

So, it seems that the VMware hypervisor is deactivating cores on the CPU when idle, but FreeBSD itself isn't. Is anyone working on giving FreeBSD's idle loop/scheduler the ability to go into deeper sleep states? It seems this would have more than just a power savings benefit now.

Intel documentation on Turbo Boost: http://download.intel.com/design/processor/applnots/320354.pdf?iid=tech_tb+paper

-- Kevin