Proposal for better support of hypervisors and their synthetic drivers at boot-time

[Larry Melia]

I’m on a team of developers working on synthetic device drivers for FreeBSD
on Hyper-V. In late March, I briefly mentioned (on this list) that we're
trying to get our drivers included with the FreeBSD distribution. Also
noted were some necessary patches to the ATA driver. The changes are
necessary to achieve significant performance gains by replacing the native
ATA driver with our synthetic storage driver when when Hyper-V is detected.
Alexander Motin, the maintainer of the ATA code-base, however, expressed
some concerns about making these modifications that are so specific to
Hyper-V and the AMD64 build. We understand his concerns and have
subsequently removed these patches from our vendor branch. Our drivers are
now completely independent and require no changes to the kernel or other
drivers. The only touch-point is the common makefile for modules/drivers.

Removing our ATA patches, on the other hand, results in a huge performance
loss. This is because the root file system is managed by the ATA driver,
which is emulated under Hyper-V. Furthermore, there may be other native
drivers that we wish to replace with synthetic ones at boot time (e.g.,
mouse driver, video driver), but, there appears to be no easy way to do
this.Therefore, we would like to work with the developer community to come
up with better solution that would improve support for synthetic drivers at
boot-time.

One possible solution:

(1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is used
for hypervisor detection, to an earlier point in the boot process.
Presently, it appears to be called after the ATA driver is selected, which
is too late in the boot process. (This was discovered after some testing
with the ATA driver.) Therefore, before the bus drivers and native
controllers are detected and selected, discovery of a host hypervisor
should be done first.

(2) Extend the boot process to dynamically insert and prioritize synthetic
drivers over native ones. Presently, FreeBSD supports the selection of
generic drivers for a given device, but these can be overridden when a more
specific driver is available. This priority scheme could be extended by
first detecting the presence of a hypervisor, then allowing this priority
mechanism to override the native drivers with their synthetic cousins
(i.e., we only want to override a native driver when a specific hypervisor
is detected and synthetic drivers for it have been configured). This could
be arranged in a separate, table-driven, input file or perhaps, by
extending the existing driver/module configuration file with additional
mappings of native to synthetic drivers.

(3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use the
more recent approach to hypervisor detection. This approach uses the
CPU-identify functions to retrieve a unique signature consisting of a fixed
string of ASCII characters. This was done on Linux about five years. For
backward compatibility, however, the existing logic would be retained, but
augmented with this new approach. It would also be conditionally added only
for x86/AMD64 builds.

When reviewing the changes we're not looking at a lot of lines of code.The
difficultly lies, however, in where the changes are made. Obviously, we
need to work closely with the committers responsible for the kernel modules
we're looking to touch. Retrospectively, these upgrades only bring FreeBSD
up to the same level of hypervisor support already present in Linux.There
are other issues that we would also like to discuss, but the improvements
described here are on the top of the list.

We look forward to working more closely with the FreeBSD community and
welcome your remarks and feedback on this proposal.

Larry Melia