Proposal for better support of hypervisors and their synthetic drivers at boot-time

[Alexander Motin]

On 23.04.2013 17:07, Larry Melia wrote:
> I’m on a team of developers working on synthetic device drivers for
> FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list)
> that we're trying to get our drivers included with the FreeBSD
> distribution. Also noted were some necessary patches to the ATA driver.
> The changes are necessary to achieve significant performance gains by
> replacing the native ATA driver with our synthetic storage driver
> when when Hyper-V is detected. Alexander Motin, the maintainer of the
> ATA code-base, however, expressed some concerns about making these
> modifications that are so specific to Hyper-V and the AMD64 build. We
> understand his concerns and have subsequently removed these patches from
> our vendor branch. Our drivers are now completely independent and
> require no changes to the kernel or other drivers. The only touch-point
> is the common makefile for modules/drivers.
>
> Removing our ATA patches, on the other hand, results in a huge
> performance loss. This is because the root file system is managed by the
> ATA driver, which is emulated under Hyper-V. Furthermore, there may be
> other native drivers that we wish to replace with synthetic ones at boot
> time (e.g., mouse driver, video driver), but, there appears to be no
> easy way to do this.Therefore, we would like to work with the developer
> community to come up with better solution that would improve support for
> synthetic drivers at boot-time.
>
> One possible solution:
>
> (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is
> used for hypervisor detection, to an earlier point in the boot process.
> Presently, it appears to be called after the ATA driver is selected,
> which is too late in the boot process. (This was discovered after some
> testing with the ATA driver.) Therefore, before the bus drivers and
> native controllers are detected and selected, discovery of a host
> hypervisor should be done first.
>
> (2) Extend the boot process to dynamically insert and prioritize
> synthetic drivers over native ones. Presently, FreeBSD supports the
> selection of generic drivers for a given device, but these can be
> overridden when a more specific driver is available. This priority
> scheme could be extended by first detecting the presence of a
> hypervisor, then allowing this priority mechanism to override the native
> drivers with their synthetic cousins (i.e., we only want to override a
> native driver when a specific hypervisor is detected and synthetic
> drivers for it have been configured). This could be arranged in a
> separate, table-driven, input file or perhaps, by extending the existing
> driver/module configuration file with additional mappings of native to
> synthetic drivers.
>
> (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use
> the more recent approach to hypervisor detection. This approach uses the
> CPU-identify functions to retrieve a unique signature consisting of a
> fixed string of ASCII characters. This was done on Linux about five
> years. For backward compatibility, however, the existing logic would be
> retained, but augmented with this new approach. It would also be
> conditionally added only for x86/AMD64 builds.
>
> When reviewing the changes we're not looking at a lot of lines of
> code.The difficultly lies, however, in where the changes are made.
> Obviously, we need to work closely with the committers responsible for
> the kernel modules we're looking to touch. Retrospectively, these
> upgrades only bring FreeBSD up to the same level of hypervisor support
> already present in Linux.There are other issues that we would also like
> to discuss, but the improvements described here are on the top of the list.
>
> We look forward to working more closely with the FreeBSD community and
> welcome your remarks and feedback on this proposal.

I am sorry, I don't really understand what is your proposition, so I 
just describe my vision of the solution here, hoping it will be useful.

You are right that FreeBSD has priority mechanism to select best driver 
for each device. That mechanism should be sufficient to prevent default 
OS driver from attaching to emulated ATA controller when Hyper-V 
environment is detected and paravirtual drivers are installed. To do 
that you should just create a very small dummy driver for PCI bus, 
implementing single probe() method. That probe() method should in your 
preferable way check that Hyper-V environment is active and match PCI ID 
of the device against the list of used by Hyper-V for emulated ATA 
controller. If both conditions match, the probe() method should return 
highest priority value to override any other ATA driver in system for 
this specific device. After that the dummy driver should just have empty 
attach() and detach() routines, not doing anything with the emulated 
device, relying on paravitrual driver to handle the disks.

Since the dummy driver can be made machine-dependent, you may use there 
any VM detection mechanisms you like, even direct MSR access. Dummy 
driver can be made completely independent from the rest of the system, 
so it can even be loaded as kernel module without requiring any base 
system modifications. If you still prefer it for some reason (for 
example to not do some complicated checks multiple times in multiple 
drivers), you may move Hyper-V detection logic out of that driver to 
some single place in base system. But you don't need to touch OS-native 
ATA driver in any way.

-- 
Alexander Motin