Generic Kernel API

[Chuck Swiger]

Scott Long wrote:
> Charles Swiger wrote:
[ ... ]
> I have a fair amount of very close experience with the OSX kernel.  See
> my comment below:

I'd say that you have some experience with the FreeBSD kernel, too.  :-)

>> I'm not strongly advocating the use of C++ in the kernel, but Apple  
>> is using g++ to build their kernels, so I'd imagine that FreeBSD  
>> could utilize the same embedded C++ dialect in our kernels if people  
>> wanted to do so.  The things that leapt out at me in comparing the  
>> FreeBSD APIs and IOKit were:
> 
> A cut down version of C++ is used for IOKit, it is not used for the 
> whole kernel.  The large majority of the kernel is written in C, not
> C++.

Agreed.

> Not all kernel modules are hardware device drivers, neither in
> OSX or in FreeBSD.  GEOM modules, filesystems, and netgraph modules are
> all valid examples of pseudo drivers that benefit from a stable API but
> do not represent hardware devices.  So IOKit is not the cure-all API.

Goodness, no.  In some ways, I actually like FreeBSD's C implementation of 
device_t's using kobj's quite a bit compared to C++ code in the IOKit, and some 
driver families (sound in particular) seem to take advantage of inheritence 
more than other drivers do.

The IOKit has some C++-related blemishes like:

3-pan% tail 
/System/Library/Frameworks/IOKit.framework/Versions/A/Headers/network/IONetworkController.h
     OSMetaClassDeclareReservedUnused( IONetworkController, 28);
     OSMetaClassDeclareReservedUnused( IONetworkController, 29);
     OSMetaClassDeclareReservedUnused( IONetworkController, 30);
     OSMetaClassDeclareReservedUnused( IONetworkController, 31);
};

#endif /* defined(KERNEL) && defined(__cplusplus) */

#endif /* !_IONETWORKCONTROLLER_H */

...reserving 32 slots and keeping a pointer variable to an undefined struct 
(*_unused) handy just in case due to the fragile base class issue.

>> 1) the notion of a system-wide driver registry, which could be  
>> obtained easily from the existing code in sys/bus.h & kern/subr_bus.c  
>> which keeps track of this:
>>
>> typedef TAILQ_HEAD(driver_list, driverlink) driver_list_t;
>>
>> [ devclass_get_devices() is close but not quite the same thing... ]
> 
> There is already a module registry.  It's used to know when to reject
> loading KLDs that contain modules that are already in the system.  This
> works for both device drivers and pseudo drivers.

True, but a list of modules was not quite was I was looking for.

>> 2) the "work loop" abstraction (long link, again):
>>
>> http://developer.apple.com/documentation/DeviceDrivers/Conceptual/ 
>> IOKitFundamentals/HandlingEvents/chapter_8_section_2.html
>>
>> Programming using callbacks or continuations, having to serialize  
>> access to driver data structures, etc is one of the most difficult  
>> areas to deal with, and race conditions and so forth are a common  
>> source of evil, tricky, hard-to-reproduce bugs.  There isn't a free  
>> lunch, the kernel has got to deal with such things, but having an  
>> abstraction like this would probably help make the lives of people  
>> writing drivers easier. [1]
> 
> I've written an IOKit driver for high performance hardware.  I'm not
> convinced that the work loop paradigm is any more efficient than
> locking.  Apple advocates it because it is indeed easier to program to
> and takes less to explain than using the different locking primitives.

The IOKit provides relatively fine-grain mutex locking (on the class or 
instance level of driver objects) and supports re-entrancy:

"An IOWorkLoop object (or simply, a work loop) is primarily a gating mechanism 
that ensures single-threaded access to the data structures used by hardware. 
For some event contexts, a work loop is also a thread. In essence, a work loop 
is a mutually exclusive (mutex) lock associated with a thread."

...while providing a API (or KPI) which lets the developer code as if he or she 
had a single worker thread, even though underneath, the system may be 
scheduling many worker threads amoungst the available CPUs and/or event sources.

Certainly that's better (more efficient) than contending over the GIANT lock.

> Remember that the target audience for much of the Apple documentation is
> people who have never programmed in a Unix kernel before, be they coming
> from Windows or coming from OS9.  In fact, the Apple docs go out of 
> their way to discourage you from writing kernel modules entirely.

Sure-- don't you agree that anything which can be done in userland, generally 
ought to be done there?  Apple has to contend with developers who are looking 
to hook into the vertical blanking handler for screensavers and clock programs 
and who knows what else, just like they did in OS 9.  Discouraging such things 
from going into the kernel is a good idea.

Also remember that Mach is closer to being a microkernel than the other BSD 
kernels are, and the philosophy is showing in the design.  That doesn't mean 
it's always the best approach, but Mach feels more consistent to me.

>> 3) the IOMemoryDescriptor and IOMemoryCursor classes, which provide  
>> an abstraction for managing virtual memory mappings and representing  
>> DMA or PIO activity (ie, building a scatter/gather list appropriate  
>> for a particular NIC or RAID controller's DMA engine):
>>
>> http://developer.apple.com/documentation/DeviceDrivers/Conceptual/ 
>> IOKitFundamentals/DataMgmt/chapter_9_section_5.html
> 
> There is already a well established and stable API for doing DMA in 
> FreeBSD.  Just about every driver in the kernel uses it.  Why change?

You mean isa_dmacascade(), isa_dma_acquire(), isa_dmainit() and bus_dma_*...?

Eww.

The forces of entropy are winning the fight to keep the ISA bus and DMA bounce 
buffers which must be less than 64K around forever, even on hardware which 
doesn't have such limitations.  :-)

> There are good ideas in the IOKit that I've advocated for FreeBSD in the
> past (interrupt filters, for example), and the object oriented approach
> there is certainly interesting, but I don't see it as a cure all to 
> stability or ease.

The IOKit isn't a cure-all, nor is an OO viewpoint always the best approach. 
There isn't too much difference between inheriting the right behavior and 
having stuff like this in every driver:

static device_method_t mypci_methods[] = {
     /* Device interface */
     DEVMETHOD(device_probe,     mypci_probe),
     DEVMETHOD(device_attach,    mypci_attach),
     DEVMETHOD(device_detach,    mypci_detach),
     DEVMETHOD(device_shutdown,  mypci_shutdown),
     DEVMETHOD(device_suspend,   mypci_suspend),
     DEVMETHOD(device_resume,    mypci_resume),
     { 0, 0 }
};

On the other hand, using inheritence for drivers seems to work pretty well in 
practice, and the notion of encapsulation seems to help Darwin avoid running 
into nearly as many lock-order reversals and layering violations.

-- 
-Chuck