[RFC] shipping kernels with default modules?

[David Schultz]

On Sat, Jun 11, 2011, mdf at freebsd.org wrote:
> On Sat, Jun 11, 2011 at 10:18 AM, David Schultz <das at freebsd.org> wrote:
> > On Sat, Jun 11, 2011, Adrian Chadd wrote:
> >> Hi guys,
> >>
> >> Has there been any further thought as of late about shipping kernels
> >> with modules only by default, rather than monolithic kernels?
> >>
> >> I tried this experiment a couple years ago and besides a little
> >> trickery with ACPI module loading, it worked out fine.
> >>
> >> Is there any reason we aren't doing this at the moment? Eg by having a
> >> default loader modules list populated from the kernel config file?
> >
> > I've been doing this for years, and it has come in quite handy.
> > For instance, when my if_msk gets wedged, the only way to fix it
> > short of rebooting seems to be reloading the driver.
> >
> > One issue, however, is that the boot loader is horrendously slow
> > at loading modules.  (Either that or my BIOS has a braindead int 13h
> > handler.)  Most of these modules aren't actually needed until much
> > later in the boot process, so a mechanism to load non-essential
> > modules after the file systems are mounted might provide a good
> > solution.
> 
> Indeed, at $WORK we're trying to get shutdown -> restart under 2
> minutes.  Several seconds of this is moving things *into* the kernel
> that need to be there (disk drivers), and everything else to a point
> in init where modules can be loaded in parallel, using the faster disk
> driver, rather than in serial with slow BIOS handlers.

Have you found that drivers can be reliably loaded in parallel
these days?  I'm always waiting for timeouts on four card readers
and two optical drives, so that would be a big win for me.  IIRC,
nothing can happen in parallel during boot because the scheduler
is initialized very late in the process.  I'm not a device driver
person, but I imagine there might be other assumptions that might
get in the way as well.

> > OS X has an interesting solution, intended to preserve the
> > flexibility of dynamic modules, while minimizing boot time.
> > It provides a kextcache utility, which packages the kernel
> > and all of the needed modules into a single binary for better
> > locality on disk.  Unlike recompiling the kernel, running
> > kextcache is fast, and the system runs it automatically when
> > hardware or driver changes necessitate it.
> 
> This would be interesting -- isn't it essentially a re-link of the
> kernel + modules?

They actually go further than that and "pre-bind" the modules,
essentially doing the work of the dynamic linker ahead of time.
I have doubts that this trick saves much time.  It also doesn't
address Doug's concern about the overhead of the additional level
of indirection needed for dynamic symbols; however, that overhead
probably isn't significant either.