svn commit: r339901 - head/sys/conf

[John Baldwin]

On 10/29/18 5:23 PM, John Baldwin wrote:
> Author: jhb
> Date: Tue Oct 30 00:23:37 2018
> New Revision: 339901
> URL: https://svnweb.freebsd.org/changeset/base/339901
> 
> Log:
>   Permit local kernel modules to be built as part of a kernel build.
>   
>   Add support for "local" modules.  By default, these modules are
>   located in LOCALBASE/sys/modules (where LOCALBASE defaults to
>   /usr/local).  Individual modules can be built along with a kernel by
>   defining LOCAL_MODULES to the list of modules.  Each is assumed to be
>   a subdirectory containing a valid Makefile.  If LOCAL_MODULES is not
>   specified, all of the modules present in LOCALBASE/sys/modules are
>   built and installed along with the kernel.
>   
>   This means that a port that installs a kernel module can choose to
>   install its source along with a suitable Makefile to
>   /usr/local/sys/modules/<foo>.  Future kernel builds will then include
>   that kernel module using the kernel configuration's opt_*.h headers
>   and install it into /boot/kernel along with other kernel-specific
>   modules.
>   
>   This is not trying to solve the issue of folks running GENERIC release
>   kernels, but is instead aimed at folks who build their own kernels.
>   For those folks this ensures that kernel modules from ports will
>   always be using the right KBI, etc.  This includes folks running any
>   KBI-breaking kernel configs (such as PAE).
>   
>   There are still some kinks to be worked out with cross-building (we
>   probably shouldn't include local modules in cross-built kernels by
>   default), but this is a sufficient starting point.

The intention is that if, say, the drm-kmod ports install a valid module
Makefile to /usr/local/sys/modules/drm/Makefile, then subsequent kernel
builds will automatically pull in that module.  Also, since "local"
modules are built after in-tree modules, if a ports module has the same
filename (foo.ko), it will be installed over top of the in-tree one
in /boot/kernel effectively replacing it.  Note that as with subdirectories
in sys/modules, a Makefile could just be a bsd.subdir.mk that installs
multiple modules.

I do think that as a further step we might consider having certain ports
stop installing binary kernel modules altogether and only ship source
(especially ports with strong KBI ties to the kernel such as virtualbox
or drm).  Probably these ports would then use a post-install step to
compile against whatever source tree is present on the host and install
the module to /boot/modules.  Anyone compiling custom kernels wouldn't
use that module though after the next kernel recompile as the next kernel
would include a kernel-specific version in /boot/kernel that would take
precedence over the version in /boot/modules.  However, that version in
/boot/modules would still be needed for folks running release kernels that
never compile a kernel.  Further thought is probably required for the
"stock kernel" case to make sure we are really happy with that approach.

Cross-building kernels is also a further consideration.  The current
patch always opts-in to building local modules unless you explicitly
out out by setting LOCAL_MODULES to an empty string or LOCALBASE to
something that doesn't exist.  This means that if you have a port/package
installed that ships kernel module sources and do a 'make tinderbox',
we will try to build that module on all architectures that build modules.
This is probably not what we want.  I did want this feature to default
to enabled for native kernels so that installing a port will Just Work(tm)
without requiring kernel config changes to enable modules and I'd like
to keep that.  Some thoughts I'm considering for cross-builds are:

1) Change the buildkernel and installkernel targets in Makefile.inc1
   to explicitly pass LOCALBASE=/nonexistent as a parameter to make
   if TARGET_ARCH != MACHINE_ARCH or TARGET != MACHINE.  A user could
   explicitly request local modules for a kernel build by setting
   LOCALBASE explicitly when invoking buildkernel/installkernel.

   (You could also point LOCALBASE to a sysroot to honor ports
   installed in a sysroot.  For example, I mount my SD card for my
   RPi on my laptop and cross-build my kernel+world and then install
   onto the SD card via DESTDIR, so I could use LOCALBASE=/mnt/usr/local
   with buildkernel/installkernel to honor ports installed for my
   RPi during my cross-builds.)

2) Have make tinderbox explicitly set LOCALBASE so that even native
   kernels on a host don't try to build kernel modules from ports.
   If we didn't do this by default users could always explicitly
   set LOCALBASE to disable them.

-- 
John Baldwin