`kldload` doesn't find pf.ko, though `ls` finds it easily enough

Thu Nov 18 00:25:34 PST 2004

On Thu, Nov 18, 2004 at 12:57:35AM +0200, Giorgos Keramidas wrote:
> On 2004-11-17 13:31, Tillman Hodgson <tillman at seekingfire.com> wrote:
> > On Wed, Nov 17, 2004 at 10:11:08AM -0800, Mike Hunter wrote:
> > > Check dmesg.  There's a bug somewhere that causes kldload to report "file
> > > not found" to the user even though it was some other error that caused the
> > > problem.  That real error can sometimes be spit out to the console /
> > > syslog / whatever.
> >
> > Oh! I should have checked syslog, my apologies.
> >
> > Nov 17 11:54:41 thoth kernel: link_elf: symbol in6_cksum undefined
> > Nov 17 11:54:41 thoth toor: /etc/rc.d/pf: ERROR: pf module failed to load.
> >
> > My custom kernel doesn't include IPv6. Does pf truly require IPv6?:
> 
> Yes, if you want to load it as a module.  This is exactly the same thing
> I bumped into last night.  Compiling a kernel with INET6 support will
> let you load pf as a module.
> 
> I'm not sure if this is a `bug' that neds fixing though ;-)
> 
I've been investigating this problem recently.  It's a royal PITA
that modules that get compiled with the kernel cannot be linked
into it.  I tried to brute-force it by applying an attached patch
(it has a portion for modules/pf/Makefile), but it soon became
clear that it's not that all simple.  The real fix would involve
carefully looking into every module makefile that generates
opt_*.h files (currently, 63 makefiles), and in the case of a
module build with the kernel (KERNBUILDDIR is defined), generate
only those headers that this "device" is responsible for.

For example, the portion for if_disc in the attached makefile
is correct, as disc(4) can be compiled with any combination of
INET and INET6 options, even when both are undefined.

And the portion for pf/Makefile should look like this, i.e.,
always generate opt_pf.h, but take opt_inet[6].h and opt_bpf.h
from the kernel.

%%%
Index: Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/pf/Makefile,v
retrieving revision 1.5
diff -u -r1.5 Makefile

--- Makefile	1 Sep 2004 07:39:12 -0000	1.5
+++ Makefile	18 Nov 2004 08:06:31 -0000
@@ -16,6 +16,7 @@
 	echo "#define DEV_PF 1" > opt_pf.h
 	echo "#define DEV_PFLOG 1" >> opt_pf.h
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > opt_inet.h
 
@@ -28,5 +29,6 @@
 
 opt_bpf.h:
 	echo "#define DEV_BPF 1" > opt_bpf.h
+.endif
 
 .include <bsd.kmod.mk>
%%%

Unfortunately, while it's indeed correct for opt_*.h files,
the SRCS list still assumes that option INET is present in
the kernel (in4_cksum.c is compiled unconditionally), and
some things are just not designed to work without some
options, like option INET in this case, but their dependency
is incorrect.  For example, attempting to build a pf module
with this patch and a kernel without INET option results in:

: In file included from /usr/src/sys/modules/pf/../../contrib/pf/net/if_pflog.c:81:
: ./machine/in_cksum.h:54: warning: "struct ip" declared inside parameter list
: ./machine/in_cksum.h:54: warning: its scope is only this definition or declaration, which is probably not what you want
: ./machine/in_cksum.h:80: warning: "struct ip" declared inside parameter list
: ./machine/in_cksum.h: In function `in_cksum_update':
: ./machine/in_cksum.h:83: error: dereferencing pointer to incomplete type
: ./machine/in_cksum.h:84: error: dereferencing pointer to incomplete type
: *** Error code 1

But if_pflog.c doesn't show a dependency on "option INET"
in sys/conf/files:

# grep if_pflog.c /sys/conf/files
contrib/pf/net/if_pflog.c               optional pflog

I think everyone who has looked into it knows how broken the
current module system is in this respect.

But then we think about opt_bpf.h...  If pf had a module
dependency on bpf, then we could unconditionally compile
BPF support into the pf module.

In the end I gave up on this, as it became clear that
modules build system needs a good redesign.  My current
feeling is that sys/modules should die, and modules
build infrastructure should be generated by config(8).
For example, you create a config file and either tell
it to build everything statically into one kernel object,
or build all (or part) that it can as modules.


Cheers,
-- 
Ruslan Ermilov
ru at FreeBSD.org
FreeBSD committer
-------------- next part --------------
Index: if_disc/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/if_disc/Makefile,v
retrieving revision 1.17
diff -u -r1.17 Makefile
--- if_disc/Makefile	11 Jan 2002 15:48:40 -0000	1.17
+++ if_disc/Makefile	3 Nov 2004 20:17:20 -0000
@@ -5,10 +5,12 @@
 KMOD=	if_disc
 SRCS=	if_disc.c opt_inet.h opt_inet6.h
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > ${.TARGET}
 
-#opt_inet6.h:
-#	echo "#define INET6 1" > ${.TARGET}
+opt_inet6.h:
+	echo "#define INET6 1" > ${.TARGET}
+.endif
 
 .include <bsd.kmod.mk>
Index: if_ef/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/if_ef/Makefile,v
retrieving revision 1.11
diff -u -r1.11 Makefile
--- if_ef/Makefile	19 Jan 2004 14:00:26 -0000	1.11
+++ if_ef/Makefile	3 Nov 2004 20:25:06 -0000
@@ -16,11 +16,13 @@
 CFLAGS+= -DEF_DEBUG
 .endif
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > opt_inet.h
 
 opt_ipx.h:
 	echo "#define IPX 1" > opt_ipx.h
+.endif
 
 opt_ef.h:
 .for frame in ETHER_II ETHER_8023 ETHER_8022 ETHER_SNAP
Index: if_faith/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/if_faith/Makefile,v
retrieving revision 1.2
diff -u -r1.2 Makefile
--- if_faith/Makefile	11 Jan 2002 15:48:46 -0000	1.2
+++ if_faith/Makefile	3 Nov 2004 20:26:19 -0000
@@ -5,10 +5,12 @@
 KMOD=	if_faith
 SRCS=	if_faith.c opt_inet.h opt_inet6.h
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > ${.TARGET}
 
 opt_inet6.h:
 	echo "#define INET6 1" > ${.TARGET}
+.endif
 
 .include <bsd.kmod.mk>
Index: if_gif/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/if_gif/Makefile,v
retrieving revision 1.3
diff -u -r1.3 Makefile
--- if_gif/Makefile	12 Aug 2002 15:27:17 -0000	1.3
+++ if_gif/Makefile	3 Nov 2004 20:33:32 -0000
@@ -6,6 +6,7 @@
 SRCS=	if_gif.c in_gif.c in6_gif.c opt_inet.h opt_inet6.h opt_mac.h \
 	opt_mrouting.h
 
+.if !defined(KERNBUILDDIR)
 opt_inet.h:
 	echo "#define INET 1" > ${.TARGET}
 
@@ -14,5 +15,6 @@
 
 opt_mrouting.h:
 	echo "#define MROUTING 1" > ${.TARGET}
+.endif
 
 .include <bsd.kmod.mk>
Index: netgraph/bluetooth/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/netgraph/bluetooth/Makefile,v
retrieving revision 1.2
diff -u -r1.2 Makefile
--- netgraph/bluetooth/Makefile	10 May 2003 21:44:37 -0000	1.2
+++ netgraph/bluetooth/Makefile	24 Oct 2004 06:26:10 -0000
@@ -3,13 +3,12 @@
 
 SUBDIR=	\
 	bluetooth \
-	hci	\
-	l2cap	\
-	socket	\
-	bt3c	\
-	h4	\
-	ubt	\
+	bt3c \
+	h4 \
+	hci \
+	l2cap \
+	socket \
+	ubt \
 	ubtbcmfw
 
 .include <bsd.subdir.mk>
-
Index: pf/Makefile
===================================================================
RCS file: /home/ncvs/src/sys/modules/pf/Makefile,v
retrieving revision 1.5
diff -u -r1.5 Makefile
--- pf/Makefile	1 Sep 2004 07:39:12 -0000	1.5
+++ pf/Makefile	3 Nov 2004 15:26:53 -0000
@@ -12,6 +12,7 @@
 
 CFLAGS+=  -I${.CURDIR}/../../contrib/pf
 
+.if !defined(KERNBUILDDIR)
 opt_pf.h:
 	echo "#define DEV_PF 1" > opt_pf.h
 	echo "#define DEV_PFLOG 1" >> opt_pf.h
@@ -28,5 +29,6 @@
 
 opt_bpf.h:
 	echo "#define DEV_BPF 1" > opt_bpf.h
+.endif
 
 .include <bsd.kmod.mk>
