Proposal for changes to network device drivers and network stack (RFC)

[Steve Kiernan]

On Thu, 17 Jan 2013 22:11:27 +0100
Andre Oppermann <oppermann at networx.ch> wrote:

> On 17.01.2013 20:23, Stephen J. Kiernan wrote:
> > The network stack as a module patch has been separated out and can be found in the following location:
> > http://people.freebsd.org/~marcel/Juniper/netstack-v2.diff
> 
> This is quite some work and a lot of changes which will a moment to review.
> 
> Can you describe the concept and the terminology you're using here some more?
> What is a netstack module and what is its scope?  How does it relate to VNET?
> What is an IOCGROUP? etc.  All this is probably obvious to you but not yet for
> us.

Sure, not a problem. First, I will repeat here what Anu sent out previously as a description of the netstack changes.

> Today, not compiling networking stack related files in the kernel breaks
> the kernel build due to dependencies the OS has on the network stack
> (calling into functions in the network stack). Network stack module isn't
> there. We've added these in JUNOS. The benefits for us are obvious (we can
> load our own version of network stack if we desire!), but most likely this
> functionality will benefit others too.
> 
> The detailed implementation is indicated later in this email. In short the
> changes are:
> 
> - Load network stack as a module. For now via loader, not dynamically
> loaded. (Is there interest in dynamic loading?).
> - To facilitate calling network stack functionality from the generic
> kernel, a new interface has been defined with the kobj framework.
> - Some files and tunables needed to move to generic kernel areas (accept
> filters)
> - Generic socket code calls into network stack for interface and route
> related ioctls. Network stack now registers these ioctl groups
> - Other changes: uuid generation (register/query uuid sources), fib/sctp
> system calls (moved to network stack code, with system calls register
> dynamically).

What we want to be able to do is select either the FreeBSD network stack or
some other network stack (currently from the boot loader.) In order to do
this, we need to be able to build a FreeBSD kernel without the network
stack and have the network stack as a KLD.

When the network stack is a module, it must be loaded early before the
kernel boots due to limitations in the implementation of some parts of the
kernel (for example, the domain handling in uipc_domain.c.) Also, with the
changes for the netstack module, the ioctl group code uses a similar
method to avoid having to locking during runtime by not allowing
registration once things are finalized.

By IOCGROUP, I mean the macro that is defined in sys/ioccom.h, which I
assume is supposed to mean ioctl group. I then use that name as a basis
for registering ioctl extensions based on the IOCGROUP. Since it is
handled from the socket code, I used SO_ as a prefix for the macro
name, thus the SO_IOCGROUP_SET() macro was born.

As for VNET, I haven't done anything special for that. However any other
network stack may not have VNET support. Although Mikolaj Golub noted
in a previous reply to the original e-mail that Anu sent out that there
are issues with virtualized global variables in modules. I did not check
to see if the patch that was mentioned has been incorporated yet.

All that was done to support the network stack as a module was to move
network specific pieces in sys/kern to the appropriate net* directory
instead. Any place in the base kernel code that needed to call into
the network stack was then changed to use the netstack kobj to call
through, which has default routines for when no network stack is loaded.

The only current caveat with the changes are there's still some
additional work that needs to be done for jails when using the network
stack as a module. However, if the network stack is compiled into the
kernel (via the NETSTACK option, which is part of the DEFAULTS kernel
configuration file with these changes) then there is no problems with
using jails.

Hopefully this explained things well enough. Let me know if you need
anything else specificly answered.

--
Stephen J. Kiernan
Juniper Networks, Inc.
stevek_at_juniper.net