Communications kernel -> userland

[Robert Watson]

On Tue, 22 Jul 2003, Adam Migus wrote:

> Perhaps I'm not understanding you right but I think Pawel's idea is
> cool.  It seems to fulfill your requirements (except being network
> specific).  I suppose if it were network specific we could optimize it
> for packet streams and if we made it complicated enough it would require
> quite an elaborate sychronization and notification mechanism.  Is that
> closer to what have in mind? 

Well, the case I had particularly in mind was the rapid flow of packets
form the kernel to the user process; Pawel's suggestion handles the flow
of new data from the user process to the kernel well, and has substantial
similarity to some of the IO Lite mechanisms I pointed at (and hopefully
with many of the same performance benefits).  In the kernel-to-userspace
case, we want to avoid the copy of what is originally kernel-owned memory
(from the mbuf allocator) to the user process memory.  If you didn't care
about stuff like confidentiality of kernel memory, etc, the simplest
approach would be to actually map the mbuf memory (and possibly cluster)
into userspace, and then notify the user process in some form of the new
mapping.  However, because mbufs and their meta-data aren't page aligned
(etc, etc, etc), you really don't want to do it explicitly that way, I
suspect. 

By synchronization, I had in mind a mechanism by which the process and
kernel would communicate about memory ownership in the shared memory
space: "I'm done with this packet", "I'm done with these packets", "I want
to continue delivery of that packet", "I modified this packet", "I'm
inserting a new packet here", "I'm dropping this packet", all without
extensive memory copying, and with a moderate amount of asynchrony (and
possibly parallelism).  In terms of functionality, it might be similar to
some of the current services that forward between IPDIVERT "in" and "out" 
(such as natd), or between BPF pseudo-devices.  This sounds like something
that likely exists in a few commercial products already, so my question to
Terry was to whether he knew of any in the literature.  IOLite is the
closest I know of, as it supports the zero-copy page and memory ownership
bits, although I don't know if they allowed it to handle packets, perhaps
just datagrams and streams.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Network Associates Laboratories