netisr

[Terry Lambert]

"Giovanni P. Tirloni" wrote:
>  I'm studying the network stack and now I'm confronted with something
>  called netisr. It seems ether_demux puts the packet in a netisr queue
>  instead of passing it directly to ip_input (if that was the packet's
>  type). Is this derived from LRP ?

No.  NETISR is a software interrupt that runs when software
interrupts run, which is to say, when the SPL is lowered as
a result of returning from the last nested hardware interrupt,
which means on hardware and clock interrupts.

It is completely antithetical to LRP, which, despite the name
"Lazy Receiver Processing", is mostly only lazy about direct
dispatch.


>  I've read their paper and it looks
>  like their network channel (ni) but I'm not sure.

No.  There are two LRP papers from Rice University.  The first
was against FreeBSD 3.2, and dealt with just the idea of LRP.
The scond makes things much more complicated than they need to
be, in order to introduce a concept called "ResCon", or resource
containers.  Neither set of code is really production, because
it uses an alternate protocol family definition and a seperately
hacked uipc_socket.c and uipc_socket2.c, along with the totally
parallel TCP/IP stack.

The closest you can come to LRP in FreeBSD 5.x is to use the
sysctl's for "direct dispatch", which will, in fact, directly
call ip_input() from interrupt.

This isn't a full LRP, since it doesn't add a receive buffer
into the proc structure for per-process enqueuing of packets.

When I implemented the 3.x version of Rice's LRP in FreeBSD 4.3,
I avoided this hack.  The main reason for the hack was to deal
with accepting connections, since at interrupt, without a proc
structure, there was no way to deal with the socket creation for
the accept, due to a lack of an appropriate credential.  The
sneaky approach I used for this was to create the accept socket
using the cred that was present on the listen socket on which
the connection had come in.  For this to be at all useful, you
need to extend kevent accept filters to allow creation of accepted
descriptor instances in the process context, and throw them onto
the kqueue that was set up against the listen socket.

I recommend that if you want to play with LRP, you add an attribute
flag to the protocol stack structure to indicate an LRP capable vs.
an LRP incapable stack, and then implement it inline, rather than
as a separate thing.  I also recommend that if you do this, you do
it using the Rice 3.x code, and ignore the "ResCon" stuff, which I
think is an interesting idea, but which adds very little in the way
of real value to things (though it does add overhead).

>  Where can I find more information on this?

If you are asking about NETISR, then I recommend W. Richard Steven's
books, specifically "TCP/IP Illustrated, Volume 2: The Implementation".

If you are asking about LRP, then any search engine search for "Lazy
Receiver Processing" should turn up the two Rice University and the
one Duke University reference, as well as dozens of other references,
including the one to the Lucent/IBM work on QLinux (which has some
other neat information on things like WFS Queueing and other things
that are actually necessary to avoid the potential for livelock at
the user/kernel boundary).

FWIW: Interrupt threads, as they are used in 5.x, are pretty much
antithetical to an LRP implementation, since you can still end up
live-locked under heavy load (or denial of service attack), which
is why you wanted LRP in the first place: to make progress under a
load heavier than you could possibly respond to in a reasonable
time.  The problem is that the requeing to the interrupt thread adds
back in the same type of transition boundary you were trying to take
out by getting rid of NETISR in the first place.

-- Terry