Documentation on writing a custom socket

[Hartmut Brandt]

Julian Elischer wrote:
> Hartmut Brandt wrote:
>> Robert Watson wrote:
>>>
>>> On Sat, 8 Mar 2008, Hans Petter Selasky wrote:
>>>
>>>> I'm planning to create a new socket type in FreeBSD called AF_Q921, 
>>>> which is to be used for ISDN telephony. Where do I find 
>>>> documentation on how to implement a new socket in the kernel ?
>>>
>>
>> [SNIP]
>>
>>> that isn't connected to the protocol stack, or by using a device 
>>> stack tied to Netgraph nodes. Could you tell us a bit more about what 
>>> you're trying to do, and perhaps we can provide some useful 
>>> pointers?  For 
>>
>> I want to jump in here about the netgraph stuff (this was the second 
>> time a response talked about using netgraph).
>>
>> While developing the ATM signaling stack (this is Q.2931) it turned 
>> out, that the netgraph notion of sending message around very rapidly 
>> became a nightmare if you want correct error handling. The number of 
>> states in Q.2931 (12 states) and the API node exploded to something 
>> like 30 or 40 because of the asynchronuous nature of the communication 
>> between stack layers and error handling. One example: To setup a 
>> connection you invoke a SETUP request. Then you wait for something 
>> like CALL_PROCEEDING, RELEASE_ACK or CONNECT_ACK. Unfortunately you 
>> also want to return an error in the case something is wrong with the 
>> request itself (no memory, bad parameters). So you add an extra 
>> message that just ACKs that the SETUP is going to be handled by the 
>> stack or rejected because of some error in it. With a normal function 
>> call based interface you would just make the setup-request function 
>> return an error code. With netgraph however you need to invent a new 
>> message, have additional states in the consumer and the protocol. Not 
>> to talk about error handling when you want to correctly handle errors 
>> like not beeing able to allocate that same response message.
>>
>> Netgraph is very nice for data-flow oriented stuff. It is not so 
>> useful to stack complicated protocol layers. If I'd write the 
>> signaling stack from scratch, I'd probably collaps all the signalling 
>> into a single netgraph node with a socket interface on the upper end. 
>> But then one may as well just implement that as a 'normal' protocol, 
>> probably...
> 
> While I'm very flattered that people have used netgraph for things
> such as these, and happy to see that it works, keep in mind that
> Netgraph was in fact written as a way to do "pluggable link level
> switching", and not really to implement whole protocol stacks.
> 
> A secondary design goal was to allow to do rapid prototyping of
> protocols and stream transforming modules.
> 
> The framework has however been used sometimes as just a way to
> configure, load and control modules that don't actually use the
> network side of netgraph at all. (e.g. ng_fec) and its ability to
> export a stream to userland for extra processing has been useful
> to some people.
> 
> I'm not saying that you were wrong to use netgraph, just explaining
> for the record why it acts the way it does.
> Having said that, nothing is static. I and others who work with
> Netgraph are very open to any suggestions that make it more
> useful. Whether that includes new control mechanisms and feedback 
> schemes, or what-ever, don't feel shy about asking us about the
> possibility of enhancing it.

Yes. I didn't want to say that netgraph is bad - it was just not my best 
  idea to use it for a signaling stack. Indeed, it is excellent if you 
use it right. The SSCOP protocol, which is the transport protocol used 
for ATM signalling is something like TCP - 3-way handshake, SACKs, flow 
control, ... all that stuff. This fits nicely into the scheme - you can 
push that on another node, for example, a raw ATM/AAL5 connection and 
get reliable data transfer. Just implementing several complex state 
machines one on top of the other does not work so good. At the end 
STREAMS suffers from the same problem. There is a reason that Sun merged 
IP and TCP into a single module in Solaris 2.4 or 2.5.

So really - netgraph is an excellent piece of work.

harti