SCTP using questions (API etc.)

[Michael Tuexen]

On Mar 5, 2008, at 8:07 PM, Vadim Goncharov wrote:
> Hi Michael Tuexen!
>
> On Wed, 5 Mar 2008 11:59:14 +0100; Michael Tuexen wrote about 'Re:  
> SCTP using questions (API etc.)':
>
>>> "substreams". SCTP can do it for me, it's wonderful, but in practice
>>> there
>>> are some questions.
>>>
>>> How long can be one particular SCTP message? Can I rely on the fact
>>> that it
>>> can be unbounded, e.g. I want to emulate a stream with transfer of
>>> 6 Gig-sized file?
>> Protocol wise there is no limitation of the message size. API wise,  
>> for
>> this size of a message you need to use the explicit EOR mode to be  
>> able
>> to pass this large message using multiple sequential send() calls.
>
> And how should I determine from my/remote stack an optimal size for  
> message
> parts when entire message is guaranteed to not fit into buffers/ 
> windows of
> both peers?
If the sendbuffer is too small for the message to fit, the send call
will return -1 and errno being set to EMSGSIZE. Or you do it in the  
application
by inspecting the sendbuffer size. You do not have to deal with the  
recv buffer
of the peer.
>
>
>>> Can a message be of zero-length data (only headers) ?
>> Empty user messages, i.e. a DATA chunk without payload is not  
>> allowed.
>> An empty SCTP message, i.e. only the common header without any chunks
>> is allowed and processed by FreeBSD when received, but ever send  
>> (well,
>> I do not know a way to force the FreeBSD implementation to send it).
>
> OK, understood. So I should include at least 1 byte of my own  
> headers into
> data and do receive into *iov with at least to parts to ensure good  
> align
> for non-header part?
What header are you talking about? An application header or any SCTP  
header?
You will never receive any SCTP header as part of a user message via
a recv() call. SCTP will give you as much of a message that fits into
the buffer you provide or it has, if the partial delivery API has been  
invoked.
>
>
>>> What is the relation between SCTP streams in both directions? Can
>>> streams
>>> be opened and closed on-demand, like SSH port forwarding (yet again
>>> multiplexing example) or they are preallocated at connection setup  
>>> all
>>> together? What is the minimum number of streams application can rely
>>> upon
>>> (or it just one stream 0 in general case) ?
>> If you restrict to protocols being in RFC status, there is no way of
>> modifying the number of streams during the lifetime of an  
>> association.
>> The number of streams in each direction is negotiated during the
>> association setup. The streams in bother directions are completely
>> independent. There is always at least one stream in each direction,
>> which
>> is stream 0.
>> However, there is an extension (currently specified in an Internet
>> Draft)
>> which allows the addition of streams during the lifetime of an
>> association.
>> The ID is at least partially supported by the FreeBSD implementation.
>> https://datatracker.ietf.org/drafts/draft-stewart-sctpstrrst/
>
> OK. Are there recommended defaults for various stacks about how many
> streams they are creating by default / what maximum of them  
> application
> can ever request?
The maximum number to request is 2^16 - 1. It is controllable by the
applications via socket options. Defaults in OSes are in the order of
10, 16, 32...
>
>
>>> Another important questions are related with blocking and non-
>>> blocking I/O.
>>>
>>> With TCP servers I have two models: in one a thread/process per  
>>> client
>>> (usually blocking), in other - a single-threaded FSM (Finite-State
>>> Machine)
>>> serving all clients using select()/poll()/kqueue()/etc. Both rely on
>>> the
>>> usual UNIX way of creading new file descriptor on accept().
>>>
>>> My server currently uses FSM model, but question is interesting for
>>> both.
>>> I didn't find any words about descriptor duplication or non-blocking
>>> I/O
>>> in SCTP man pages
>> Have you looked at
>> http://www.ietf.org/internet-drafts/draft-ietf-tsvwg- 
>> sctpsocket-16.txt
>
> Yes, I've looked to it briefly before writing my previous letter and
> have re-read it after your pointing. I've found answers for some of
> my question, but that's overall is too complex for me for the first  
> time,
> so I'll ask some another questions below :)
>
>>> How can I put request to kernel for a connect, for example, and then
>>> sleep
>>> until connect will complete or event in some another descriptor will
>>> occur?
>> If you use the 1-to-1 style API, it should be similar to using TCP.
>> Put the socket in non-blocking mode, enable notifications,
>> call connect() and wait until the fd becomes readable. You should  
>> get an
>> indication that that association has been established or could not
>> start.
>
> Yes, that's possible, as I see after reading draft-ietf-tsvwg- 
> sctpsocket.
> But several more questions arise. What notifications do I really need
> on 1-to-1 non-blocking socket API mode? What use is 'context' in this
> 1-to-1 context and why after a failed send I must receive entire  
> failed
> sent message (which can be very long) instead of just an error code?
The context is something you provide in the send call and is given
back to you. So you can use it to find some state/buffer/whatever again.
>
>
> In usual FSM I can use kqueue()/kevent() with arbitrary void* to my
> data, also telling me how many bytes I can read from or write to
> the socket (RCVLOWAT etc.), as well as indicating error/EOF conditions
> so I don't need to do additional syscalls. Is this working with SCTP?
Haven't tried it... Sounds like it would make sense to make sure that
it works.
>
>
> If I can't write to TCP socket (due to window shortage from peer),
> I leave data in my own application buffers, but SCTP tells something
> about unsent messages delivered later, looks somewhat weird, do I  
> really
> need this? Also, all that msg*/cmsg* staff is too complex, and I see
> there are simplier sctp_send()/sctp_sendx() interfaces, will they be
> enough and really simplier for me?..
sctp_sendx() purpose is to use the multiple addresses provided during
the implicit setup of the association. So I think you are not looking  
for
this. sctp_send() can be used to provide the stream id, payload protocol
identifier and to on with using the CMSG stuff. So you might be looking
for this function.
>
>
>>> How can I put each client to it's fd and then do a kqueue()/kevent()
>>> on a
>>> set of those fd's (and other items) ? It is very handy to have this
>>> architecture as kevent() allows to store an arbitrary void* in it's
>>> structure which I can later use to quickly dispatch events.
>>>
>>> And, of course, all this usual C10K-problem-solving-TCP-server
>>> tricks I want
>>> with basic SCTP SEQPACKET benefits: multiple streams and message
>>> record
>>> separation (I don't need other SCTP features currently). Where can I
>>> find
>>> answers to these questions, like it was with W.R.Stevens books for
>>> TCP ?..
>> Have you looked at the third edition of 'Unix Network Programming'?
>> Randall Stewart wrote a couple of sections covering SCTP...
>
> Unfortunately, I have only 2nd edition currently available here,  
> though
> heard about 3rd, yes. May be several months later...
It is really worth buying if you are interested in SCTP socket  
programming...
>
>
> -- 
> WBR, Vadim Goncharov. ICQ#166852181        
> mailto:vadim_nuclight at mail.ru
> [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/ 
> nuclight]
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>