Need advice on sys5 shm and zero copy sockets

[Julian Elischer]

On 2/8/13 4:22 AM, gary mazzaferro wrote:
> Hi,
>
> I was told to post this question here (Ken Merry), it would be a good
> place to get some help. I'm not sure this is doable without a kernel
> module, which I don't want to add.
>
> I'll explain what I'm attempting..
>
> I'm designing a high speed rest motor for cloud execution environment.
>
> 1) I'd like to eliminate copy from the tcp stack to the application(s).
>
> 2) I'm also sharing the buffers across processes and jails. So I'd
> like to preserve the zero-copy in a msg pipe/unix socket
>
> 3) Some buffers will go to disk file systems.
>
>
> Wish list:
> 4) I'd like it to work with sctp because I like it for local  networking :)
>
> 5) I'd like to provision memory pools on a per
> application/connection/ip port basis.
>
> Ultimate Goal:
> 6) Additionally, I'm injecting "code" from a foreign process into the
> workflow of another process (state machine). The connection between
> them will be a signal and shared state information.
>
> I'm assuming item (6) is a separate issue, but it may impact the direction..
>
> I've tried shm with zero copy sockets with linux and it just will not work !!
>
> BTW, I'm returning to freebsd after far too many years
>
> cheers,
> gary
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
>
>

this sound somewhat like what I did back in the 90s with BSD4.3
unfortunately it was not done with TCP (or sctp of course)

what we did was to create a special shared memeory device driver.
Then we added ioctls to the disk driver layer to write named
blocks of memory from that device to the raw device (we didin't use a
filesystem).
We also changed the network drivers to use named blocks of memory
in that device for packet reception. We added a special protocol
which used recvmsg() and  and sendmesg() to pass ownership of
those named blocks between the process that had mmapped them and
the protocol.
The protocol had an IP header but also a small protocol specific 
header of our own..

we sent packets that were larger than a page.

zero copy from disk->process, process->network and network->disk (and 
reverse of course)

of course it was all on proprietary protocols so not of use to you.


julian