Mixing Asynchronous Network I/O and POSIX Threads

[Richard Yao]

Dear Jilles,

I am using sigwaitinfo() with all interrupts masked to avoid the
possibility of race conditions in signal handlers, but I have not used
any realtime signals. Linux 2.6.35 found a way to invoke the SIGIO
handler despite it being masked, but that issue would not occur under
production conditions and that is a bug for a different mailing list.

Being unable to F_SETOWN individual threads would cause problems
because it causes network traffic to become serialized. My code must
run on Linux, but if I were to write FreeBSD-specific code so it would
also work on FreeBSD, would using kqueue enable me to specify which
threads handle events on specific file descriptors?

Yours truly,
Richard Yao

On Sun, Sep 18, 2011 at 3:44 PM, Jilles Tjoelker <jilles at stack.nl> wrote:
> On Sun, Sep 18, 2011 at 11:32:08AM -0400, Richard Yao wrote:
>> I wrote a program for Linux that uses Asynchronous Network I/O and
>> POSIX Threads. It uses a mix of gettid(), fcntl() and F_SETOWN to
>> specify which thread handles each connection's SIGIO interrupts.
>> gettid() is Linux-specific and I would prefer to do this in a way that
>> also works with FreeBSD. Is that possible?
>
> In FreeBSD, you can only F_SETOWN processes or process groups, not
> threads, so a direct port is probably not possible. Another Linux
> feature not in FreeBSD is F_SETSIG (to send a realtime signal instead of
> SIGIO and provide siginfo_t information).
>
> My recommendation is not to use SIGIO and SIGURG at all. If you use
> signal handlers, your program is very likely to suffer from race
> condition problems unless you unmask the signal only at well-defined
> safe points. If you use sigtimedwait() or similar, select()/poll() can
> provide similar functionality. Alternatively, if you have many
> descriptors to watch, use non-portable facilities like BSD kqueue, Linux
> epoll, Solaris ports or portable wrappers around them. Realtime signals
> are nasty for this -- the signal queue has a finite size and when it
> overflows you need a "resync" step that is expensive both in CPU and
> programmer time.
>
> --
> Jilles Tjoelker
>