Managing userland data pointers in kqueue/kevent

[Adrian Chadd]

... holy crap.

On 13 May 2013 08:37, Eugen-Andrei Gavriloaie <shiretu at gmail.com> wrote:
> Hi,
>
> Well, Paul already asked this question like 3-4 times now. Even insisting on it. I will also ask it again:
> If user code is responsible of tracking down the data associated with the signalled entity, what is the point of having user data?
> Is rendered completely useless…

.. why does everything have to have a well defined purpose that is
also suited for use in _all_ situations?

> Not to mention, that your suggestion with FD index is a definite no-go. The FD values are re-used. Especially in MT environments. Imagine one kqueue call taking place in thread A and another one in thread B. Both threads waiting for events.

.. so don't do that. I mean, you're already having to write your code
to _not_ touch FDs in other threads. I've done this before, it isn't
that hard and it doesn't hurt performance.

> When A does his magic, because of internal business rules, it decides to close FD number 123. It closes it and it connects somewhere else by opening a new one. Surprise, we MAY  get the value 123 again as a new socket, we put it on our index, etc. Now, thread B comes in and it has stale/old events for the old 123 FD. Somethings bad like EOF for the OLD version of FD number 123 (the one we just closed anyway). Guess what… thread B will deallocate the perfectly good thingy inside the index associated with 123.

So you just ensure that nothing at all calls a close(123); but calls
fd_close(123) which will in turn close(123) and free all the state
associated with it.

You have fd_close() either grab a lock, or you ensure that only the
owning thread can call fd_close(123) and if any other thread calls it,
the behaviour is undefined.

> And regarding the "thread happiness", that is not happiness at all IMHO…

Unless you're writing a high connection throughput web server, the
overhead of grabbing a lock in userland during the fd shutdown process
is trivial. Yes, I've written those. It doesn't hurt you that much.

I'm confused as to why this is still an issue. Sure, fix the kqueue
semantics and do it in a way that doesn't break backwards
compatibility. But please don't claim that it's stopping you from
getting real work done. I've written network apps with kqueue that
scales to 8+ cores and (back in mid-2000's) gigabit + of small HTTP
transactions. This stuff isn't at all problematic.


Adrian