Managing userland data pointers in kqueue/kevent

[Paul LeoNerd Evans]

I'm trying to build a high-level language wrapper around kqueue/kevent,
specifically, a Perl wrapper.

(In fact I am trying to fix this bug:
  http://rt.cpan.org/Public/Bug/Display.html?id=61481
)

My plan is to use the  void *udata  field of a kevent watcher to store a
pointer to some user-provided Perl data structure (an SV*), to associate
with the event. Typically this could be a code reference for an event
callback or similar, but the exact nature doesn't matter. It's a pointer
to a reference-counted data structure. SvREFCNT_dec(sv) is the function
used to decrement the reference counter.

To account for the fact that the kernel stores a pointer here, I'm
artificially increasing the reference count on the object, so that it
still remains alive even if the rest of the Perl code drops it, to rely
on getting it back out of the kernel in an individual kevent. At some
point when the kernel has finished looking after the event, this count
needs to be decreased again, so the structure can be freed.

I am having trouble trying to work out how to do this, or rather, when.
I have the following problems:

 * If the event was registered using EV_ONESHOT, when it gets fired the
   flags that come back in the event stucture do not include EV_ONESHOT.

 * Some events can only happen once, such as watching for EVFILT_PROC
   NOTE_EXIT events.

 * The kernel can silently drop watches, such as when the process calls
   close() on a filehandl with an EVFILT_READ or EVFILT_WRITE watch.

 * There doesn't seem to be a way to query that pointer back out of the
   kernel, in case the user code wants to EV_DELETE the watch.

These problems all mean that I never quite know when I ought to call
SvREFCNT_dec() on that pointer.

My current best-attack plan looks like the following:

 a) Store a structure in the  void *udata  that contains the actual SV*
    pointer and a flag to remember if the event had been installed as
    EV_ONESHOT (or remember if it was one of the event types that is
    oneshot anyway)

 b) Store an entire mapping in userland from filter+identity to pointer,
    so that if userland wants to EV_DELETE the watch early, it has the
    pointer to be able to drop it.

I can't think of a solution to the close() problem at all, though.

Part a of my solution seems OK (though I'd wonder why the flags back
from the kernel don't contain EV_ONESHOT), but part b confuses me. I had
thought the point of kqueue/kevent is the O(1) nature of it, which is
among why the kernel is storing that  void *udata  pointer in the first
place. If I have to store a mapping from every filter+identity back to
my data pointer, why does the kernel store one at all? I could just
ignore the udata field and use my mapping for my own purposes.

Have I missed something here, then? I was hoping there'd be a nice way
for kernel to give me back those pointers so I can just decrement a
refcount on it, and have it reclaimed. 

-----

I have an idea on a small addition to the kernel API that would make
this issue much simpler to manage, if there is nothing else.

By the addition of a new event flag, called something like
EV_FREEWATCH, the kernel can be told "tell userland whenever I am about
to drop this event watcher". So now, after a EV_ONESHOT or any of the
single events are fired, or when it gets EV_DELETEed, or when the kernel
itself drops because of a close() on a filehandle, it can fire an event
back up to userland with this flag, passing up the pointer.

Now, all userland has to do to correctly manage the memory is to always
set that flag on EV_ADD, and if the flag ever comes back in an event out
of the kernel, it can  SvREFCNT_dec(ev->udata);

-- 
Paul "LeoNerd" Evans

leonerd at leonerd.org.uk
ICQ# 4135350       |  Registered Linux# 179460
http://www.leonerd.org.uk/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20101112/985d20c9/attachment.pgp