KQueue dropping events?

[John-Mark Gurney]

Brandon Erhart wrote this message on Thu, Apr 08, 2004 at 12:32 -0600:
> I am writing a web sucker downer (mirror) for a project on indexing the web 
> (got myself a 1TB raid, just gonna d/l text ..). I am using the KQueue API 
> in FreeBSD 4.9-REL to take care of watching over my sockets. I seem to be 
> running into a nasty problem, however.
> 
> Here's a scenario. I set the outgoing connections to, say, 5000. The 
> problem is, the amount of connections my program shows as being connected 
> is roughly 1/5 to sometimes even 1/8th of what is actually connected. I see 
> what is "actually connected" by doing a netstat. The program would say 
> 750/5000 connections, while a netstat would show 4500 connections in the 
> ESTABLISHED state.

[...]

> It's pretty straight forward. I have no idea why my program would be 
> reporting a smaller amount. Is it possible that my program is not getting 
> ALL the information it needs from kevent()? Perhaps the KQueue is becoming 
> "full"? Is this possible? Should I be pulling more than the 16 events off 
> the kqueue at a time?

There is no way that a kqueue can become full.. (Unless you are doing
process tracing)  All the memory for the kqueue is allocated when the
event is put on the kqueue, so, assuming the kqueue call to add the
event is not failing, there is no way that it can become full...

As for what is happening, are you making sure that you close the sockets
after you are done?

I'm assuming that in your checking for connections, that you make sure
the connect succeeds and then adds the connection to the kqueue and that
also succeeds...

Also, don't forget that if you aren't doing the connect in an async
manner, that since you are only handling 16 events at a time, that it
is possible that the delays before connect returns are preventing all
the connections from being open...

If this last case is it, I'd make sure that you make the socket
non-blocking, and set a kqueue event waiting for it to become writable..

my psuedo code:
for (;;) {
	kevent(...)
	handle events
	while (cons < maxcons)
		create socket non-blocking, connect and add event to kqueue
	other logic
}

You will then have to handle the connection when it succeeds special,
and then switch the state of that socket from connecting to conencted,
and then handle it as normal..

There is nothing that I know of with kqueue that will restrict the number
of sockets to be used with kqueue..

Also, make sure that your per user and per process limits aren't too
small...

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."