The Almighty KQueue

[Brandon Erhart]

Hi,

I told you all I'd be back for more :)   I am not sure if this is the most 
appropriate place to post this, but it seems that you all know what the 
hell you're talking about, so I figure you can help me out here.

As I said in my previous post about the FIN_WAIT states, I am writing a web 
sucker downer (mirror). What I don't believe I mentioned before is that I 
am using the KQueue API in FreeBSD 4.9-REL to take care of watching over my 
sockets. I seem to be running into a nasty problem, however.

Here's a scenario. I set the outgoing connections to, say, 5000. The 
problem is, the amount of connections my program shows as being connected 
is roughly 1/2 to sometimes even 1/8th of what is actually connected. I see 
what is "actually connected" by doing a netstat. The program would say 
750/5000 connections, while a netstat would show 4500 connections in the 
ESTABLISHED state.

You may be saying, "it must be a bug in your connection tracking logic". I 
honestly don't think that's it. I have only TWO places in my code where I 
check if the connection was successful (and therefore increasing the global 
connection counter) -- in the callback function for data read() from 
KQueue-monitored fds (all of the sockets), and then also in my main loop (I 
check for read, write and connect timeouts there, right after my call to 
kevent()).

Basically the main loop looks like, in psuedo-code of course:

while (there_is_still_events)
{
	if (kevent())   <-- i pull one event from the kqueue
	{
		execute_the_callback_function;
	}

	check_for_read()_write()_and_connect()_timeouts;

	check_for_"client_descriptors(basically just a structure that holds info 
on the kqueue event)"_that_need_to_be_connected_to_the_next_server_and_do_so;
}

It's pretty straight forward. I have no idea why my program would be 
reporting a smaller amount. Is it possible that my program is not getting 
ALL the information it needs from kevent()? Perhaps the KQueue is becoming 
"full"? Is this possible? Should I be pulling more than one event off the 
kqueue at a time?

I have been up ALL NIGHT trying to debug this, and cannot figure it out.

Any and all help is appreciated!

Thanks,

Brandon