Kqueue + Libevent

[about bus]

I've added some debug output in libevent, before and after kevent function
call.

### good one request
21:30:05   evhttp_get_request_connection: fd: 196 new request from
127.0.0.1:60010
21:30:05   kq_dispatch: fd: 196 set    kevent filter: EVFILT_READ flags:
EV_ADD
21:30:05   kq_dispatch: fd: 196 return kevent filter: EVFILT_READ flags: 0
data: 729
21:30:05   kq_dispatch: fd: 196 set    kevent filter: EVFILT_READ flags:
EV_DELETE
21:30:05   kq_dispatch: fd: 196 set    kevent filter: EVFILT_WRITE flags:
EV_ADD
21:30:05   kq_dispatch: fd: 196 return kevent filter: EVFILT_WRITE flags: 0
data: 28672
21:30:05   kq_dispatch: fd: 196 set    kevent filter: EVFILT_WRITE flags:
EV_DELETE
21:30:05   kq_dispatch: fd: 196 return kevent filter: EVFILT_WRITE flags:
EV_ERROR data: 9 (EBADF)

### timeouted request
21:30:06   evhttp_get_request_connection: fd: 196 new request from
127.0.0.1:64308
21:30:06   kq_dispatch: fd: 196 set    kevent filter: EVFILT_READ flags:
EV_ADD
21:30:36   kq_dispatch: fd: 196 return kevent filter: EVFILT_READ flags:
EV_EOF data: 450
21:30:36   kq_dispatch: fd: 196 set    kevent filter: EVFILT_READ flags:
EV_DELETE
21:30:36   kq_dispatch: fd: 196 set    kevent filter: EVFILT_WRITE flags:
EV_ADD
21:30:36   kq_dispatch: fd: 196 return kevent filter: EVFILT_WRITE flags: 0
data: 28672
21:30:36   kq_dispatch: fd: 196 set    kevent filter: EVFILT_WRITE flags:
EV_DELETE
21:30:36   kq_dispatch: fd: 196 return kevent filter: EVFILT_WRITE flags:
EV_ERROR data: 9 (EBADF)

There is problem between 21:30:06 and 21:30:36 timestamps.
Libevent received socket from Kqueue with EV_EOF flag when nginx has closed
connection.
There is 30 seconds timeout value in nginx config file.

On Fri, Aug 19, 2011 at 2:10 PM, Adrian Chadd <adrian at freebsd.org> wrote:

> I've not seen this happen before, are you sure that libevent's kqueue
> code is properly removing all the pending events for a given FD when
> it's closed?
>
>
> On 19 August 2011 18:33, about bus <aboutbus at gmail.com> wrote:
> > Hello!
> >
> > I've got some interesting problem with my own server which use Libevent
> and
> > Kqueue.
> > Kqueue holds some sockets for a long time (while reading data) and gives
> it
> > back to application when another side closes connection.
> >
> > I have server with FreeBSD 7.2 which serves several thousands requests
> per
> > second.
> > Nginx -> statical requests, images
> >         -> backend for dynamical requests: Libevent based http server
> > written on C
> >
> > Sometimes, about 0.1% of request, Nginx displays error: "Operation timed
> out
> > while reading response from upstream."
> > Switching Libevent from Kqueue to Poll in my http server fixes the
> problem.
> > Poll works fine without errors in Nginx log file.
> >
> > Debug output for timeouted requests from Libevent:
> > 1) Libevent accepts incoming connection from Nginx.
> > 2) Libevent adds socket to Kqueue for waiting incoming data with HTTP
> > request.
> > 3) No events on socket for 60 seconds, no received data. (60 - timeout
> value
> > in Nginx config)
> > 4) On the another side Nginx closes connection and displays timeout
> error.
> > 5) At the same time Kqueue reports about incoming data and returns socket
> to
> > Libevent.
> > 6) Libevent reads correct HTTP request from socket and passes it to my
> > handler.
> >
> > Why Kqueue does not reports about read event for so long time?
> > Why Kqueue do it only when another side closes connection?
> > It is strange, because Poll does not have such problems.
> > _______________________________________________
> > freebsd-hackers at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> > To unsubscribe, send any mail to "
> freebsd-hackers-unsubscribe at freebsd.org"
> >
>