strange timeout error returned by kevent() in 6.0

[Igor Sysoev]

On Thu, 1 Dec 2005, Igor Sysoev wrote:

> On Thu, 1 Sep 2005, Igor Sysoev wrote:
>
>> On Thu, 1 Sep 2005, Igor Sysoev wrote:
>> 
>>> I found strange timeout errors returned by kevent() in 6.0 using
>>> my http server named nginx.  The nginx's run on three machines:
>>> two 4.10-RELEASE and one 6.0-BETA3.  All machines serve the same
>>> content (simple cluster) and each handles about 200 requests/second.
>>> 
>>> On 6.0 sometimes (2 or 3 times per hour) in the daytime kevent()
>>> returns EV_EOF in flags and ETIMEDOUT in fflags, nevertheless:
>>> 
>>> 1) nginx does not set any kernel timeout for sockets;
>>> 2) the total request time for such failed requests is small, 30 and so 
>>> seconds.
>> 
>> I have changed code to ignore the ETIMEDOUT error returned by kevent()
>> and found that subsequent sendfile() returned the ENOTCONN.
>> 
>> By the way, why sendfile() may return ENOTCONN ?
>> I saw this error code on 4.x too.
>
> Recently I've found that kevent() in FreeBSD 5.4 may return wrong
> the ETIMEDOUT too.
>
> Also I've found that recv() on FreeBSD 6.0 may return wrong ETIMEDOUT
> error for socket that has no any kernel timeout. It seems this
> ETIMEDOUT error masks another error.

It's seems that this ETIMEDOUT is caused by a retransmit failure, when
data were retransmited 12 times with backoff timeout. The whole timeout
is small, 30-50 seconds, because the initial RTO is very small: 5-10 ms.


Igor Sysoev
http://sysoev.ru/en/