[6.x] problem with AIO, non-blocking sockets on freebSD and IE7 on windows.

[Julian Elischer]

If one has an event-driven process that accepts tcp connections, 
one needs to set eh non-blocking socket option and use kqueue or 
similar to schedule work.

This is ok for data transfers, however when it comes to the 
close() call there is a problem.  The problem in in 
the following code in so_close()


                if (so->so_options & SO_LINGER) {
                        if ((so->so_state & SS_ISDISCONNECTING) &&
                            (so->so_state & SS_NBIO))
                                goto drop;
                        while (so->so_state & SS_ISCONNECTED) {
                                error = tsleep(&so->so_timeo,
                                    PSOCK | PCATCH, "soclos", so->so_linger * hz);
                                if (error)
                                        break;
                        }
                }
        }

drop:
   [ continues on to destroy socket ]


because SS_NBIO is set, the socket acts as if SO_LINGER was set, with a timeout of 0.
the result of this, is the following behaviour:

========================================== 
The first + last packet output is below:
========================================== 
Source         Destination    Info
172.20.51.115  172.20.3.43    2597 > http [SYN] Seq=0 Len=0 
172.20.3.43    172.20.51.115  http > 2597 [SYN, ACK] Seq=0 Ack=1 Len=0 
172.20.51.115  172.20.3.43    2597 > http [ACK] Seq=1 Ack=1 Len=0
172.20.51.115  172.20.3.43    POST http://creative.gettyimages.com/source/<truncated>
HTTP/1.1
172.20.51.115  172.20.3.43    HTTP
172.20.3.43    172.20.51.115  http > 2597 [ACK] Seq=1 Ack=1261 Len=0

---------------

172.20.3.43    172.20.51.115  http > 2597 [ACK] Seq=1 Ack=13729 Len=0
172.20.51.115  172.20.3.43    HTTP
172.20.3.43    172.20.51.115  HTTP/1.0 407 Proxy Authentication Required
(text/html)
172.20.3.43    172.20.51.115  HTTP
172.20.3.43    172.20.51.115  http > 2597 [FIN, ACK] Seq=1858 Ack=13729 Len=0
172.20.51.115  172.20.3.43    2597 > http [ACK] Seq=13731 Ack=1859 Len=0
172.20.3.43    172.20.51.115  http > 2597 [RST] Seq=1 Len=0
172.20.3.43    172.20.51.115  http > 2597 [RST] Seq=1859 Len=0


The ACK that comes from the client is wquite legal and in fact a FIN should follow.
however we react to it by sending a reset.

This makes IE7 throw a "generic IE error page". Even though it has all the information it 
needs.
Less that a good result for the user.

The answer is to NOT destroy the socket immediately, but to schedule it for self destruction
in FIN_WAIT_1_TIME seconds (or so_linger secs) or when the FIN turns up, whichever occurs first.

however so_close is in the wrong layer to decide to do this I think... socket code in general
has no timer related stuff.. TCPhas timers, so I thin it would require a new call into TCP
to tell it to put the session in question onto a (new) timer..

thoughts anyone?

Julian