sockets without owner.

[Chuck Swiger]

Stefan Lambrev wrote:
> I'm having very strange problem.
> I have near 200 sockets reported by netstat -An, which are NOT reported 
> by sockstat and fstat.
> All of them look like (output from netstat -An) :
> 
> ffffff0169282000 tcp4       0      0  192.168.13.12.4965 
> 192.168.13.3.8080  FIN_WAIT_2
> 
> I'm trying to figure out what keeps them active (tcpdump shows traffic 
> from 192.168.13.12 to 192.168.13.3)
> The application that create them is stopped before more then a week but 
> request are still "flying" around.

FIN_WAIT_2 means that one side has closed the TCP connection and is waiting 
for the other side to finish sending any remaining data and acknowledge the 
close of the connection by sending a FIN.  The TCP stack can remain in this 
state until the other side acknowledges or resets the connection, even though 
the local process is gone.  (In fact, it's not unusual for a process to close 
and then quit.)

The TCP stack is supposed to wait 2*MSL (ie, 120 seconds), and then move to 
TIME_WAIT and then CLOSED, IIRC, if it doesn't hear anything back.

> I have access to both IPs - 192.168.13.12 is http balancer and 
> 192.168.13.3 is apache server (both were restarted)

Sounds like a glich in the TCP state engine of the load-balancer, probably. 
This sort of thing happens when using LBs somewhat frequently....

-- 
-Chuck