Unexpected EADDRINUSE

[Bob Halley]

Some of our regression tests fail randomly on FreeBSD with an unexpected 
EADDRINUSE error.  We didn't see this problem with 4.7, but we see it in 
4.11, 5.4, and 6.0.  We don't see this behavior on any of our other 
supported platforms.

We investigated the problem, and managed to come up with a way to 
reproduce the problem outside of our regression test suite.

To reproduce, first start the sink with "python sink.py".  Then run 
"python conn.py".  When we do this, we get EADDRINUSE after a short 
time.  Since we're binding the connecting socket to the wild port, this 
shouldn't happen.

If the

    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

call is removed, the problem seems to go away.  The setting of 
SO_REUSEADDR is there because that's what some perhaps too-generic 
make-a-socket library code did; we realize it's not technically needed.  
Nevertheless, we don't see why it should fail either :).

I looked at the differences between 4.7 and later kernels but didn't 
manage to figure out how the changes caused this change in behavior.

Regards,

/Bob