close() of active socket does not work on FreeBSD 6

[Arne H. Juul]

On Mon, 11 Dec 2006, Kostik Belousov wrote:
> On Mon, Dec 11, 2006 at 04:07:09PM +0100, Arne H. Juul wrote:
>> Looking at the Java VM source code it does some tricks with dup2() to
>> reopen the close()'d filedescriptor, making it point to a filedescriptor
>> that's pre-connected to a closed socket.
>>
>> A small C program that duplicates this (using pipes to make it a bit
>> simpler) follows.  I'm not sure if any standards demand that this
>> works like it used to on FreeBSD 4 / libc_r, but since Java uses it it
>> would be really nice if this could be made to work in FreeBSD 6 (libthr
>> and libpthread).   Or maybe somebody has another suggestions on how to
>> implement the Java close() semantics?
>
> I think that -arch@ is proper ML to discuss the issue.
>
> Your test example hangs becase read() takes one more hold count on the
> file descriptor operated upon. As result, when calling close, f_count
> of the rpipe (aka p[0]) is 2, close() decrements it, f_count becomes
> 1. Since f_count > 0, fdrop_locked simply returns instead of calling
> fo_close (see kern_descrip.c).
>
> I cannot find the statement in SUSv3 that would require interruption of
> the read() upon close() from another thread; this looks like undefined
> behaviour from the standard point of view.

The best authority I've found says that the standards are silent (so
the current FreeBSD 6 behaviour is allowed), I'm asking whether it is
best practice and why it's changed since FreeBSD 4.

> I think that JVM is more appropriate place for fix, but others may have
> different view point.

If it was just the JVM I would agree, but any threaded program that uses
blocking I/O in some threads will probably need the same kind of handling
at some point.  And if you think about what that handling looks like,
it's not exactly pretty:

* when calling any potentially blocking system call (read/readv,
   write/writev, recv/recvfrom/recvmsg, send/sendto/sendmsg, accept,
   connect, poll, select, maybe others that I didn't think of) the
   application must:

   ** take a mutex
   ** remember in some structure (linked list or similar) keyed off
      the file descriptor that "this thread will now do blocking I/O"
   ** release the mutex
   ** perform the actual operation
   ** take the mutex again
   ** check if the operation was interrupted in a special way, if so
      return with EBADF
   ** release the mutex

* instead of calling close() and dup2() the application must:
   ** take the mutex
   ** for each thread in the FD-associated structure, interrupt it
      in some special way (I'm guessing that setting a special flag
      and then sending SIGIO should work).
   ** actually do the close() / dup2()
   ** release the mutex

This is exactly the sort of issue that should be solved by the
thread library / kernel threads implementation and not in every
threaded application that needs it, in my view.

   -  Arne H. J.