process shared mutexes?

[Konstantin Belousov]

On Mon, Nov 21, 2016 at 04:25:42PM +0100, Volker Lendecke wrote:
> On Mon, Nov 21, 2016 at 05:10:40PM +0200, Konstantin Belousov wrote:
> > Please see the libthr(3) man page, in particular, read the RUN-TIME
> > SETTINGS section, the description of the kern.ipc.umtx_vnode_persistent
> > sysctl.
> > 
> > Does setting the sysctl to 1 allow your program to run ?
> 
> Yes, that does make it work. The description says that the umtx vnode
> is dropped on the last munmap. If I #if 0 the middle mmap, it works,
> although there is no mmap around anymore. So the description is not
> 100% accurate I'd say.
No, not umtx vnode is dropped, but the shared umutexes attached to the
file' page.

What you observed is the consequence of an implementation detail.  It is
impossible to execute umtx cleanup while dereferencing vm object, due to
locking issues.  An asynchronous task is scheduled to perform the cleanup.
But when the task is run, it is quite possible that your process is already
executed second mmap() and pthread_mutex_lock(), creating another reference
on the umtx data.

The mmap(NULL, 0xb, ...) delays execution of that part of the program, so
the task wins more reliably.  This initially puzzled me as well, since I
was not able to observe your reported behaviour on the amd64 host, either
on 64 or 32bit binary.

> 
> When does the recycling happen exactly? Can it break running
> applications?
See above.  I cannot answer the second question.

> 
> And -- how can I make sure for Samba that this is set properly at
> runtime? We already have a runtime mutex test for some ancient Linux
> kernels that were broken. We could add this as a subtest too. But --
> what happens if the admin resets this while Samba is running? Does
> the kernel make sure that existing files still get the correct
> behaviour when the sysctl changes?

It is reasonable to expect that administrator set the knob very early
during the boot, and not twiddle it at runtime, at least if explicit
instructions to do so are provided.  Samba may, additionally, check
the sysctl value during the initialization and provide a hint to
user if needed.

On Mon, Nov 21, 2016 at 04:39:45PM +0100, Volker Lendecke wrote:
> On Mon, Nov 21, 2016 at 05:10:40PM +0200, Konstantin Belousov wrote:
> > Please see the libthr(3) man page, in particular, read the RUN-TIME
> > SETTINGS section, the description of the kern.ipc.umtx_vnode_persistent
> > sysctl.
> 
> One more question: man libtrh(3) says that this behaviour is allowed
> by Posix. Do you have a reference that is available online? Or do I
> have to buy the Posix standards documents?

Look at the Single Unix Specification, particularly to the following
paragraph in the mmap() description:

The state of synchronization objects such as mutexes, semaphores,
barriers, and conditional variables placed in shared memory mapped
with MAP_SHARED becomes undefined when the last region in any process
containing the synchronization object is unmapped.

The reference is available at
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
but you might need to register before getting the online access or
downloading the entire SUSv4tc2 in archive.