Bizarre behaviour of Linux binary under 7.1

[Christopher Key]

Hello,

I recently upgraded from 6.3 (i386) to 7.1p3 (amd64) with a view to 
experimenting with zfs. Mostly, everything went smoothly, but I am getting 
some very odd behaviour from a linux utility.

The program is very simple, it has two executables, A and B. A is invoked 
by the user, and based upon the options given builds a list of files to 
process. B is then repeatedly invoked by A with the name of a file to 
process, and the name of a non existent file to write the results to. When 
B returns, A reads the results from the output file, deletes it and moves 
on.

This all worked fine on 6.3, but cannot be made to work as intended on 7.1.

After appropriate use of truss invoking B directly, I found that the source 
of problems was B being unable to create its output file /tmp/...:

linux_open("/tmp/1234.tmp",0x42,0600)            ERR#13 'Permission denied'

which is odd.  /tmp has suitable permissions:

#ls -al /tmp
drwxrwxrwt  12 root   wheel      720 14 Mar 11:25 .
drwxr-xr-x  21 root   wheel      512 13 Mar 10:32 ..
...

and I can quite happily create a identically named file in /tmp myself:
#echo test >/tmp/1234.tmp
#cat /tmp/1234.tmp
test
#rm /tmp/1234.tmp



Bizarrely, however, if I instead invoke B and request its output go to 
/var/tmp/... instead of /tmp/..., it completes successfully.

As a temporary workaround, I therefore tried to create a wrapper around B:

#mv /usr/local/bin/B /usr/local/bin/B2

#cat /usr/local/bin/B #!/bin/sh B2 "$1" "$2" "/var$3" mv "/var$3" "$3"


the idea being that the file would be written to /var/tmp/... by (as now) 
B2, then moved across by my script to where it was expected.

When invoked directly, this works quite happily. However, even more 
bizarrely, when I now call A, allowing it to invoke (my) B, I get exactly 
the same behaviour from my wrapper script as (the original) B was showing 
previously, specifically, it is unable to create the file /tmp/....

As a final workaround, I inserted instead added a sleep to my script in 
place of mv ..., and instead had an external process detect the presence of 
/var/tmp/... and move it across to /tmp. This, unsurprisingly, worked.

Interestingly, if I rewrote my wrapper script to,

B2 "$1" "$2" "/var$3"
sleep 3
cat "/var$3" > "$3" && rm "/var$3"

and had the external process simply touch /tmp/..., my wrapper script 
worked, suggesting that the permissions problem is to do with creating a 
new file, not writing to an existing one.


A few final points:
I've tried both an md based /tmp and tmpfs with the same result.
Everything worked perfectly on 6.3 i386.
If I run A as root, everything works without error.

My guess is that there's something a bit strange in linux_compat, either as 
a result of going to amd64 or to 7.1, and that affects both linux 
executables, and any processes that they create, but I'm not really sure 
beyond that. Can anyone shed any light on what might be going on?


Kind Regards,

Christopher Key