Bizarre behaviour of Linux binary under 7.1

[Michael Powell]

Christopher Key wrote:

> Hello,
> 
> I recently upgraded from 6.3 (i386) to 7.1p3 (amd64) with a view to
> experimenting with zfs. Mostly, everything went smoothly, but I am getting
> some very odd behaviour from a linux utility.
> 
> The program is very simple, it has two executables, A and B. A is invoked
> by the user, and based upon the options given builds a list of files to
> process. B is then repeatedly invoked by A with the name of a file to
> process, and the name of a non existent file to write the results to. When
> B returns, A reads the results from the output file, deletes it and moves
> on.
> 
> This all worked fine on 6.3, but cannot be made to work as intended on
> 7.1.
> 
> After appropriate use of truss invoking B directly, I found that the
> source of problems was B being unable to create its output file /tmp/...:
> 
> linux_open("/tmp/1234.tmp",0x42,0600)            ERR#13 'Permission
> denied'
> 
> which is odd.  /tmp has suitable permissions:
> 
> #ls -al /tmp
> drwxrwxrwt  12 root   wheel      720 14 Mar 11:25 .
> drwxr-xr-x  21 root   wheel      512 13 Mar 10:32 ..
> ...
> 
> and I can quite happily create a identically named file in /tmp myself:
> #echo test >/tmp/1234.tmp
> #cat /tmp/1234.tmp
> test
> #rm /tmp/1234.tmp
> 
> 
> 
> Bizarrely, however, if I instead invoke B and request its output go to
> /var/tmp/... instead of /tmp/..., it completes successfully.
> 
> As a temporary workaround, I therefore tried to create a wrapper around B:
> 
> #mv /usr/local/bin/B /usr/local/bin/B2
> 
> #cat /usr/local/bin/B #!/bin/sh B2 "$1" "$2" "/var$3" mv "/var$3" "$3"
> 
> 
> the idea being that the file would be written to /var/tmp/... by (as now)
> B2, then moved across by my script to where it was expected.
> 
> When invoked directly, this works quite happily. However, even more
> bizarrely, when I now call A, allowing it to invoke (my) B, I get exactly
> the same behaviour from my wrapper script as (the original) B was showing
> previously, specifically, it is unable to create the file /tmp/....
> 
> As a final workaround, I inserted instead added a sleep to my script in
> place of mv ..., and instead had an external process detect the presence
> of /var/tmp/... and move it across to /tmp. This, unsurprisingly, worked.
> 
> Interestingly, if I rewrote my wrapper script to,
> 
> B2 "$1" "$2" "/var$3"
> sleep 3
> cat "/var$3" > "$3" && rm "/var$3"
> 
> and had the external process simply touch /tmp/..., my wrapper script
> worked, suggesting that the permissions problem is to do with creating a
> new file, not writing to an existing one.
> 
> 
> A few final points:
> I've tried both an md based /tmp and tmpfs with the same result.
> Everything worked perfectly on 6.3 i386.
> If I run A as root, everything works without error.
> 
> My guess is that there's something a bit strange in linux_compat, either
> as a result of going to amd64 or to 7.1, and that affects both linux
> executables, and any processes that they create, but I'm not really sure
> beyond that. Can anyone shed any light on what might be going on?
> 
> 
> Kind Regards,
> 
> Christopher Key
> 

Whenever you do an upgrade between major versions there is ABI breakage. 
This necessitates either the rebuild of all installed ports or 
deleting/reinstalling. 

Since my needs are relatively simple from a server perspective when I change 
from one major version to another I start over from scratch and then pull in 
configs from backups. When upgrading within a major version this is not 
required, e.g., from 7.0 to 7.1 or 6.3 to 6.4. It's only a consideration 
when it is a jump like 6.x to 7.x.

The other approach is to use portupgrade to force all ports to be rebuilt 
linked against 7.x libs. One thing to watch out for is if you're not careful 
it is possible for some ports in a dependency chain to not be rebuilt and 
still linked against 6.x and some do get rebuilt linked against the new 7.x 
libs. This can give you flaky behavior. Byt forcing a massive 
upgrade/rebuild of everything causes all ports will get linked against 7.x 
libs during the rebuild.

Don't know if this is the source of your problem, but it may be something 
you can easily rule out.  

-Mike