Bizarre behaviour of Linux binary under 7.1

[Christopher Key]

On Mar 14 2009, Michael Powell wrote:

>Christopher Key wrote:
>
>> Hello,
>> 
>> I recently upgraded from 6.3 (i386) to 7.1p3 (amd64) with a view to 
>> experimenting with zfs. Mostly, everything went smoothly, but I am 
>> getting some very odd behaviour from a linux utility.
>> 
>> The program is very simple, it has two executables, A and B. A is 
>> invoked by the user, and based upon the options given builds a list of 
>> files to process. B is then repeatedly invoked by A with the name of a 
>> file to process, and the name of a non existent file to write the 
>> results to. When B returns, A reads the results from the output file, 
>> deletes it and moves on.
>> 
>> This all worked fine on 6.3, but cannot be made to work as intended on
>> 7.1.
>> 
>> After appropriate use of truss invoking B directly, I found that the
>> source of problems was B being unable to create its output file /tmp/...:
>> 
>> linux_open("/tmp/1234.tmp",0x42,0600)            ERR#13 'Permission
>> denied'
>> 
>> which is odd.  /tmp has suitable permissions:
>> 
>> #ls -al /tmp
>> drwxrwxrwt  12 root   wheel      720 14 Mar 11:25 .
>> drwxr-xr-x  21 root   wheel      512 13 Mar 10:32 ..
>> ...
>> 
>> and I can quite happily create a identically named file in /tmp myself:
>> #echo test >/tmp/1234.tmp
>> #cat /tmp/1234.tmp
>> test
>> #rm /tmp/1234.tmp
>> 
>> 
>> 
>> Bizarrely, however, if I instead invoke B and request its output go to
>> /var/tmp/... instead of /tmp/..., it completes successfully.
>> 
>> As a temporary workaround, I therefore tried to create a wrapper around 
>> B:
>> 
>> #mv /usr/local/bin/B /usr/local/bin/B2
>> 
>> #cat /usr/local/bin/B #!/bin/sh B2 "$1" "$2" "/var$3" mv "/var$3" "$3"
>> 
>> 
>> the idea being that the file would be written to /var/tmp/... by (as now)
>> B2, then moved across by my script to where it was expected.
>> 
>> When invoked directly, this works quite happily. However, even more
>> bizarrely, when I now call A, allowing it to invoke (my) B, I get exactly
>> the same behaviour from my wrapper script as (the original) B was showing
>> previously, specifically, it is unable to create the file /tmp/....
>> 
>> As a final workaround, I inserted instead added a sleep to my script in
>> place of mv ..., and instead had an external process detect the presence
>> of /var/tmp/... and move it across to /tmp. This, unsurprisingly, worked.
>> 
>> Interestingly, if I rewrote my wrapper script to,
>> 
>> B2 "$1" "$2" "/var$3"
>> sleep 3
>> cat "/var$3" > "$3" && rm "/var$3"
>> 
>> and had the external process simply touch /tmp/..., my wrapper script
>> worked, suggesting that the permissions problem is to do with creating a
>> new file, not writing to an existing one.
>> 
>> 
>> A few final points:
>> I've tried both an md based /tmp and tmpfs with the same result.
>> Everything worked perfectly on 6.3 i386.
>> If I run A as root, everything works without error.
>> 
>> My guess is that there's something a bit strange in linux_compat, either
>> as a result of going to amd64 or to 7.1, and that affects both linux
>> executables, and any processes that they create, but I'm not really sure
>> beyond that. Can anyone shed any light on what might be going on?
>> 
>> 
>> Kind Regards,
>> 
>> Christopher Key
>> 
>
>Whenever you do an upgrade between major versions there is ABI breakage. 
>This necessitates either the rebuild of all installed ports or 
>deleting/reinstalling. 
>
> Since my needs are relatively simple from a server perspective when I 
> change from one major version to another I start over from scratch and 
> then pull in configs from backups. When upgrading within a major version 
> this is not required, e.g., from 7.0 to 7.1 or 6.3 to 6.4. It's only a 
> consideration when it is a jump like 6.x to 7.x.
>
> The other approach is to use portupgrade to force all ports to be rebuilt 
> linked against 7.x libs. One thing to watch out for is if you're not 
> careful it is possible for some ports in a dependency chain to not be 
> rebuilt and still linked against 6.x and some do get rebuilt linked 
> against the new 7.x libs. This can give you flaky behavior. Byt forcing a 
> massive upgrade/rebuild of everything causes all ports will get linked 
> against 7.x libs during the rebuild.
>
>Don't know if this is the source of your problem, but it may be something 
>you can easily rule out.  
>

Thanks Mike,

This was a fresh install to an effectively clean disk. I then csupped and 
rebuilt with a custom kernel, before installing ports. The application in 
question isn't in the ports tree, and distributed in binary form, 
containing only the two executables. If relevant, I'm using linux_base-fc4.

Having written the above, it occurs to me that I should probably try a 
generic kernel too. I'll give that a go and report back.

Chris