Bug in #! processing

[Sławek Żak]

Ceri Davies <ceri at submonkey.net> writes:

> On Thu, Sep 30, 2004 at 01:59:48PM +0200, S?awek ?ak wrote:
>> Ceri Davies <ceri at submonkey.net> writes:
>> > On Wed, Sep 29, 2004 at 01:47:06PM +0200, S?awek ?ak wrote:
>> 
>> >>     You should speel buggy as 'POSIX' in this case I guess.
>> 
>> > You're actually guessing though, right?  I can't find this in the
>> > standard; if you know it's there then I'd appreciate a reference.
>> 
>>     No reference on this. Vague memories of brokedness only.
>
> Good, I'm not losing my eyesight then.  In theory this means that we're
> free to do whatever we want, as the commit log for revision 1.21 of
> imgact_shell.c suggests.

    Yep. It seems so. But the vendor of Allegro CL would have to break all other
    platform support if they change -#! to something else. It's a no-win
    situation, although ...
    
>> > I believe that the FreeBSD behaviour is closer to "correct" than
>> > anything else we're seeing in this thread.  I should be able to specify
>> >
>> > 	#!/usr/bin/perl -w -0
>> >
>> > or whatever without having everything other than the first argument
>> > ignored.
>> 
>>     Would be nice. I admit.
>
> It *is* nice, and I do use it.

    Same for me. I *love* to use #!/usr/bin/env some-interpreter -a -b -c which
    frees my mind from $PATH considerations on this hellish mess I have to
    manage.
    
>> I like the bahavior of FreeBSD besides special
>>     treatment of # on the first line after #!. Allowing for comments on the
>>     first line is a strange excuse. Have you ever seen a script commenting on
>>     the interpreter execution or had a need to do so?
>
> No, but since this has been possible in FreeBSD for over 4.5 years, you
> can guarantee that someone is using it.

    What do you think of sysctl named say kern.exec_hash_compat (set to 1 by
    default) or kernel option (also set to old behavior) to `fix' the situation?
    
>> >>     It is confirmed by other
>> >>     supposedly compliant systems. I've checked before AIX 5.2, Solaris 8/9. Two
>> >>     raisins in the pie are Tru64 5.1B and HP-UX 11, which return some erm,
>> >>     strange results. For such script:
>> >> 
>> >> #!./main 1 2 3 -#!
>> >> print ok
>> >> 
>> >>     You get:
>> >> 
>> >> Main.c test
>> >> ./main
>> >> 1 2 3 -#!
>> >> ./tst.sh
>> >
>> > Linux 2.4.20 does this too.
>> 
>>     That's as silly as can get. When called as:
>> 
>>            interpreter -a1 -a2 -a3 script
>> 
>>     the argument parsing done by interpreter must be different then when invoked
>>     via #! mechanics. Argh!
>
> Yeah, it seems pretty gross. At least they get there though, unlike the
> Solaris/AIX examples ;-)

    True. But it's a trashbin class compatibility solution IMHO.
    
>> >>     The behavior I'd like to have, and which seems correct is not bothering with
>> >>     second, 3rd and so on occurence of #! in the first line of script. Feasible?
>> >>     I guess so. The only commercial product on my systems uses -#! switch on all
>> >>     platforms as a script file mark.
>> >
>> > That seems wrong too.  #! shouldn't be magic anywhere other than at the
>> > beginning of a file.
>> 
>>     Do you think that -#! argument is magic? Why is it so? It's not magic and
>>     should be passed without exec(2) interference.
>
> I don't think it should be, but you seemed to suggest that it should be
> in the paragraph above.  I may have misunderstood, in which case we're
> agreed on this.

    Um. Let mi clarify my position on this. I think that treatment of # anywhere
    besides the absolutely first character in file (and even this with !
    immediately following it) as special comment character is bad. The reason is
    stated below - there is more than shell that is executed by hash-bang magic
    and it can't/shouldn't be forced to interpret # as comment too.
    
>> >>     I don't see any explanation for current
>> >>     behavior therefore I'm reporting it.
>> >
>> > The explanation is that we only process that line up to a '#' or
>> > newline.  Backing out revision 1.21 of sys/kern/imgact_shell.c is one
>> > fix, or perhaps allowing a '#' character to be escaped.  I'm not sure if
>> > I see an overwhelming reason for either.
>> 
>>     I don't see a convincing use for comments on the first line of script. Hash
>>     is special already when treated as comment character. # is not a comment in
>>     any `scripting language'. It is a shell legacy and shouldn't be forced on
>>     the remaining universe.
>
> I agree(ish); I don't think that the kernel should do anything special
> here either and I think that the "correct" thing to do would be to back
> out that revision.  Unfortunately the FreeBSD userbase can write a lot
> of scripts in 4 and a half years and we probably can't get away with it.
> Perhaps it could be done in -CURRENT, but I'd really like to see some
> other opinions.  For clarity, what I'm proposing is the application of
> the attached diff.  Opinions from anyone?

    I'll look in your diff, but if anyone oposes to backing out 1.21 I would
    gladly accept the above mentioned shim defaulting to 4.x compatible
    behavior, possibly toggled to new behavior for CURRENT.

Thanks, /S