/libexec/ld-elf.so.1: Cannot execute objects on /

[Miroslav Lachman]

John Baldwin wrote:
> On Saturday, December 11, 2010 11:51:41 am Miroslav Lachman wrote:
>> Miroslav Lachman wrote:
>>> Garrett Cooper wrote:
>>>> 2010/4/20 Miroslav Lachman<000.fbsd at quip.cz>:
>>>>> I have large storage partition (/vol0) mounted as noexec and nosuid.
>>>>> Then
>>>>> one directory from this partition is mounted by nullfs as "exec and
>>>>> suid" so
>>>>> anything on it can be executed.
>>>>>
>>>>> The directory contains full installation of jail. Jail is running
>>>>> fine, but
>>>>> some ports (PHP for example) cannot be compiled inside the jail with
>>>>> message:
>>>>>
>>>>> /libexec/ld-elf.so.1: Cannot execute objects on /
>>>>>
>>>>> The same apply to executing of apxs
>>>>>
>>>>> root at rainnew ~/# /usr/local/sbin/apxs -q MPM_NAME
>>>>> /libexec/ld-elf.so.1: Cannot execute objects on /
>>>>>
>>>>> apxs:Error: Sorry, no shared object support for Apache.
>>>>> apxs:Error: available under your platform. Make sure.
>>>>> apxs:Error: the Apache module mod_so is compiled into.
>>>>> apxs:Error: your server binary '/usr/local/sbin/httpd'..
>>>>>
>>>>> (it should return "prefork")
>>>>>
>>>>> So I think there is some bug in checking the mountpoint options,
>>>>> where the
>>>>> check is made on "parent" of the nullfs instead of the nullfs target
>>>>> mountpoint.
>>>>>
>>>>> It is on 6.4-RELEASE i386 GENERIC. I did not test it on another release.
>>>>>
>>>>> This is list of related mount points:
>>>>>
>>>>> /dev/mirror/gm0s2d on /vol0 (ufs, local, noexec, nosuid, soft-updates)
>>>>> /vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local)
>>>>> /usr/ports on /vol0/jail/rain_new/usr/ports (nullfs, local)
>>>>> devfs on /vol0/jail/rain_new/dev (devfs, local)
>>>>>
>>>>> If I changed /vol0 options to (ufs, local, soft-updates) the above
>>>>> error is
>>>>> gone and apxs / compilation works fine.
>>>>>
>>>>> Can somebody look at this problem?
>>>>
>>>> Can you please provide output from ktrace / truss for the issue?
>>>
>>> I did
>>> # ktrace /usr/local/sbin/apxs -q MPM_NAME
>>>
>>> The output is here http://freebsd.quip.cz/ld-elf/ktrace.out
>>>
>>> Let me know if you need something else.
>>>
>>> Thank you for your interest!
>>
>> The problem is still there in FreeBSD 8.1-RELEASE amd64 GENERIC (and in
>> 7.x).
>>
>> Can somebody say if this is a bug or an expected "feature"?
>
> I think this is the expected behavior as nullfs is simply re-exposing /vol0
> and it shouldn't be able to create a more privileged mount than the underlying
> mount I think (e.g. a read/write nullfs mount of a read-only filesystem would
> not make the underlying files read/write).  It can be used to provide less
> privilege (e.g. a readonly nullfs mount of a read/write filesystem does not
> allow writes via the nullfs layer).
>
> That said, I'm not sure exactly where the permission check is failing.
> execve() only checks MNT_NOEXEC on the "upper" vnode's mountpoint (i.e. the
> nullfs mountpoint) and the VOP_ACCESS(.., V_EXEC) check does not look at
> MNT_NOEXEC either.
>
> I do think there might be bugs in that a nullfs mount that specifies noexec or
> nosuid might not enforce the noexec or nosuid bits if the underlying mount
> point does not have them set (from what I can see).

Thank you for your explanation. Then it is strange, that there is bug, 
that allows execution on originally non executable mountpoint.
It should be mentioned in the bugs section of the mount_nullfs man page.

It would be useful, if 'mount' output shows inherited options for nullfs.

If parent is:
/dev/mirror/gm0s2d on /vol0 (ufs, local, noexec, nosuid, soft-updates)

Then nullfs line will be:
/vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local, noexec, 
nosuid)

instead of just
/vol0/jail/.nullfs/rain on /vol0/jail/rain_new (nullfs, local)


Then I can understand what is expected behavior, but our current state 
is half working, if I can execute scripts and binaries, run jail on it, 
but can't execute "apxs -q MPM_NAME" and few others.

Miroslav Lachman