Upcoming FreeBSD Security Advisory

[Borja Marcos]

On Dec 3, 2009, at 12:27 PM, Ivan Voras wrote:

> Borja Marcos wrote:
>> On Dec 1, 2009, at 2:20 AM, FreeBSD Security Officer wrote:
>>> A short time ago a "local root" exploit was posted to the full-disclosure
>>> mailing list; as the name suggests, this allows a local user to execute
>>> arbitrary code as root.
>> Dr. Strangelove, or How I learned to love the MAC subsystem.
> 
> Hi,
> 
> Could you point to, or write, some tutorial-like documentation on how you use the MAC for this particular purpose?
> 
> I tried reading the mac* man pages in several instances before but can't seem to connect the theory described in there with how to apply it in a practical way.

Could write, indeed. A problem with the MAC subsystem documentation is that it's too formal. But, my fault, I should have contributed long ago.

Let me at least explain how I'm using it for fun and profit ;) Maybe I should write something for the wiki and enhance it over time.

I have been using it for some time in a shared hosting server based on FreeBSD. It hosts many small websites, close to 1000 now, I think, so jail management was quite clumsy.

The server is a shared hosting server based on Apache. Each user has an ftp account, chrooted to his home directory, of course. Users can upload PHP and CGI scripts, without restrictions in principle.

My goals were:

1- Guarantee operating system integrity. Such a setup can suffer plenty of security compromises

2- Avoid escalation from one user to another. A compromised user account should not allow the user to read another user's files.

3- Keep it reasonably simple. Users only can manage their files by ftp. Although a next iteration could offer something more complete.

4- Allow CGI and PHP scripts written by customers to work without special modifications. If you give them a long list of requirements and restrictions, that will spell trouble.


The goals were hard to meet. For instance, the Unix permissions model, which is very outdated for this kind of application, presents a serious problem. It's an elementary good practice to run a web server as an unprivileged user.  So, html files (and, hence, php files) must be readable by all. But that means that a compromised user account, in case of escaping a chroot(), would be able to read other users' files.

Integrity is hard to keep when you allow users to run php, cgi's... 

The solution adopted was to use the FreeBSD's MAC subsystem, exactly two elements:

- The mac/biba policy, that assigns an integrity label to processes (subjects) and resources (objects) so that the following restrictions apply:

---- A high integrity subject cannot read a lower integrity object. Think about a classical /tmp/.whatever_rc bobby trap left by an untrusted (lower integrity).. user. Your /usr/bin/whatever program, ran with your higher integrity level, would not be able to read the booby trap, so you would be safe. The problem is that it's awkward to do some administration tasks, but not impossible.

---- A low integrity subject cannot modify a high integrity object. Our toothless root in my previous message, imagine that it comes from a compromised PHP script that was, of course, being executed with a low integrity label, cannot modify anything in the operating system, as anything is marked as high integrity by default. Again, it cannot leave bobby traps for other processes to execute. Not a crontab, an "atjob", etc, because it lacks the necessary integrity level to do so. Integrity level cannot be increased once a process has been created with a low level, and as this applies as well to kmem et all, I think that a root exploit to overcome this is less likely to work than the typical privilege escalation exploit.

---- There exists a special integrity label, biba/equal, which means "do not check integrity". For instance, the system administrator can fork a shell with no checks when it's necessary to examine a user's directory ("setpmac biba/equal bin/csh" does the trick), and the backup program can be executed with a biba/equal integrity label. It will be able to read any files without restriction.

---- Integrity labels can be applied to other resources such as network interfaces. Imagine you have a secondary network used for network based backups. If you label that network interface properly, a low integrity process spawned from a customer account will not be able to access that secondary network.


- The mac/ugidfw policy, that allows you to setup a sort of "ipfw-like" restrictions. In  our case, customers have uid numbers assigned belonging to a given range, say, 10000 - 20000, and the ugidfw policy is set up so that processes with a uid belonging to that interval cannot access resources belonging to a a different uid inside that interval. Processes with a uid belonging to this interval will have no interference from this module as long as the resource being accessed is owned by a non-customer uid. In that case only regular Unix permissions apply.

This allows us to have user files with universal read permissions, and run Apache as an untrusted user. Apache can read each customer's files, but a customer cannot read other customers' files.

PHP runs as a cgi (the websites are very low volume usually, so it's not an issue at all) and we use Apache's mod_suexec so that each user's CGI programs are executed with the user's credentials.

As mod_suexec uses the operating system mechanisms to acquire the user's credentials, instead of just doing a setuid(), we use /etc/login.conf to apply a MAC label (in this case a biba/low label) to the customer accounts.



Setting it up was a bit of a pain in the ass. Depending on how complex your setup is, expect to spend some time playing with ktrace. This added layer of security can create some unexpected problems. Programs expect to be able to write to /tmp or /var/tmp (I had to label them as biba/equal), etc, etc.

But the results have been good so far, and it's not so much of a hassle to manage it. In this case, a successful run of a root exploit such as our star exploit of this week, customer data could have been compromised although I can reasonably say that the operating system integrity would be more than reasonably safe.

There's a wrong assumption I made: the MAC subsystem should make a root exploit hard to achieve, and the latest security issue shows that indeed that's not necessarily the case. I chose not to chroot the runnnig CGI's so that they saw a complete operating system, avoiding the costs of lots of phone calls to support because their script got a text file and ran awk on it, etc, etc, you know. Keeping lots of copies of the OS is quite ineffective. And restricting access to mostly harmless programs such as ping can be a problem as well. One of my compromises (wrong, maybe) was to offer the closest thing to a complete system as possible.


Best regards,






Borja.