per user quotas inside jail?

Fri Aug 23 18:49:20 UTC 2013

On Fri, August 23, 2013 1:23 pm, Konstantin Belousov wrote:
> On Fri, Aug 23, 2013 at 01:05:24PM -0500, Valeri Galtsev wrote:
>> On Fri, August 23, 2013 11:05 am, Konstantin Belousov wrote:
>> > On Fri, Aug 23, 2013 at 09:24:32AM -0500, Valeri Galtsev wrote:
>> >> Dear Experts,
>> >> After searching the web, reading FreeBSD Docs, trying some hacks
>> found on
>> >> some discussion boards... I feel it is not easily possible. Yet, as
>> always
>> >> there may be some expert who knows how to do it:
>> >> How can one have per user quotas inside jail?
>> >> Basically, I would like to give users shell access to some server,
>> but
>> that I prefer to have in jail, where I will mount all filesystems they
>> need access to... and the only question is: how do I restrict them so
>> one
>> >> (or few) user doesn't fill up the whole filesystem. My mind is not
>> married
>> >> to any particular filesystem, UFS2, XFS, ZFS... - the only thing I
>> would
>> >> stay away from is NFS exporting on host and then NFS mounting in jail
>> (which may be easiest if not the only way quota wise).
>> >
>> > UFS quotas work regardless of jailed/non-jailed user.  The only
>> confusing
>> > issue is that quotas are per host uid.  In other words, if host and
>> jail
>> user, or two users from different jails has the same uid, you get one
>> quota setting applied and accounted for them.
>> >
>> > Usual mitigation is to ensure that user uids are globally unique.
>> >
>>
>> Thanks, Konstantin.
>>
>> Still it doesn't work for me. My system is:
>>
>> 9.1-RELEASE-p5 amd64
>>
>> Kernel: the same as GENERIC, with one option added:
>>
>> options         QUOTA                   # Add disk quota support
>>
>> filesystem with quota enabled is directly mounted (UFS; rw,userquota)
>> into
>> directory inside jail. User (with the same username and UID) exists on
>> the
>> host system and in jail. Quotas work on the host system. Quotas don't
>> work
>> inside jail, so this user can fill up the whole filesystem when logged
>> into jail (jail accepts ssh connections with different hostname...)
>>
>> Apart from that I tried a hack which I lifted from someone's FreeBSD 7
>> hack (only the variable name changed since then), namely:
>>
>> in kernel, in:
>>
>> /usr/src/sys/kern/vfs_syscalls.c
>>
>> I kicked out two lines:
>>
>>         if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS))
>>                 return (EPERM);
>>
>> (which basically obliterate that if done from inside jail as far as I
>> understand),
>>
>> rebuilt and installed this kernel; in file
>>
>> /etc/rc.d/quota
>>
>> removed line
>>
>> # KEYWORD: nojail
>>
>> Yet, I'm still where I was: quotas work outside jail, not inside jail...
>>
>> So, I'm at loss. I guess I will have to dive into zfs following Aaron
>> Kaufman's suggestion... Sigh.
>
> UFS quotas work per mount. So if jail root is on a filesystem which
> has no quotas configured, obviously the thing cannot work.
>
> You did not provided any details of your configuration, which makes
> a diagnostic impossible.
>

Hi Konstantin,

Thanks a lot for helping me! Sorry, my usual fault, not sufficient details...

Jails are set up pretty much as in:

http://www.freebsd.org/doc/handbook/jails-application.html

(directory names and locations are slightly different). Someone mentioned,
ezjail does virtually the same too - if that helps.

In /jail/mroot there is the structure resembling real system (binaries,
libraries,...) except for a few things that have to be writable inside
jail; those are replaced with symlinks pointing to these inside
subdirectory s:

ls -l /jail/mroot
total 48
drwxr-xr-x   2 root  wheel  1024 Aug 19 13:02 bin
drwxr-xr-x   7 root  wheel  1024 Aug 19 13:03 boot
dr-xr-xr-x   2 root  wheel   512 Aug 19 13:02 dev
lrwxr-xr-x   1 root  wheel     5 Aug 19 13:11 etc -> s/etc
lrwxr-xr-x   1 root  wheel     6 Aug 19 13:11 home -> s/home
drwxr-xr-x   3 root  wheel  1536 Aug 19 13:03 lib
drwxr-xr-x   3 root  wheel   512 Aug 19 13:03 libexec
drwxr-xr-x   2 root  wheel   512 Aug 19 13:02 media
drwxr-xr-x   2 root  wheel   512 Aug 19 13:02 mnt
dr-xr-xr-x   2 root  wheel   512 Aug 19 13:02 proc
drwxr-xr-x   2 root  wheel  2560 Aug 19 13:03 rescue
lrwxr-xr-x   1 root  wheel     6 Aug 19 13:11 root -> s/root
drwxr-xr-x   2 root  wheel   512 Aug 19 13:11 s
drwxr-xr-x   2 root  wheel  2560 Aug 19 13:03 sbin
lrwxr-xr-x   1 root  wheel    11 Aug 19 13:02 sys -> usr/src/sys
lrwxr-xr-x   1 root  wheel     5 Aug 19 13:11 tmp -> s/tmp
drwxr-xr-x  14 root  wheel   512 Aug 19 13:11 usr
lrwxr-xr-x   1 root  wheel     5 Aug 19 13:11 var -> s/var

particular jail lives in its root directory:

/jail/shell

/jail/mroot is nullfs readonly mounted onto /jail/shell, rw unique for
each shell filesystem is mounted into /jail/shell/s (and populated with
appropriate /etc, /var ....), filesystem that has to be with quotas is
mounted (UFS; rw,userquota) into

/jail/shell/s/home

This last one is the one in question: quotas on this work when user will
ssh to host system and will write to /jail/shell/s/home; quota does not
work if user will ssh into jail (which is accessible from network with
different hostname). When the user writes into /home in jail (into /s/home
actually, symlink points there which on host system is our
/jail/shell/s/home), the quotas do not work.

I don't quite understand what quota on jail root filesystem (enabled or
not enabled) has to do with quota on different filesystem that is mounted
inside that filesystem. Outside jail / has no quotas, different filesystem
mounted somewhere inside (/jail/shell/s/home or just /home or /var) with
quotas and it does honor quotas. Am I missing something trivial or
fundamental?

Thanks again for helping me! What other details could help?

Sincerely yours,
Valeri

++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++