per user quotas inside jail?
Konstantin Belousov
kostikbel at gmail.com
Sat Aug 24 15:08:43 UTC 2013
On Fri, Aug 23, 2013 at 01:49:19PM -0500, Valeri Galtsev wrote:
>
> On Fri, August 23, 2013 1:23 pm, Konstantin Belousov wrote:
> > On Fri, Aug 23, 2013 at 01:05:24PM -0500, Valeri Galtsev wrote:
> >> On Fri, August 23, 2013 11:05 am, Konstantin Belousov wrote:
> >> > On Fri, Aug 23, 2013 at 09:24:32AM -0500, Valeri Galtsev wrote:
> >> >> Dear Experts,
> >> >> After searching the web, reading FreeBSD Docs, trying some hacks
> >> found on
> >> >> some discussion boards... I feel it is not easily possible. Yet, as
> >> always
> >> >> there may be some expert who knows how to do it:
> >> >> How can one have per user quotas inside jail?
> >> >> Basically, I would like to give users shell access to some server,
> >> but
> >> that I prefer to have in jail, where I will mount all filesystems they
> >> need access to... and the only question is: how do I restrict them so
> >> one
> >> >> (or few) user doesn't fill up the whole filesystem. My mind is not
> >> married
> >> >> to any particular filesystem, UFS2, XFS, ZFS... - the only thing I
> >> would
> >> >> stay away from is NFS exporting on host and then NFS mounting in jail
> >> (which may be easiest if not the only way quota wise).
> >> >
> >> > UFS quotas work regardless of jailed/non-jailed user. The only
> >> confusing
> >> > issue is that quotas are per host uid. In other words, if host and
> >> jail
> >> user, or two users from different jails has the same uid, you get one
> >> quota setting applied and accounted for them.
> >> >
> >> > Usual mitigation is to ensure that user uids are globally unique.
> >> >
> >>
> >> Thanks, Konstantin.
> >>
> >> Still it doesn't work for me. My system is:
> >>
> >> 9.1-RELEASE-p5 amd64
> >>
> >> Kernel: the same as GENERIC, with one option added:
> >>
> >> options QUOTA # Add disk quota support
> >>
> >> filesystem with quota enabled is directly mounted (UFS; rw,userquota)
> >> into
> >> directory inside jail. User (with the same username and UID) exists on
> >> the
> >> host system and in jail. Quotas work on the host system. Quotas don't
> >> work
> >> inside jail, so this user can fill up the whole filesystem when logged
> >> into jail (jail accepts ssh connections with different hostname...)
> >>
> >> Apart from that I tried a hack which I lifted from someone's FreeBSD 7
> >> hack (only the variable name changed since then), namely:
> >>
> >> in kernel, in:
> >>
> >> /usr/src/sys/kern/vfs_syscalls.c
> >>
> >> I kicked out two lines:
> >>
> >> if (!prison_allow(td->td_ucred, PR_ALLOW_QUOTAS))
> >> return (EPERM);
> >>
> >> (which basically obliterate that if done from inside jail as far as I
> >> understand),
> >>
> >> rebuilt and installed this kernel; in file
> >>
> >> /etc/rc.d/quota
> >>
> >> removed line
> >>
> >> # KEYWORD: nojail
> >>
> >> Yet, I'm still where I was: quotas work outside jail, not inside jail...
> >>
> >> So, I'm at loss. I guess I will have to dive into zfs following Aaron
> >> Kaufman's suggestion... Sigh.
> >
> > UFS quotas work per mount. So if jail root is on a filesystem which
> > has no quotas configured, obviously the thing cannot work.
> >
> > You did not provided any details of your configuration, which makes
> > a diagnostic impossible.
> >
>
> Hi Konstantin,
>
> Thanks a lot for helping me! Sorry, my usual fault, not sufficient details...
>
> Jails are set up pretty much as in:
>
> http://www.freebsd.org/doc/handbook/jails-application.html
>
> (directory names and locations are slightly different). Someone mentioned,
> ezjail does virtually the same too - if that helps.
>
> In /jail/mroot there is the structure resembling real system (binaries,
> libraries,...) except for a few things that have to be writable inside
> jail; those are replaced with symlinks pointing to these inside
> subdirectory s:
>
> ls -l /jail/mroot
> total 48
> drwxr-xr-x 2 root wheel 1024 Aug 19 13:02 bin
> drwxr-xr-x 7 root wheel 1024 Aug 19 13:03 boot
> dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 dev
> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 etc -> s/etc
> lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 home -> s/home
> drwxr-xr-x 3 root wheel 1536 Aug 19 13:03 lib
> drwxr-xr-x 3 root wheel 512 Aug 19 13:03 libexec
> drwxr-xr-x 2 root wheel 512 Aug 19 13:02 media
> drwxr-xr-x 2 root wheel 512 Aug 19 13:02 mnt
> dr-xr-xr-x 2 root wheel 512 Aug 19 13:02 proc
> drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 rescue
> lrwxr-xr-x 1 root wheel 6 Aug 19 13:11 root -> s/root
> drwxr-xr-x 2 root wheel 512 Aug 19 13:11 s
> drwxr-xr-x 2 root wheel 2560 Aug 19 13:03 sbin
> lrwxr-xr-x 1 root wheel 11 Aug 19 13:02 sys -> usr/src/sys
> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 tmp -> s/tmp
> drwxr-xr-x 14 root wheel 512 Aug 19 13:11 usr
> lrwxr-xr-x 1 root wheel 5 Aug 19 13:11 var -> s/var
>
> particular jail lives in its root directory:
>
> /jail/shell
>
> /jail/mroot is nullfs readonly mounted onto /jail/shell, rw unique for
> each shell filesystem is mounted into /jail/shell/s (and populated with
> appropriate /etc, /var ....), filesystem that has to be with quotas is
> mounted (UFS; rw,userquota) into
>
> /jail/shell/s/home
>
> This last one is the one in question: quotas on this work when user will
> ssh to host system and will write to /jail/shell/s/home; quota does not
> work if user will ssh into jail (which is accessible from network with
> different hostname). When the user writes into /home in jail (into /s/home
> actually, symlink points there which on host system is our
> /jail/shell/s/home), the quotas do not work.
>
> I don't quite understand what quota on jail root filesystem (enabled or
> not enabled) has to do with quota on different filesystem that is mounted
> inside that filesystem. Outside jail / has no quotas, different filesystem
> mounted somewhere inside (/jail/shell/s/home or just /home or /var) with
> quotas and it does honor quotas. Am I missing something trivial or
> fundamental?
>
> Thanks again for helping me! What other details could help?
I decided that I have no desire to try to understand all the layers of
indirections which are only relevant to you anyway. Instead, I demostrate
you what I mean by working quotas. Below is the transcript of the simple
test.
sandy% mount -v /mnt ~
mount: /dev/ada1p4: Operation not permitted
/dev/ada1p4 on /mnt (ufs, local, with quotas, soft-updates, writes: sync 2 async 37, reads: sync 7 async 0)
sandy% sudo repquota -uah | grep kostik ~
kostik -- 14G 0 0 - 461057 0 0 -
sandy% sudo jail -u kostik / test1 127.0.0.1 /bin/sh ~
$ dd if=/dev/zero bs=1m of=/mnt/1/dddd count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 10.765265 secs (99741328 bytes/sec)
$ ^D% sandy% sudo repquota -uah | grep kostik ~
kostik -- 15G 0 0 - 461058 0 0 -
You could see that the accounted space and inodes are properly increased
after the dd.
IMO, you should make sure that the users operate on the filesystem which
has quotas enabled. Or, you should provide a simple to reproduce test
case, among the lines of the script I pasted above, for me to recreate
the issue locally.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-jail/attachments/20130824/2e147860/attachment.sig>
More information about the freebsd-jail
mailing list