From peter at pean.org Wed Apr 1 03:42:16 2009 From: peter at pean.org (=?ISO-8859-1?Q?Peter_Ankerst=E5l?=) Date: Wed Apr 1 03:42:24 2009 Subject: spamd inside jail. Message-ID: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> I cant start spamd inside my new jails running 7.2-PRERELEASE. [3679] warn: server socket setup failed, retry 1: spamd: could not create INET socket on ip:783: Can't assign requested address [3679] warn: server socket setup failed, retry 2: spamd: could not create INET socket on ip:783: Can't assign requested address [3679] error: spamd: could not create INET socket on ip:783: Can't assign requested address spamd: could not create INET socket on ip:783: Can't assign requested address running on FreeBSD new.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #2: Tue Mar 24 00:06:58 UTC 2009 peter@:/usr/obj/usr/src/sys/SAMURAI amd64 But on FreeBSD old.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #1: Sat Mar 28 15:30:25 CET 2009 peter@:/usr/obj/usr/src/sys/NINJA amd64 everything still works fine. other programs can bind to adresses and so on but not spamd. I tried to write something like this: http://www.perlmonks.org/?node_id=416119 but with tcp and port 783, not warnings or error messages. Do you think this is a jail-issue or a spamd issue? I have tried to config spamd to bind to alot of different addresses with same result 0.0.0.0, 127.0.0.1, the jails ip and so on. -- Peter Ankerst?l peter@pean.org http://www.pean.org/ From peter at pean.org Wed Apr 1 03:50:47 2009 From: peter at pean.org (=?ISO-8859-1?Q?Peter_Ankerst=E5l?=) Date: Wed Apr 1 03:50:54 2009 Subject: spamd inside jail. In-Reply-To: References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> Message-ID: <1D527067-D3BF-42C2-B3E0-92BBC8B129E2@pean.org> On Apr 1, 2009, at 12:46 PM, Redd Vinylene wrote: > On Wed, Apr 1, 2009 at 12:42 PM, Peter Ankerst?l > wrote: > I cant start spamd inside my new jails running 7.2-PRERELEASE. > > [3679] warn: server socket setup failed, retry 1: spamd: could not > create INET socket on ip:783: Can't assign requested address > [3679] warn: server socket setup failed, retry 2: spamd: could not > create INET socket on ip:783: Can't assign requested address > [3679] error: spamd: could not create INET socket on ip:783: Can't > assign requested address > spamd: could not create INET socket on ip:783: Can't assign > requested address > > running on > FreeBSD new.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #2: > Tue Mar 24 00:06:58 UTC 2009 peter@:/usr/obj/usr/src/sys/ > SAMURAI amd64 > > But on > FreeBSD old.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #1: > Sat Mar 28 15:30:25 CET 2009 peter@:/usr/obj/usr/src/sys/NINJA > amd64 > > everything still works fine. > > other programs can bind to adresses and so on but not spamd. > > I tried to write something like this: http://www.perlmonks.org/?node_id=416119 > but with tcp and port 783, not warnings or error messages. Do you > think this is > a jail-issue or a spamd issue? > > I have tried to config spamd to bind to alot of different addresses > with same result > 0.0.0.0, 127.0.0.1, the jails ip and so on. > > -- > Peter Ankerst?l > peter@pean.org > http://www.pean.org/ > > > For what it's worth, other than taking up a lot of resources, spamd > will hardly stop any spam. > > -- > http://www.home.no/reddvinylene I dont know if I was clear about that Im using p5-Mail-Spamassasin. But if you have any pointers what I should use instead I would be very happy. Maybe even a guide? :) It needs to be something simple. Not a large mysql-database and admin- interface. Jast a daemon that postfix could use to check for spam. Thanks! -- Peter Ankerst?l peter@pean.org http://www.pean.org/ From reddvinylene at gmail.com Wed Apr 1 04:01:17 2009 From: reddvinylene at gmail.com (Redd Vinylene) Date: Wed Apr 1 04:01:23 2009 Subject: spamd inside jail. In-Reply-To: <1D527067-D3BF-42C2-B3E0-92BBC8B129E2@pean.org> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> <1D527067-D3BF-42C2-B3E0-92BBC8B129E2@pean.org> Message-ID: On Wed, Apr 1, 2009 at 12:50 PM, Peter Ankerst?l wrote: > > On Apr 1, 2009, at 12:46 PM, Redd Vinylene wrote: > > On Wed, Apr 1, 2009 at 12:42 PM, Peter Ankerst?l wrote: > >> I cant start spamd inside my new jails running 7.2-PRERELEASE. >> >> [3679] warn: server socket setup failed, retry 1: spamd: could not create >> INET socket on ip:783: Can't assign requested address >> [3679] warn: server socket setup failed, retry 2: spamd: could not create >> INET socket on ip:783: Can't assign requested address >> [3679] error: spamd: could not create INET socket on ip:783: Can't assign >> requested address >> spamd: could not create INET socket on ip:783: Can't assign requested >> address >> >> running on >> FreeBSD new.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #2: Tue Mar >> 24 00:06:58 UTC 2009 peter@:/usr/obj/usr/src/sys/SAMURAI amd64 >> >> But on >> FreeBSD old.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #1: Sat Mar >> 28 15:30:25 CET 2009 peter@:/usr/obj/usr/src/sys/NINJA amd64 >> >> everything still works fine. >> >> other programs can bind to adresses and so on but not spamd. >> >> I tried to write something like this: >> http://www.perlmonks.org/?node_id=416119 >> but with tcp and port 783, not warnings or error messages. Do you think >> this is >> a jail-issue or a spamd issue? >> >> I have tried to config spamd to bind to alot of different addresses with >> same result >> 0.0.0.0, 127.0.0.1, the jails ip and so on. >> >> -- >> Peter Ankerst?l >> peter@pean.org >> http://www.pean.org/ >> >> > For what it's worth, other than taking up a lot of resources, spamd will > hardly stop any spam. > > -- > http://www.home.no/reddvinylene > > I dont know if I was clear about that Im using p5-Mail-Spamassasin. > But if you have any pointers what I should use instead I would be very > happy. > Maybe even a guide? :) > > It needs to be something simple. Not a large mysql-database and > admin-interface. > Jast a daemon that postfix could use to check for spam. > > Thanks! > -- > Peter Ankerst?l > peter@pean.org > http://www.pean.org/ > > Indeed, check out the postgrey port. I think it's exactly what you want. -- http://www.home.no/reddvinylene From pi at opsec.eu Wed Apr 1 04:05:05 2009 From: pi at opsec.eu (Kurt Jaeger) Date: Wed Apr 1 04:05:11 2009 Subject: spamd inside jail. In-Reply-To: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> Message-ID: <20090401110502.GD27326@home.opsec.eu> Hi! > I cant start spamd inside my new jails running 7.2-PRERELEASE. I use spamd_enable="YES" spamd_flags="-A -i " and it works fine. -- pi@opsec.eu +49 171 3101372 11 years to go ! From reddvinylene at gmail.com Wed Apr 1 04:10:30 2009 From: reddvinylene at gmail.com (Redd Vinylene) Date: Wed Apr 1 04:10:37 2009 Subject: spamd inside jail. In-Reply-To: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> Message-ID: On Wed, Apr 1, 2009 at 12:42 PM, Peter Ankerst?l wrote: > I cant start spamd inside my new jails running 7.2-PRERELEASE. > > [3679] warn: server socket setup failed, retry 1: spamd: could not create > INET socket on ip:783: Can't assign requested address > [3679] warn: server socket setup failed, retry 2: spamd: could not create > INET socket on ip:783: Can't assign requested address > [3679] error: spamd: could not create INET socket on ip:783: Can't assign > requested address > spamd: could not create INET socket on ip:783: Can't assign requested > address > > running on > FreeBSD new.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #2: Tue Mar > 24 00:06:58 UTC 2009 peter@:/usr/obj/usr/src/sys/SAMURAI amd64 > > But on > FreeBSD old.machine.tld 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #1: Sat Mar > 28 15:30:25 CET 2009 peter@:/usr/obj/usr/src/sys/NINJA amd64 > > everything still works fine. > > other programs can bind to adresses and so on but not spamd. > > I tried to write something like this: > http://www.perlmonks.org/?node_id=416119 > but with tcp and port 783, not warnings or error messages. Do you think > this is > a jail-issue or a spamd issue? > > I have tried to config spamd to bind to alot of different addresses with > same result > 0.0.0.0, 127.0.0.1, the jails ip and so on. > > -- > Peter Ankerst?l > peter@pean.org > http://www.pean.org/ > > For what it's worth, other than taking up a lot of resources, spamd will hardly stop any spam. -- http://www.home.no/reddvinylene From reddvinylene at gmail.com Wed Apr 1 04:53:15 2009 From: reddvinylene at gmail.com (Redd Vinylene) Date: Wed Apr 1 04:53:22 2009 Subject: spamd inside jail. In-Reply-To: <49D352B8.5070403@beardz.net> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> <49D352B8.5070403@beardz.net> Message-ID: On Wed, Apr 1, 2009 at 1:40 PM, Jase Thew wrote: > Redd Vinylene wrote: > > For what it's worth, other than taking up a lot of resources, spamd will >> hardly stop any spam. >> >> > It seems to be doing a pretty effective job of stopping spam here. > > Regards, > > Jase. > Without greylisting? -- http://www.home.no/reddvinylene From bazerka at beardz.net Wed Apr 1 04:56:29 2009 From: bazerka at beardz.net (Jase Thew) Date: Wed Apr 1 04:56:35 2009 Subject: spamd inside jail. In-Reply-To: References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> Message-ID: <49D352B8.5070403@beardz.net> Redd Vinylene wrote: > For what it's worth, other than taking up a lot of resources, spamd will > hardly stop any spam. > It seems to be doing a pretty effective job of stopping spam here. Regards, Jase. From reddvinylene at gmail.com Wed Apr 1 04:57:19 2009 From: reddvinylene at gmail.com (Redd Vinylene) Date: Wed Apr 1 04:57:28 2009 Subject: spamd inside jail. In-Reply-To: <49D352B8.5070403@beardz.net> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> <49D352B8.5070403@beardz.net> Message-ID: On Wed, Apr 1, 2009 at 1:40 PM, Jase Thew wrote: > Redd Vinylene wrote: > > For what it's worth, other than taking up a lot of resources, spamd will >> hardly stop any spam. >> >> > It seems to be doing a pretty effective job of stopping spam here. > > Regards, > > Jase. > When I ran spamd I used to get hundreds of spam emails a day. But the minute I switched to greylisting in Postfix, it all stopped. -- http://www.home.no/reddvinylene From reddvinylene at gmail.com Wed Apr 1 05:01:26 2009 From: reddvinylene at gmail.com (Redd Vinylene) Date: Wed Apr 1 05:01:33 2009 Subject: spamd inside jail. In-Reply-To: <20090401115702.GE27326@home.opsec.eu> References: <5FE6B06E-8E94-4EC0-AEDA-657DB3DD45F3@pean.org> <49D352B8.5070403@beardz.net> <20090401115702.GE27326@home.opsec.eu> Message-ID: On Wed, Apr 1, 2009 at 1:57 PM, Kurt Jaeger wrote: > Hi! > > > > For what it's worth, other than taking up a lot of resources, spamd > will > > >> hardly stop any spam. > > > > It seems to be doing a pretty effective job of stopping spam here. > > > Without greylisting? > > Yes. > > -- > pi@opsec.eu +49 171 3101372 11 years to > go ! > Fine. But I still don't like it. -- http://www.home.no/reddvinylene From peter at pean.org Thu Apr 2 10:43:50 2009 From: peter at pean.org (=?ISO-8859-1?Q?Peter_Ankerst=E5l?=) Date: Thu Apr 2 10:43:57 2009 Subject: Adding ips to running jail. Message-ID: Is it possible to add ip-addresses to an already running jail? -- Peter Ankerst?l peter@pean.org http://www.pean.org/ From bzeeb-lists at lists.zabbadoz.net Thu Apr 2 14:20:07 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Thu Apr 2 14:20:15 2009 Subject: Adding ips to running jail. In-Reply-To: References: Message-ID: <20090402211902.J15361@maildrop.int.zabbadoz.net> On Thu, 2 Apr 2009, Peter Ankerst?l wrote: > Is it possible to add ip-addresses to an already running jail? Not yet but possibly soon (in FreeBSD 8). -- Bjoern A. Zeeb The greatest risk is not taking one. From bugmaster at FreeBSD.org Mon Apr 6 04:06:58 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Apr 6 04:08:18 2009 Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org Message-ID: <200904061106.n36B6uZD061910@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/132092 jail [jail] jail can listen on *:port when jail_socket_unix o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 4 problems total. From peter at pean.org Mon Apr 6 07:42:16 2009 From: peter at pean.org (=?ISO-8859-1?Q?Peter_Ankerst=E5l?=) Date: Mon Apr 6 07:42:23 2009 Subject: jail + jailed zfs volume. Message-ID: <0A6042BC-49E2-4DA1-B152-61EFC11FE175@pean.org> I got it to work, but how do I get it to configure correctly after a reboot, or a restart of the jail? The jailid can change and so on? -- Peter Ankerst?l peter@pean.org http://www.pean.org/ From scheidell at secnap.net Mon Apr 6 10:47:39 2009 From: scheidell at secnap.net (Michael Scheidell) Date: Mon Apr 6 10:54:41 2009 Subject: anyone using ssl accellorator cards in jail? In-Reply-To: <1239038646.16390.2139.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> References: <49D22ADE.6070005@secnap.net> <1239038646.16390.2139.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> Message-ID: <49DA403B.6020204@secnap.net> Brian A. Seklecki wrote: > On Tue, 2009-03-31 at 07:38 -0700, Michael Scheidell wrote: > >> trying to speed things up. >> > > I suspect that syscalls that support acceleration will simply fall right > through the jail into the host kernel. > > I'll be testing that some time next week -- so I'll let you know. I > don't think file handle access to /dev/crypto is required for Engine > support. > > Again, I'll let you know ~BAS > > thanks Brian. wonder if you need one card per virtual ip? -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 > *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2009 Hot Company Award Finalist, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 _________________________________________________________________________ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ _________________________________________________________________________ From lavalamp at spiritual-machines.org Mon Apr 6 14:09:29 2009 From: lavalamp at spiritual-machines.org (Brian A. Seklecki) Date: Mon Apr 6 14:09:35 2009 Subject: anyone using ssl accellorator cards in jail? In-Reply-To: <49D22ADE.6070005@secnap.net> References: <49D22ADE.6070005@secnap.net> Message-ID: <1239038646.16390.2139.camel@soundwave.ws.pitbpa0.priv.collaborativefusion.com> On Tue, 2009-03-31 at 07:38 -0700, Michael Scheidell wrote: > trying to speed things up. I suspect that syscalls that support acceleration will simply fall right through the jail into the host kernel. I'll be testing that some time next week -- so I'll let you know. I don't think file handle access to /dev/crypto is required for Engine support. Again, I'll let you know ~BAS From jon at terabear.com Thu Apr 9 10:47:58 2009 From: jon at terabear.com (Jon Lybrook) Date: Thu Apr 9 10:48:06 2009 Subject: archives search not working + newbie question Message-ID: <49DE2628.1020203@terabear.com> Hi, Maybe this is a known issue with the mailman archives for this list, but a search results in: "Unable to read word database file '/usr/local/mailman/archives/private/freebsd-jail/htdig/db.words.db' Did you run htdig?" There's also a note saying the archive has yet to be built: http://lists.freebsd.org/pipermail/freebsd-jail/ I *was* searching for the answer to a problem I'm having setting up jails on FreeBSD 7.1. which maybe someone can help me with directly. Much of the documentation I'm reading sounds simple enough: cd /usr/src sudo make world DESTDIR = /my/jail ... But there's nothing in /usr/src, so the command results in make: don't know how to make world. Stop I use portmaster to update my packages, which are up to date. I've also found documentation for installing from the cdrom: cd /cdrom/7.1-RELEASE/base sudo sh install.sh DESTDIR=/my/jail This gives an intimidating message of: You are about to extract the base distribution into / - are you SURE you want to do this over your installed system (y/n) Any help on this would be appreciated. Thanks, Jon From alexey at renatasystems.org Thu Apr 9 12:27:55 2009 From: alexey at renatasystems.org (Alexey V. Degtyarev) Date: Thu Apr 9 12:28:02 2009 Subject: archives search not working + newbie question In-Reply-To: <49DE2628.1020203@terabear.com> References: <49DE2628.1020203@terabear.com> Message-ID: <20090409192751.GG50878@hs-4.renatasystems.org> You need to have a corresponding source tree in your /usr/src directory. You can use either sysinstall(8) program to fetch the sources (Configure -> Distributions -> [x] src) or the csup(1)'s method: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/synching.html The easiest way is to use sysinstall(8) as root. > I *was* searching for the answer to a problem I'm having setting up jails > on FreeBSD 7.1. which maybe someone can help me with directly. Much of the > documentation I'm reading sounds simple enough: > > cd /usr/src > sudo make world DESTDIR = /my/jail > ... > > But there's nothing in /usr/src, so the command results in > make: don't know how to make world. Stop -- Alexey V. Degtyarev From linimon at FreeBSD.org Thu Apr 9 19:15:41 2009 From: linimon at FreeBSD.org (linimon@FreeBSD.org) Date: Thu Apr 9 19:15:47 2009 Subject: kern/133265: [jail] is there a solution how to run nfs client in jail environment? Message-ID: <200904100215.n3A2FfDF075508@freefall.freebsd.org> Old Synopsis: is there a solution how to run nfs client in jail environment? New Synopsis: [jail] is there a solution how to run nfs client in jail environment? Responsible-Changed-From-To: freebsd-bugs->freebsd-jail Responsible-Changed-By: linimon Responsible-Changed-When: Fri Apr 10 02:15:18 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=133265 From jako.andras at eik.bme.hu Fri Apr 10 03:20:24 2009 From: jako.andras at eik.bme.hu (JAKO Andras) Date: Fri Apr 10 03:20:30 2009 Subject: lo0's IPv6 address overwritten Message-ID: Hi, Starting a jail with one IPv4 and one IPv6 address on a few days old RELENG_7 overwrites lo0's ::1 with the jail's IPv6 address. (The jail's addresses are preconfigured on lo1.) Is this expected behaviour? Or did I made something the wrong way? Here's ifconfig(8)'s output before and after executing jail(8), and also from inside of the jail. Thanks, Andras [root@splash /usr/home/goya]# uname -a FreeBSD splash.eik.bme.hu 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #8: Tue Apr 7 23:15:27 CEST 2009 root@splash.eik.bme.hu:/usr/obj/usr/src/sys/SPLASH amd64 [root@splash /usr/home/goya]# ifconfig -a bge0: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:58 media: Ethernet autoselect (none) status: no carrier bge1: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:59 media: Ethernet autoselect (10baseT/UTP ) status: active em0: flags=8843 metric 0 mtu 1500 options=19b ether 00:1b:21:30:ee:7e inet6 fe80::21b:21ff:fe30:ee7e%em0 prefixlen 64 scopeid 0x3 inet 152.66.115.62 netmask 0xffffff00 broadcast 152.66.115.255 inet6 2001:738:2001:2001::62 prefixlen 64 media: Ethernet autoselect (1000baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 lo1: flags=8049 metric 0 mtu 16384 inet 152.66.116.116 netmask 0xffffffff inet6 2001:738:2001:1000::2 prefixlen 128 [root@splash /usr/home/goya]# jail -l -U root -i /usr/jail/ro.noc r-noc.net.bme.hu "152.66.116.116,2001:738:2001:1000::2" /bin/sh /etc/rc 1 Loading configuration files. /etc/rc: WARNING: $hostname is not set -- see rc.conf(5). Creating and/or trimming log files:. ln: /dev/log: Operation not permitted Starting syslogd. ELF ldconfig path: /lib /usr/lib /usr/lib/compat 32-bit compatibility ldconfig path: /usr/lib32 Clearing /tmp (X related). Starting local daemons:. Updating motd. Starting cron. Local package initialization:. Fri Apr 10 11:42:40 CEST 2009 [root@splash /usr/home/goya]# ifconfig bge0: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:58 media: Ethernet autoselect (none) status: no carrier bge1: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:59 media: Ethernet autoselect (10baseT/UTP ) status: active em0: flags=8843 metric 0 mtu 1500 options=19b ether 00:1b:21:30:ee:7e inet6 fe80::21b:21ff:fe30:ee7e%em0 prefixlen 64 scopeid 0x3 inet 152.66.115.62 netmask 0xffffff00 broadcast 152.66.115.255 inet6 2001:738:2001:2001::62 prefixlen 64 media: Ethernet autoselect (1000baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 inet6 2001:738:2001:1000::2 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 lo1: flags=8049 metric 0 mtu 16384 inet 152.66.116.116 netmask 0xffffffff inet6 2001:738:2001:1000::2 prefixlen 128 [root@splash /usr/home/goya]# jexec 1 /bin/sh # ifconfig bge0: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:58 media: Ethernet autoselect (none) status: no carrier bge1: flags=8802 metric 0 mtu 1500 options=9b ether 00:23:7d:d5:46:59 media: Ethernet autoselect (10baseT/UTP ) status: active em0: flags=8843 metric 0 mtu 1500 options=19b ether 00:1b:21:30:ee:7e media: Ethernet autoselect (1000baseTX ) status: active lo0: flags=8049 metric 0 mtu 16384 inet6 2001:738:2001:1000::2 prefixlen 128 lo1: flags=8049 metric 0 mtu 16384 inet 152.66.116.116 netmask 0xffffffff inet6 2001:738:2001:1000::2 prefixlen 128 # From bzeeb-lists at lists.zabbadoz.net Fri Apr 10 07:05:07 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Fri Apr 10 07:05:13 2009 Subject: lo0's IPv6 address overwritten In-Reply-To: References: Message-ID: <20090410135647.E15361@maildrop.int.zabbadoz.net> On Fri, 10 Apr 2009, JAKO Andras wrote: > Hi, > > Starting a jail with one IPv4 and one IPv6 address on a few days old > RELENG_7 overwrites lo0's ::1 with the jail's IPv6 address. (The jail's > addresses are preconfigured on lo1.) > > Is this expected behaviour? Or did I made something the wrong way? > > Here's ifconfig(8)'s output before and after executing jail(8), and also > from inside of the jail. testing this on a bit older HEAD: ifconfig lo1 create inet6 2001:738:2001:1000::2/128 ifconfig lo0 ; ifconfig lo1 lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 lo1: flags=8049 metric 0 mtu 16384 options=3 inet6 2001:738:2001:1000::2 prefixlen 128 jail -l -U root -i / hostname "2001:738:2001:1000::2" /bin/sh 1 # sysctl security.jail.jailed security.jail.jailed: 1 # ifconfig lo0; ifconfig lo1 lo0: flags=8049 metric 0 mtu 16384 options=3 lo1: flags=8049 metric 0 mtu 16384 options=3 inet6 2001:738:2001:1000::2 prefixlen 128 [ another xterm ] bz@:~> sysctl security.jail.jailed security.jail.jailed: 0 bz@:~> ifconfig lo0 ; ifconfig lo1 lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 lo1: flags=8049 metric 0 mtu 16384 options=3 inet6 2001:738:2001:1000::2 prefixlen 128 .. ifconfig lo1 destroy I wonder what's going onfor you. Can you check with netstat -rn -f inet6 that what you are seeing is indeed true? Can you try starting the jail to get an interactive shell and not running any scripts like I did and check what happens then? /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From jako.andras at eik.bme.hu Fri Apr 10 09:51:52 2009 From: jako.andras at eik.bme.hu (JAKO Andras) Date: Fri Apr 10 09:51:58 2009 Subject: lo0's IPv6 address overwritten In-Reply-To: <20090410135647.E15361@maildrop.int.zabbadoz.net> References: <20090410135647.E15361@maildrop.int.zabbadoz.net> Message-ID: > testing this on a bit older HEAD: > > ifconfig lo1 create inet6 2001:738:2001:1000::2/128 > ifconfig lo0 ; ifconfig lo1 > lo0: flags=8049 metric 0 mtu 16384 > options=3 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 lo1: > flags=8049 metric 0 mtu 16384 > options=3 > inet6 2001:738:2001:1000::2 prefixlen 128 jail -l -U root -i / > hostname "2001:738:2001:1000::2" /bin/sh > 1 > # sysctl security.jail.jailed > security.jail.jailed: 1 > # ifconfig lo0; ifconfig lo1 > lo0: flags=8049 metric 0 mtu 16384 > options=3 > lo1: flags=8049 metric 0 mtu 16384 > options=3 > inet6 2001:738:2001:1000::2 prefixlen 128 This works fine here too. > I wonder what's going onfor you. Can you check with > netstat -rn -f inet6 > that what you are seeing is indeed true? It's always the same: Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 => default fe80::1%em0 UGS em0 ::1 ::1 UHL lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 2001:738:2001:1000::2 link#5 UHL lo0 2001:738:2001:2001::/64 link#3 UC em0 2001:738:2001:2001::62 00:1b:21:30:ee:7e UHL lo0 fe80::/10 ::1 UGRS lo0 fe80::%em0/64 link#3 UC em0 fe80::1%em0 00:06:52:7c:64:40 UHLW em0 fe80::21b:21ff:fe30:ee7e%em0 00:1b:21:30:ee:7e UHL lo0 fe80::%lo0/64 fe80::1%lo0 U lo0 fe80::1%lo0 link#4 UHL lo0 ff01:3::/32 link#3 UC em0 ff01:4::/32 ::1 UC lo0 ff01:5::/32 2001:738:2001:1000::2 UC lo1 ff02::/16 ::1 UGRS lo0 ff02::%em0/32 link#3 UC em0 ff02::%lo0/32 ::1 UC lo0 ff02::%lo1/32 2001:738:2001:1000::2 UC lo1 > Can you try starting the jail to get an interactive shell and not > running any scripts like I did and check what happens then? That works, and ifconfig doesn't show any change on lo0. I found that when I start a telnet in the jail to an arbitrary global IPv6 address, lo0's ::1 changes to the jail's IPv6 address. The routing table doesn't change. Andras From bzeeb-lists at lists.zabbadoz.net Fri Apr 10 10:05:08 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Fri Apr 10 10:05:15 2009 Subject: lo0's IPv6 address overwritten In-Reply-To: References: <20090410135647.E15361@maildrop.int.zabbadoz.net> Message-ID: <20090410165753.U15361@maildrop.int.zabbadoz.net> On Fri, 10 Apr 2009, JAKO Andras wrote: Hi, > This works fine here too. Good. >> I wonder what's going onfor you. Can you check with >> netstat -rn -f inet6 >> that what you are seeing is indeed true? > > It's always the same: I cannot see the /128 on lo0 so that's fine too. >> Can you try starting the jail to get an interactive shell and not >> running any scripts like I did and check what happens then? > > That works, and ifconfig doesn't show any change on lo0. Good, as I said above. > I found that when I start a telnet in the jail to an arbitrary global IPv6 > address, lo0's ::1 changes to the jail's IPv6 address. The routing table > doesn't change. telnet to where? To the jail IP? To an IP of the base system? To world? Which version of RELENG_7 are you on (as what does a few days mean)? /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From jako.andras at eik.bme.hu Fri Apr 10 10:28:59 2009 From: jako.andras at eik.bme.hu (JAKO Andras) Date: Fri Apr 10 10:29:05 2009 Subject: lo0's IPv6 address overwritten In-Reply-To: <20090410165753.U15361@maildrop.int.zabbadoz.net> References: <20090410135647.E15361@maildrop.int.zabbadoz.net> <20090410165753.U15361@maildrop.int.zabbadoz.net> Message-ID: > > I found that when I start a telnet in the jail to an arbitrary global IPv6 > > address, lo0's ::1 changes to the jail's IPv6 address. The routing table > > doesn't change. > > telnet to where? To the jail IP? To an IP of the base system? To world? I started telnet inside the jail, to the world: [root@splash /usr/home/goya]# jail -l -U root -i /usr/jail/ro.noc r-noc.net.bme.hu "2001:738:2001:1000::2" /bin/sh 1 # ifconfig lo0 ; ifconfig lo1 lo0: flags=8049 metric 0 mtu 16384 lo1: flags=8049 metric 0 mtu 16384 inet6 2001:738:2001:1000::2 prefixlen 128 # telnet 2001:738::abcd Trying 2001:738::abcd... ^C# # ifconfig lo0 ; ifconfig lo1 lo0: flags=8049 metric 0 mtu 16384 inet6 2001:738:2001:1000::2 prefixlen 128 lo1: flags=8049 metric 0 mtu 16384 inet6 2001:738:2001:1000::2 prefixlen 128 > Which version of RELENG_7 are you on (as what does a few days mean)? amd64, 7 April: FreeBSD r-noc.net.bme.hu 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #8: Tue Apr 7 23:15:27 CEST 2009 root@splash...:/usr/obj/usr/src/sys/SPLASH amd64 Andras From bzeeb-lists at lists.zabbadoz.net Fri Apr 10 10:35:14 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Fri Apr 10 10:35:20 2009 Subject: lo0's IPv6 address overwritten In-Reply-To: References: <20090410135647.E15361@maildrop.int.zabbadoz.net> <20090410165753.U15361@maildrop.int.zabbadoz.net> Message-ID: <20090410173415.S15361@maildrop.int.zabbadoz.net> On Fri, 10 Apr 2009, JAKO Andras wrote: >>> I found that when I start a telnet in the jail to an arbitrary global IPv6 >>> address, lo0's ::1 changes to the jail's IPv6 address. The routing table >>> doesn't change. >> >> telnet to where? To the jail IP? To an IP of the base system? To world? > > I started telnet inside the jail, to the world: > > [root@splash /usr/home/goya]# jail -l -U root -i /usr/jail/ro.noc > r-noc.net.bme.hu "2001:738:2001:1000::2" /bin/sh > 1 > # ifconfig lo0 ; ifconfig lo1 > lo0: flags=8049 metric 0 mtu 16384 > lo1: flags=8049 metric 0 mtu 16384 > inet6 2001:738:2001:1000::2 prefixlen 128 > # telnet 2001:738::abcd > Trying 2001:738::abcd... > ^C# > # ifconfig lo0 ; ifconfig lo1 > lo0: flags=8049 metric 0 mtu 16384 > inet6 2001:738:2001:1000::2 prefixlen 128 > lo1: flags=8049 metric 0 mtu 16384 > inet6 2001:738:2001:1000::2 prefixlen 128 *wow*, that's indeed ... confusing. I'll try to (get someone to) look into this. /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From jilles at stack.nl Sat Apr 11 09:40:09 2009 From: jilles at stack.nl (Jilles Tjoelker) Date: Sat Apr 11 09:40:16 2009 Subject: kern/133265: [jail] is there a solution how to run nfs client in jail environment? Message-ID: <200904111640.n3BGe8MR049684@freefall.freebsd.org> The following reply was made to PR kern/133265; it has been noted by GNATS. From: Jilles Tjoelker To: bug-followup@FreeBSD.org, pg@fincombank.com Cc: Subject: Re: kern/133265: [jail] is there a solution how to run nfs client in jail environment? Date: Sat, 11 Apr 2009 18:33:57 +0200 Consider using mount in the outside system, specifying a path under the jail root, for example basebox# mount -t nfs somehost:/path /jails/jail001/nfsmount -- Jilles Tjoelker From bugmaster at FreeBSD.org Mon Apr 13 04:06:57 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Apr 13 04:34:05 2009 Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org Message-ID: <200904131106.n3DB6t4U084987@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/132092 jail [jail] jail can listen on *:port when jail_socket_unix o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 5 problems total. From lars at sixfeetup.com Wed Apr 15 06:58:58 2009 From: lars at sixfeetup.com (Lars R. Noldan) Date: Wed Apr 15 07:23:06 2009 Subject: JailResourceLimits Message-ID: I recently stumbled across the following wikipage discussing Jail Resource Limits, (http://wiki.freebsd.org/JailResourceLimits) and was wondering if anyone can tell me which versions of FreeBSD are supported. A previous article I stumbled across for memory resource limits required the BSD4 scheduler, which while supported by FreeBSD 7.1-Release isn't the default. Thank you in advance for any help / suggestions you can offer. -- Thanks, Lars S i x F e e t U p , I n c . | http://www.sixfeetup.com Phone: +1 (317) 861-5948 x609 lars@sixfeetup.com ANNOUNCING the first Plone Immersive Training Experience | Sept. 10-11-12, 2009 http://sixfeetup.com/immerse From k.menshikov at peterhost.ru Fri Apr 17 07:06:53 2009 From: k.menshikov at peterhost.ru (=?UTF-8?B?0JzQtdC90YzRiNC40LrQvtCyINCa0L7QvdGB0YLQsNC90YLQuNC9?=) Date: Fri Apr 17 07:37:36 2009 Subject: CPU limit for jails under ULE scheduler Message-ID: <49E889B4.6030707@peterhost.ru> Hello all! I`m rewrire original cdjones patch for cpu limit for jail under ULE scheduler. So, this work simple. We count cpu usage for all jails, and if jail use cpu more than have shared cpu, we move his threads to IDLE queue. Jailed thread can use all avaliable cpu time, if system has avaliable cpu. If system under heavy load, jailed thread can`t use cpu long as ratio (shared cpu for jail/ all shared cpu) < (estimate usage cpu for jail / all usage cpu) . Unjailed thread are not subject to this regime. Interactive thread also are not subject to this regime. Add 2 sysctl kern.sched.total_sched_shares - total count shares cpu in system, increase if we have more cpu kern.sched.flush_estcpu_interval - flush estcpu interval in ticks, default is 2560 = 2 * 128 * 10, NCPU*stathz*sec, increase if we have more cpu For use cpu limit, you need use flag -S NSharedCPU in /usr/sbin/jail program. My example jail -S100 /usr/jails/root/ root.kostjn.pht 192.168.0.245 /bin/csh I`m tested this under 10 simultaneous process in jail and in main system. test program is infinity cycle an 8 core xeon, use RELENG_7. First run process in jail, and after in main system. This one process tracking cpu usage Jail root 1052 0.0 0.0 3692 784 p1 RJ 7:38PM 0:00.39 /test.o root 1052 21.2 0.0 3692 784 p1 RJ 7:38PM 0:02.40 /test.o root 1052 35.6 0.0 3692 784 p1 RJ 7:38PM 0:04.40 /test.o root 1052 47.5 0.0 3692 784 p1 RJ 7:38PM 0:06.41 /test.o root 1052 39.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 33.2 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 27.6 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 22.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 19.0 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 15.8 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 13.0 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 10.8 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 8.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /tes Main system root 1088 14.9 0.0 3692 780 p0 R 7:38PM 0:01.57 /root/test.o root 1088 30.8 0.0 3692 780 p0 R 7:38PM 0:03.60 /root/test.o root 1088 43.8 0.0 3692 780 p0 R 7:38PM 0:05.60 /root/test.o root 1088 51.0 0.0 3692 780 p0 R 7:38PM 0:07.25 /root/test.o root 1088 50.8 0.0 3692 780 p0 R 7:38PM 0:08.28 /root/test.o root 1088 49.1 0.0 3692 780 p0 R 7:38PM 0:09.21 /root/test.o root 1088 48.1 0.0 3692 780 p0 R 7:38PM 0:10.24 /root/test.o root 1088 46.2 0.0 3692 780 p0 R 7:38PM 0:11.17 /root/test.o root 1088 42.9 0.0 3692 780 p0 R 7:38PM 0:11.95 /root/test.o So we see, that after run in main system, jailed process can`t usage cpu. I`m don`t have big expirience in kernel programming, consequently best if you see source code. Please communicate me about all problem in this patch. This is initial version, without tune jail parameter in runtime. Thank. Sorry for my bad english :) Original cdjones cpu and memory limit patch http://wiki.freebsd.org/JailResourceLimits -------------- next part -------------- diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/kern_jail.c sys.new/kern/kern_jail.c --- sys/kern/kern_jail.c 2009-03-10 22:33:50.000000000 +0300 +++ sys.new/kern/kern_jail.c 2009-04-17 18:51:34.000000000 +0400 @@ -531,6 +532,7 @@ kern_jail(struct thread *td, struct jail } #endif pr->pr_linux = NULL; + pr->pr_sched_shares = j->sched_shares; pr->pr_securelevel = securelevel; if (prison_service_slots == 0) pr->pr_slots = NULL; diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/sched_ule.c sys.new/kern/sched_ule.c --- sys/kern/sched_ule.c 2009-03-30 23:20:56.000000000 +0400 +++ sys.new/kern/sched_ule.c 2009-04-17 19:10:07.000000000 +0400 @@ -61,6 +61,7 @@ __FBSDID("$FreeBSD: src/sys/kern/sched_u #include #include #include +#include #ifdef KTRACE #include #include @@ -186,6 +187,22 @@ static int sched_interact = SCHED_INTERA static int realstathz; static int tickincr; static int sched_slice; + +#define ESTCPU_SHIFT 10 +/* + * estcpu: Global counter ticks from stat timer + * flush_estcpu_interval: Number ticks, after that we to zero estcpu, + * flush_estcpu_interval = mp_ncpus*stathz*10, + * default 2*128*10 = 2560 + * total_sched_shares: Total count shares cpu, 1000 per core, + * default 2*1000 = 2000 +*/ + + +static int estcpu; +static int flush_estcpu_interval = 2560; +static int total_sched_shares = 2000; + #ifdef PREEMPTION #ifdef FULL_PREEMPTION static int preempt_thresh = PRI_MAX_IDLE; @@ -2200,6 +2219,7 @@ sched_clock(struct thread *td) { struct tdq *tdq; struct td_sched *ts; + struct prison *pr = td->td_proc->p_ucred->cr_prison; THREAD_LOCK_ASSERT(td, MA_OWNED); tdq = TDQ_SELF(); @@ -2234,6 +2254,20 @@ sched_clock(struct thread *td) td->td_sched->ts_runtime += tickincr; sched_interact_update(td); } + + /* Increase counter and flush if need */ + estcpu++; + if (pr != NULL) + pr->pr_estcpu++; + + if (estcpu > flush_estcpu_interval){ + estcpu = 0; + LIST_FOREACH(pr, &allprison, pr_list) { + pr->pr_estcpu = 0; + } + CTR0(KTR_SCHED,"Flush estcpu and pr_estcpu for all jails"); + } + /* * We used up one time slice. */ @@ -2375,6 +2409,8 @@ tdq_add(struct tdq *tdq, struct thread * int cpumask; #endif + struct prison *pr = td->td_proc->p_ucred->cr_prison; + TDQ_LOCK_ASSERT(tdq, MA_OWNED); KASSERT((td->td_inhibitors == 0), ("sched_add: trying to run inhibited thread")); @@ -2383,6 +2419,32 @@ tdq_add(struct tdq *tdq, struct thread * KASSERT(td->td_flags & TDF_INMEM, ("sched_add: thread swapped out")); + /* We move thread in IDLE queue if prison estimate cpu more than shares + * cpu and thread is not interactive. Use ESTCPU_SHIFT to avoid + * rounding away results */ + if(pr != NULL) + CTR6(KTR_SCHED,"pid %i, prison %i, pr_estcpu %i,\ + estcpu %i shares %i interact %i", + td->td_proc->p_pid,pr->pr_id,pr->pr_estcpu, + estcpu, pr->pr_sched_shares, sched_interact_score(td)); + if (pr != NULL && pr->pr_sched_shares != 0 && + sched_interact_score(td) > sched_interact && + estcpu != 0 && total_sched_shares != 0){ + + if ((pr->pr_estcpu << ESTCPU_SHIFT) / (estcpu) > + (pr->pr_sched_shares << ESTCPU_SHIFT) / (total_sched_shares)) + { + td->td_priority = PRI_MIN_IDLE; + td->td_pri_class = PRI_IDLE; + CTR2(KTR_SCHED,"prison %i excess cpu limit!!! new pri = %i ",pr->pr_id,td->td_priority); + + } else { + CTR1(KTR_SCHED,"prison %i use cpu less limit",pr->pr_id); + sched_priority(td); + td->td_pri_class = PRI_TIMESHARE; + } + } + ts = td->td_sched; class = PRI_BASE(td->td_pri_class); TD_SET_RUNQ(td); @@ -2746,6 +2808,10 @@ SYSCTL_INT(_kern_sched, OID_AUTO, intera "Interactivity score threshold"); SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW, &preempt_thresh, 0,"Min priority for preemption, lower priorities have greater precedence"); +SYSCTL_INT(_kern_sched, OID_AUTO, flush_estcpu_interval, CTLFLAG_RW, &flush_estcpu_interval, + 0,"Number ticks stat timer after thar we zero estcpu counter"); +SYSCTL_INT(_kern_sched, OID_AUTO, total_sched_shares, CTLFLAG_RW, &total_sched_shares, + 0,"Total number shared cpu for system"); #ifdef SMP SYSCTL_INT(_kern_sched, OID_AUTO, pick_pri, CTLFLAG_RW, &pick_pri, 0, "Pick the target cpu based on priority rather than load."); diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/sys/jail.h sys.new/sys/jail.h --- sys/sys/jail.h 2009-02-18 23:12:08.000000000 +0300 +++ sys.new/sys/jail.h 2009-04-17 18:53:43.000000000 +0400 @@ -31,6 +31,7 @@ struct jail { uint32_t ip6s; struct in_addr *ip4; struct in6_addr *ip6; + uint32_t sched_shares; }; #define JAIL_API_VERSION 2 @@ -132,6 +133,9 @@ struct prison { struct task pr_task; /* (d) destroy task */ struct mtx pr_mtx; void **pr_slots; /* (p) additional data */ + uint32_t pr_estcpu; /* (p) cpu usage */ + uint32_t pr_sched_shares; /* (c) number virtual cpu */ + int pr_ip4s; /* (c) number of v4 IPs */ struct in_addr *pr_ip4; /* (c) v4 IPs of jail */ int pr_ip6s; /* (c) number of v6 IPs */ diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines usr.sbin/jail/jail.c usr.sbin.new/jail/jail.c --- usr.sbin/jail/jail.c 2009-02-07 16:19:08.000000000 +0300 +++ usr.sbin.new/jail/jail.c 2009-04-17 18:57:15.000000000 +0400 @@ -83,6 +83,7 @@ main(int argc, char **argv) int ch, error, i, ngroups, securelevel; int hflag, iflag, Jflag, lflag, uflag, Uflag; char path[PATH_MAX], *jailname, *ep, *username, *JidFile, *ip; + uint32_t sched_shares = 0; static char *cleanenv; const char *shell, *p = NULL; long ltmp; @@ -94,7 +95,7 @@ main(int argc, char **argv) jailname = username = JidFile = cleanenv = NULL; fp = NULL; - while ((ch = getopt(argc, argv, "hiln:s:u:U:J:")) != -1) { + while ((ch = getopt(argc, argv, "hilS:n:s:u:U:J:")) != -1) { switch (ch) { case 'h': hflag = 1; @@ -115,6 +116,9 @@ main(int argc, char **argv) errx(1, "invalid securelevel: `%s'", optarg); securelevel = ltmp; break; + case 'S': + sched_shares = (uint32_t)strtol(optarg,NULL,10); + break; case 'u': username = optarg; uflag = 1; @@ -152,6 +156,8 @@ main(int argc, char **argv) if (jailname != NULL) j.jailname = jailname; + j.sched_shares = sched_shares; + /* Handle IP addresses. If requested resolve hostname too. */ bzero(&hints, sizeof(struct addrinfo)); hints.ai_protocol = IPPROTO_TCP; @@ -264,9 +270,10 @@ static void usage(void) { - (void)fprintf(stderr, "%s%s%s\n", + (void)fprintf(stderr, "%s%s%s%s\n", "usage: jail [-hi] [-n jailname] [-J jid_file] ", "[-s securelevel] [-l -u username | -U username] ", + "[-S number shared cpu] ", "path hostname [ip[,..]] command ..."); exit(1); } -------------- next part -------------- diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/kern_jail.c sys.new/kern/kern_jail.c --- sys/kern/kern_jail.c 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/kern/kern_jail.c 2009-04-17 20:23:40.000000000 +0400 @@ -156,6 +156,7 @@ jail(struct thread *td, struct jail_args goto e_dropvnref; pr->pr_ip = j.ip_number; pr->pr_linux = NULL; + pr->pr_sched_shares = j->sched_shares; pr->pr_securelevel = securelevel; if (prison_service_slots == 0) pr->pr_slots = NULL; diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/sched_ule.c sys.new/kern/sched_ule.c --- sys/kern/sched_ule.c 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/kern/sched_ule.c 2009-04-17 20:23:40.000000000 +0400 @@ -61,6 +61,7 @@ __FBSDID("$FreeBSD: src/sys/kern/sched_u #include #include #include +#include #ifdef KTRACE #include #include @@ -186,6 +187,22 @@ static int sched_interact = SCHED_INTERA static int realstathz; static int tickincr; static int sched_slice; + +#define ESTCPU_SHIFT 10 +/* + * estcpu: Global counter ticks from stat timer + * flush_estcpu_interval: Number ticks, after that we to zero estcpu, + * flush_estcpu_interval = mp_ncpus*stathz*10, + * default 2*128*10 = 2560 + * total_sched_shares: Total count shares cpu, 1000 per core, + * default 2*1000 = 2000 +*/ + + +static int estcpu; +static int flush_estcpu_interval = 2560; +static int total_sched_shares = 2000; + #ifdef PREEMPTION #ifdef FULL_PREEMPTION static int preempt_thresh = PRI_MAX_IDLE; @@ -2200,6 +2217,7 @@ sched_clock(struct thread *td) { struct tdq *tdq; struct td_sched *ts; + struct prison *pr = td->td_proc->p_ucred->cr_prison; THREAD_LOCK_ASSERT(td, MA_OWNED); tdq = TDQ_SELF(); @@ -2234,6 +2252,20 @@ sched_clock(struct thread *td) td->td_sched->ts_runtime += tickincr; sched_interact_update(td); } + + /* Increase counter and flush if need */ + estcpu++; + if (pr != NULL) + pr->pr_estcpu++; + + if (estcpu > flush_estcpu_interval){ + estcpu = 0; + LIST_FOREACH(pr, &allprison, pr_list) { + pr->pr_estcpu = 0; + } + CTR0(KTR_SCHED,"Flush estcpu and pr_estcpu for all jails"); + } + /* * We used up one time slice. */ @@ -2375,6 +2407,8 @@ tdq_add(struct tdq *tdq, struct thread * int cpumask; #endif + struct prison *pr = td->td_proc->p_ucred->cr_prison; + TDQ_LOCK_ASSERT(tdq, MA_OWNED); KASSERT((td->td_inhibitors == 0), ("sched_add: trying to run inhibited thread")); @@ -2383,6 +2417,32 @@ tdq_add(struct tdq *tdq, struct thread * KASSERT(td->td_flags & TDF_INMEM, ("sched_add: thread swapped out")); + /* We move thread in IDLE queue if prison estimate cpu more than shares + * cpu and thread is not interactive. Use ESTCPU_SHIFT to avoid + * rounding away results */ + if(pr != NULL) + CTR6(KTR_SCHED,"pid %i, prison %i, pr_estcpu %i,\ + estcpu %i shares %i interact %i", + td->td_proc->p_pid,pr->pr_id,pr->pr_estcpu, + estcpu, pr->pr_sched_shares, sched_interact_score(td)); + if (pr != NULL && pr->pr_sched_shares != 0 && + sched_interact_score(td) > sched_interact && + estcpu != 0 && total_sched_shares != 0){ + + if ((pr->pr_estcpu << ESTCPU_SHIFT) / (estcpu) > + (pr->pr_sched_shares << ESTCPU_SHIFT) / (total_sched_shares)) + { + td->td_priority = PRI_MIN_IDLE; + td->td_pri_class = PRI_IDLE; + CTR2(KTR_SCHED,"prison %i excess cpu limit!!! new pri = %i ",pr->pr_id,td->td_priority); + + } else { + CTR1(KTR_SCHED,"prison %i use cpu less limit",pr->pr_id); + sched_priority(td); + td->td_pri_class = PRI_TIMESHARE; + } + } + ts = td->td_sched; class = PRI_BASE(td->td_pri_class); TD_SET_RUNQ(td); @@ -2741,6 +2801,10 @@ SYSCTL_INT(_kern_sched, OID_AUTO, intera "Interactivity score threshold"); SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW, &preempt_thresh, 0,"Min priority for preemption, lower priorities have greater precedence"); +SYSCTL_INT(_kern_sched, OID_AUTO, flush_estcpu_interval, CTLFLAG_RW, &flush_estcpu_interval, + 0,"Number ticks stat timer after thar we zero estcpu counter"); +SYSCTL_INT(_kern_sched, OID_AUTO, total_sched_shares, CTLFLAG_RW, &total_sched_shares, + 0,"Total number shared cpu for system"); #ifdef SMP SYSCTL_INT(_kern_sched, OID_AUTO, pick_pri, CTLFLAG_RW, &pick_pri, 0, "Pick the target cpu based on priority rather than load."); diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/sys/jail.h sys.new/sys/jail.h --- sys/sys/jail.h 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/sys/jail.h 2009-04-17 20:26:54.000000000 +0400 @@ -18,6 +18,7 @@ struct jail { char *path; char *hostname; u_int32_t ip_number; + uint32_t sched_shares; }; struct xprison { @@ -74,6 +75,8 @@ struct prison { struct task pr_task; /* (d) destroy task */ struct mtx pr_mtx; void **pr_slots; /* (p) additional data */ + uint32_t pr_estcpu; /* (p) cpu usage */ + uint32_t pr_sched_shares; /* (c) number virtual cpu */ }; #endif /* _KERNEL || _WANT_PRISON */ diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines usr.sbin/jail/jail.c usr.sbin.new/jail/jail.c --- usr.sbin/jail/jail.c 2008-11-25 05:59:29.000000000 +0300 +++ usr.sbin.new/jail/jail.c 2009-04-17 20:31:17.000000000 +0400 @@ -57,6 +57,7 @@ main(int argc, char **argv) gid_t groups[NGROUPS]; int ch, i, iflag, Jflag, lflag, ngroups, securelevel, uflag, Uflag; char path[PATH_MAX], *ep, *username, *JidFile; + uint32_t sched_shares = 0; static char *cleanenv; const char *shell, *p = NULL; long ltmp; @@ -67,7 +68,7 @@ main(int argc, char **argv) username = JidFile = cleanenv = NULL; fp = NULL; - while ((ch = getopt(argc, argv, "ils:u:U:J:")) != -1) { + while ((ch = getopt(argc, argv, "ilS:s:u:U:J:")) != -1) { switch (ch) { case 'i': iflag = 1; @@ -82,6 +83,9 @@ main(int argc, char **argv) errx(1, "invalid securelevel: `%s'", optarg); securelevel = ltmp; break; + case 'S': + sched_shares = (uint32_t)strtol(optarg,NULL,10); + break; case 'u': username = optarg; uflag = 1; @@ -115,6 +119,7 @@ main(int argc, char **argv) j.version = 0; j.path = path; j.hostname = argv[1]; + j.sched_shares = sched_shares; if (inet_aton(argv[2], &in) == 0) errx(1, "Could not make sense of ip-number: %s", argv[2]); j.ip_number = ntohl(in.s_addr); @@ -182,9 +187,10 @@ static void usage(void) { - (void)fprintf(stderr, "%s%s%s\n", + (void)fprintf(stderr, "%s%s%s%s\n", "usage: jail [-i] [-J jid_file] [-s securelevel] [-l -u ", "username | -U username]", + "[-S number shared cpu] ", " path hostname ip-number command ..."); exit(1); } From bugmaster at FreeBSD.org Mon Apr 20 11:06:55 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Apr 20 11:08:19 2009 Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org Message-ID: <200904201106.n3KB6sk2033058@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/132092 jail [jail] jail can listen on *:port when jail_socket_unix o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 5 problems total. From k.menshikov at peterhost.ru Mon Apr 20 15:18:15 2009 From: k.menshikov at peterhost.ru (=?UTF-8?B?0JzQtdC90YzRiNC40LrQvtCyINCa0L7QvdGB0YLQsNC90YLQuNC9?=) Date: Mon Apr 20 15:18:22 2009 Subject: CPU limit for Jails(patch for ULE scheduler) Message-ID: <49EC926D.6020404@peterhost.ru> Hello all! Many users want have limits on resourse for jail, for examle cpu and memory limit. I`m rewrire original cdjones patch for cpu limit for jail under ULE scheduler. So, this work simple. We count cpu usage for all jails, and if jail use cpu more than have shared cpu, we move his threads to IDLE queue and return to TIMESHARE in reverse case. Jailed thread can use all avaliable cpu time, if system has avaliable cpu. If system under heavy load, jailed thread can`t use cpu long as ratio (shared cpu for jail/ all shared cpu) < (estimate usage cpu for jail / all usage cpu) . Unjailed thread and interactive thread are not subject to this regime. Add 2 sysctl kern.sched.total_sched_shares - total count shares cpu in system, increase if we have more cpu kern.sched.flush_estcpu_interval - flush estcpu interval in ticks, default is 2560 = 2 * 128 * 10, NCPU*stathz*sec, increase if we have more cpu For use cpu limit, you need use flag -S NSharedCPU in /usr/sbin/jail program. My example jail -S100 /usr/jails/root/ root.kostjn.pht 192.168.0.245 /bin/csh I`m tested this under 10 simultaneous process in jail and in main system. test program is infinity cycle an 8 core xeon, use RELENG_7. First run process in jail, and after in main system. This one process tracking cpu usage Jail root 1052 0.0 0.0 3692 784 p1 RJ 7:38PM 0:00.39 /test.o root 1052 21.2 0.0 3692 784 p1 RJ 7:38PM 0:02.40 /test.o root 1052 35.6 0.0 3692 784 p1 RJ 7:38PM 0:04.40 /test.o root 1052 47.5 0.0 3692 784 p1 RJ 7:38PM 0:06.41 /test.o root 1052 39.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 33.2 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 27.6 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 22.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 19.0 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 15.8 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 13.0 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 10.8 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /test.o root 1052 8.9 0.0 3692 784 p1 RJ 7:38PM 0:06.62 /tes Main system root 1088 14.9 0.0 3692 780 p0 R 7:38PM 0:01.57 /root/test.o root 1088 30.8 0.0 3692 780 p0 R 7:38PM 0:03.60 /root/test.o root 1088 43.8 0.0 3692 780 p0 R 7:38PM 0:05.60 /root/test.o root 1088 51.0 0.0 3692 780 p0 R 7:38PM 0:07.25 /root/test.o root 1088 50.8 0.0 3692 780 p0 R 7:38PM 0:08.28 /root/test.o root 1088 49.1 0.0 3692 780 p0 R 7:38PM 0:09.21 /root/test.o root 1088 48.1 0.0 3692 780 p0 R 7:38PM 0:10.24 /root/test.o root 1088 46.2 0.0 3692 780 p0 R 7:38PM 0:11.17 /root/test.o root 1088 42.9 0.0 3692 780 p0 R 7:38PM 0:11.95 /root/test.o So we see, that after run in main system, jailed process can`t usage cpu. Please communicate me about all problem in this patch. This is initial version, without tune jail parameter in runtime. So, this work. But i`m not sure, that is best way. Attempt increase priority for jailed thread not work, because non interactive thread (that utilize many cpu) already have small prioriry(numerical high). Attempt decrease number ticks in cpu time slice, also not good idea, because, this increase number context switching on high load. May be you see other way for do this? Share you idea. Thank. Original cdjones cpu and memory limit patch http://wiki.freebsd.org/JailResourceLimits -------------- next part -------------- diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/kern_jail.c sys.new/kern/kern_jail.c --- sys/kern/kern_jail.c 2009-03-10 22:33:50.000000000 +0300 +++ sys.new/kern/kern_jail.c 2009-04-17 18:51:34.000000000 +0400 @@ -531,6 +532,7 @@ kern_jail(struct thread *td, struct jail } #endif pr->pr_linux = NULL; + pr->pr_sched_shares = j->sched_shares; pr->pr_securelevel = securelevel; if (prison_service_slots == 0) pr->pr_slots = NULL; diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/sched_ule.c sys.new/kern/sched_ule.c --- sys/kern/sched_ule.c 2009-03-30 23:20:56.000000000 +0400 +++ sys.new/kern/sched_ule.c 2009-04-17 19:10:07.000000000 +0400 @@ -61,6 +61,7 @@ __FBSDID("$FreeBSD: src/sys/kern/sched_u #include #include #include +#include #ifdef KTRACE #include #include @@ -186,6 +187,22 @@ static int sched_interact = SCHED_INTERA static int realstathz; static int tickincr; static int sched_slice; + +#define ESTCPU_SHIFT 10 +/* + * estcpu: Global counter ticks from stat timer + * flush_estcpu_interval: Number ticks, after that we to zero estcpu, + * flush_estcpu_interval = mp_ncpus*stathz*10, + * default 2*128*10 = 2560 + * total_sched_shares: Total count shares cpu, 1000 per core, + * default 2*1000 = 2000 +*/ + + +static int estcpu; +static int flush_estcpu_interval = 2560; +static int total_sched_shares = 2000; + #ifdef PREEMPTION #ifdef FULL_PREEMPTION static int preempt_thresh = PRI_MAX_IDLE; @@ -2200,6 +2219,7 @@ sched_clock(struct thread *td) { struct tdq *tdq; struct td_sched *ts; + struct prison *pr = td->td_proc->p_ucred->cr_prison; THREAD_LOCK_ASSERT(td, MA_OWNED); tdq = TDQ_SELF(); @@ -2234,6 +2254,20 @@ sched_clock(struct thread *td) td->td_sched->ts_runtime += tickincr; sched_interact_update(td); } + + /* Increase counter and flush if need */ + estcpu++; + if (pr != NULL) + pr->pr_estcpu++; + + if (estcpu > flush_estcpu_interval){ + estcpu = 0; + LIST_FOREACH(pr, &allprison, pr_list) { + pr->pr_estcpu = 0; + } + CTR0(KTR_SCHED,"Flush estcpu and pr_estcpu for all jails"); + } + /* * We used up one time slice. */ @@ -2375,6 +2409,8 @@ tdq_add(struct tdq *tdq, struct thread * int cpumask; #endif + struct prison *pr = td->td_proc->p_ucred->cr_prison; + TDQ_LOCK_ASSERT(tdq, MA_OWNED); KASSERT((td->td_inhibitors == 0), ("sched_add: trying to run inhibited thread")); @@ -2383,6 +2419,32 @@ tdq_add(struct tdq *tdq, struct thread * KASSERT(td->td_flags & TDF_INMEM, ("sched_add: thread swapped out")); + /* We move thread in IDLE queue if prison estimate cpu more than shares + * cpu and thread is not interactive. Use ESTCPU_SHIFT to avoid + * rounding away results */ + if(pr != NULL) + CTR6(KTR_SCHED,"pid %i, prison %i, pr_estcpu %i,\ + estcpu %i shares %i interact %i", + td->td_proc->p_pid,pr->pr_id,pr->pr_estcpu, + estcpu, pr->pr_sched_shares, sched_interact_score(td)); + if (pr != NULL && pr->pr_sched_shares != 0 && + sched_interact_score(td) > sched_interact && + estcpu != 0 && total_sched_shares != 0){ + + if ((pr->pr_estcpu << ESTCPU_SHIFT) / (estcpu) > + (pr->pr_sched_shares << ESTCPU_SHIFT) / (total_sched_shares)) + { + td->td_priority = PRI_MIN_IDLE; + td->td_pri_class = PRI_IDLE; + CTR2(KTR_SCHED,"prison %i excess cpu limit!!! new pri = %i ",pr->pr_id,td->td_priority); + + } else { + CTR1(KTR_SCHED,"prison %i use cpu less limit",pr->pr_id); + sched_priority(td); + td->td_pri_class = PRI_TIMESHARE; + } + } + ts = td->td_sched; class = PRI_BASE(td->td_pri_class); TD_SET_RUNQ(td); @@ -2746,6 +2808,10 @@ SYSCTL_INT(_kern_sched, OID_AUTO, intera "Interactivity score threshold"); SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW, &preempt_thresh, 0,"Min priority for preemption, lower priorities have greater precedence"); +SYSCTL_INT(_kern_sched, OID_AUTO, flush_estcpu_interval, CTLFLAG_RW, &flush_estcpu_interval, + 0,"Number ticks stat timer after thar we zero estcpu counter"); +SYSCTL_INT(_kern_sched, OID_AUTO, total_sched_shares, CTLFLAG_RW, &total_sched_shares, + 0,"Total number shared cpu for system"); #ifdef SMP SYSCTL_INT(_kern_sched, OID_AUTO, pick_pri, CTLFLAG_RW, &pick_pri, 0, "Pick the target cpu based on priority rather than load."); diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/sys/jail.h sys.new/sys/jail.h --- sys/sys/jail.h 2009-02-18 23:12:08.000000000 +0300 +++ sys.new/sys/jail.h 2009-04-17 18:53:43.000000000 +0400 @@ -31,6 +31,7 @@ struct jail { uint32_t ip6s; struct in_addr *ip4; struct in6_addr *ip6; + uint32_t sched_shares; }; #define JAIL_API_VERSION 2 @@ -132,6 +133,9 @@ struct prison { struct task pr_task; /* (d) destroy task */ struct mtx pr_mtx; void **pr_slots; /* (p) additional data */ + uint32_t pr_estcpu; /* (p) cpu usage */ + uint32_t pr_sched_shares; /* (c) number virtual cpu */ + int pr_ip4s; /* (c) number of v4 IPs */ struct in_addr *pr_ip4; /* (c) v4 IPs of jail */ int pr_ip6s; /* (c) number of v6 IPs */ diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines usr.sbin/jail/jail.c usr.sbin.new/jail/jail.c --- usr.sbin/jail/jail.c 2009-02-07 16:19:08.000000000 +0300 +++ usr.sbin.new/jail/jail.c 2009-04-17 18:57:15.000000000 +0400 @@ -83,6 +83,7 @@ main(int argc, char **argv) int ch, error, i, ngroups, securelevel; int hflag, iflag, Jflag, lflag, uflag, Uflag; char path[PATH_MAX], *jailname, *ep, *username, *JidFile, *ip; + uint32_t sched_shares = 0; static char *cleanenv; const char *shell, *p = NULL; long ltmp; @@ -94,7 +95,7 @@ main(int argc, char **argv) jailname = username = JidFile = cleanenv = NULL; fp = NULL; - while ((ch = getopt(argc, argv, "hiln:s:u:U:J:")) != -1) { + while ((ch = getopt(argc, argv, "hilS:n:s:u:U:J:")) != -1) { switch (ch) { case 'h': hflag = 1; @@ -115,6 +116,9 @@ main(int argc, char **argv) errx(1, "invalid securelevel: `%s'", optarg); securelevel = ltmp; break; + case 'S': + sched_shares = (uint32_t)strtol(optarg,NULL,10); + break; case 'u': username = optarg; uflag = 1; @@ -152,6 +156,8 @@ main(int argc, char **argv) if (jailname != NULL) j.jailname = jailname; + j.sched_shares = sched_shares; + /* Handle IP addresses. If requested resolve hostname too. */ bzero(&hints, sizeof(struct addrinfo)); hints.ai_protocol = IPPROTO_TCP; @@ -264,9 +270,10 @@ static void usage(void) { - (void)fprintf(stderr, "%s%s%s\n", + (void)fprintf(stderr, "%s%s%s%s\n", "usage: jail [-hi] [-n jailname] [-J jid_file] ", "[-s securelevel] [-l -u username | -U username] ", + "[-S number shared cpu] ", "path hostname [ip[,..]] command ..."); exit(1); } -------------- next part -------------- diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/kern_jail.c sys.new/kern/kern_jail.c --- sys/kern/kern_jail.c 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/kern/kern_jail.c 2009-04-17 20:23:40.000000000 +0400 @@ -156,6 +156,7 @@ jail(struct thread *td, struct jail_args goto e_dropvnref; pr->pr_ip = j.ip_number; pr->pr_linux = NULL; + pr->pr_sched_shares = j->sched_shares; pr->pr_securelevel = securelevel; if (prison_service_slots == 0) pr->pr_slots = NULL; diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/kern/sched_ule.c sys.new/kern/sched_ule.c --- sys/kern/sched_ule.c 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/kern/sched_ule.c 2009-04-17 20:23:40.000000000 +0400 @@ -61,6 +61,7 @@ __FBSDID("$FreeBSD: src/sys/kern/sched_u #include #include #include +#include #ifdef KTRACE #include #include @@ -186,6 +187,22 @@ static int sched_interact = SCHED_INTERA static int realstathz; static int tickincr; static int sched_slice; + +#define ESTCPU_SHIFT 10 +/* + * estcpu: Global counter ticks from stat timer + * flush_estcpu_interval: Number ticks, after that we to zero estcpu, + * flush_estcpu_interval = mp_ncpus*stathz*10, + * default 2*128*10 = 2560 + * total_sched_shares: Total count shares cpu, 1000 per core, + * default 2*1000 = 2000 +*/ + + +static int estcpu; +static int flush_estcpu_interval = 2560; +static int total_sched_shares = 2000; + #ifdef PREEMPTION #ifdef FULL_PREEMPTION static int preempt_thresh = PRI_MAX_IDLE; @@ -2200,6 +2217,7 @@ sched_clock(struct thread *td) { struct tdq *tdq; struct td_sched *ts; + struct prison *pr = td->td_proc->p_ucred->cr_prison; THREAD_LOCK_ASSERT(td, MA_OWNED); tdq = TDQ_SELF(); @@ -2234,6 +2252,20 @@ sched_clock(struct thread *td) td->td_sched->ts_runtime += tickincr; sched_interact_update(td); } + + /* Increase counter and flush if need */ + estcpu++; + if (pr != NULL) + pr->pr_estcpu++; + + if (estcpu > flush_estcpu_interval){ + estcpu = 0; + LIST_FOREACH(pr, &allprison, pr_list) { + pr->pr_estcpu = 0; + } + CTR0(KTR_SCHED,"Flush estcpu and pr_estcpu for all jails"); + } + /* * We used up one time slice. */ @@ -2375,6 +2407,8 @@ tdq_add(struct tdq *tdq, struct thread * int cpumask; #endif + struct prison *pr = td->td_proc->p_ucred->cr_prison; + TDQ_LOCK_ASSERT(tdq, MA_OWNED); KASSERT((td->td_inhibitors == 0), ("sched_add: trying to run inhibited thread")); @@ -2383,6 +2417,32 @@ tdq_add(struct tdq *tdq, struct thread * KASSERT(td->td_flags & TDF_INMEM, ("sched_add: thread swapped out")); + /* We move thread in IDLE queue if prison estimate cpu more than shares + * cpu and thread is not interactive. Use ESTCPU_SHIFT to avoid + * rounding away results */ + if(pr != NULL) + CTR6(KTR_SCHED,"pid %i, prison %i, pr_estcpu %i,\ + estcpu %i shares %i interact %i", + td->td_proc->p_pid,pr->pr_id,pr->pr_estcpu, + estcpu, pr->pr_sched_shares, sched_interact_score(td)); + if (pr != NULL && pr->pr_sched_shares != 0 && + sched_interact_score(td) > sched_interact && + estcpu != 0 && total_sched_shares != 0){ + + if ((pr->pr_estcpu << ESTCPU_SHIFT) / (estcpu) > + (pr->pr_sched_shares << ESTCPU_SHIFT) / (total_sched_shares)) + { + td->td_priority = PRI_MIN_IDLE; + td->td_pri_class = PRI_IDLE; + CTR2(KTR_SCHED,"prison %i excess cpu limit!!! new pri = %i ",pr->pr_id,td->td_priority); + + } else { + CTR1(KTR_SCHED,"prison %i use cpu less limit",pr->pr_id); + sched_priority(td); + td->td_pri_class = PRI_TIMESHARE; + } + } + ts = td->td_sched; class = PRI_BASE(td->td_pri_class); TD_SET_RUNQ(td); @@ -2741,6 +2801,10 @@ SYSCTL_INT(_kern_sched, OID_AUTO, intera "Interactivity score threshold"); SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW, &preempt_thresh, 0,"Min priority for preemption, lower priorities have greater precedence"); +SYSCTL_INT(_kern_sched, OID_AUTO, flush_estcpu_interval, CTLFLAG_RW, &flush_estcpu_interval, + 0,"Number ticks stat timer after thar we zero estcpu counter"); +SYSCTL_INT(_kern_sched, OID_AUTO, total_sched_shares, CTLFLAG_RW, &total_sched_shares, + 0,"Total number shared cpu for system"); #ifdef SMP SYSCTL_INT(_kern_sched, OID_AUTO, pick_pri, CTLFLAG_RW, &pick_pri, 0, "Pick the target cpu based on priority rather than load."); diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines sys/sys/jail.h sys.new/sys/jail.h --- sys/sys/jail.h 2008-11-25 05:59:29.000000000 +0300 +++ sys.new/sys/jail.h 2009-04-17 20:26:54.000000000 +0400 @@ -18,6 +18,7 @@ struct jail { char *path; char *hostname; u_int32_t ip_number; + uint32_t sched_shares; }; struct xprison { @@ -74,6 +75,8 @@ struct prison { struct task pr_task; /* (d) destroy task */ struct mtx pr_mtx; void **pr_slots; /* (p) additional data */ + uint32_t pr_estcpu; /* (p) cpu usage */ + uint32_t pr_sched_shares; /* (c) number virtual cpu */ }; #endif /* _KERNEL || _WANT_PRISON */ diff -U3 -r --show-c-function --ignore-all-space --ignore-tab-expansion --ignore-blank-lines usr.sbin/jail/jail.c usr.sbin.new/jail/jail.c --- usr.sbin/jail/jail.c 2008-11-25 05:59:29.000000000 +0300 +++ usr.sbin.new/jail/jail.c 2009-04-17 20:31:17.000000000 +0400 @@ -57,6 +57,7 @@ main(int argc, char **argv) gid_t groups[NGROUPS]; int ch, i, iflag, Jflag, lflag, ngroups, securelevel, uflag, Uflag; char path[PATH_MAX], *ep, *username, *JidFile; + uint32_t sched_shares = 0; static char *cleanenv; const char *shell, *p = NULL; long ltmp; @@ -67,7 +68,7 @@ main(int argc, char **argv) username = JidFile = cleanenv = NULL; fp = NULL; - while ((ch = getopt(argc, argv, "ils:u:U:J:")) != -1) { + while ((ch = getopt(argc, argv, "ilS:s:u:U:J:")) != -1) { switch (ch) { case 'i': iflag = 1; @@ -82,6 +83,9 @@ main(int argc, char **argv) errx(1, "invalid securelevel: `%s'", optarg); securelevel = ltmp; break; + case 'S': + sched_shares = (uint32_t)strtol(optarg,NULL,10); + break; case 'u': username = optarg; uflag = 1; @@ -115,6 +119,7 @@ main(int argc, char **argv) j.version = 0; j.path = path; j.hostname = argv[1]; + j.sched_shares = sched_shares; if (inet_aton(argv[2], &in) == 0) errx(1, "Could not make sense of ip-number: %s", argv[2]); j.ip_number = ntohl(in.s_addr); @@ -182,9 +187,10 @@ static void usage(void) { - (void)fprintf(stderr, "%s%s%s\n", + (void)fprintf(stderr, "%s%s%s%s\n", "usage: jail [-i] [-J jid_file] [-s securelevel] [-l -u ", "username | -U username]", + "[-S number shared cpu] ", " path hostname ip-number command ..."); exit(1); } From 000.fbsd at quip.cz Tue Apr 21 22:03:54 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Tue Apr 21 22:04:01 2009 Subject: CPU limit for Jails(patch for ULE scheduler) In-Reply-To: <49EC926D.6020404@peterhost.ru> References: <49EC926D.6020404@peterhost.ru> Message-ID: <49EE42C5.3010409@quip.cz> ????????? ?????????? wrote: > Hello all! > Many users want have limits on resourse for jail, for examle cpu and > memory limit. > I`m rewrire original cdjones patch for cpu limit for jail under ULE > scheduler. > So, this work simple. > We count cpu usage for all jails, and if jail use cpu more than have > shared cpu, we move his threads to IDLE queue and return to TIMESHARE in > reverse case. > Jailed thread can use all avaliable cpu time, if system has avaliable cpu. > If system under heavy load, jailed thread can`t use cpu long as ratio > (shared cpu for jail/ all shared cpu) < (estimate usage cpu for jail / > all usage cpu) . > Unjailed thread and interactive thread are not subject to this regime. > Add 2 sysctl > kern.sched.total_sched_shares - total count shares cpu in system, > increase if we have more cpu > kern.sched.flush_estcpu_interval - flush estcpu interval in ticks, > default is 2560 = 2 * 128 * 10, NCPU*stathz*sec, increase if we have > more cpu > For use cpu limit, you need use flag -S NSharedCPU in /usr/sbin/jail > program. > My example jail -S100 /usr/jails/root/ root.kostjn.pht 192.168.0.245 > /bin/csh > > I`m tested this under 10 simultaneous process in jail and in main > system. test program is infinity cycle an 8 core xeon, use RELENG_7. > First run process in jail, and after in main system. > This one process tracking cpu usage [...] > So we see, that after run in main system, jailed process can`t usage cpu. > > Please communicate me about all problem in this patch. > This is initial version, without tune jail parameter in runtime. > > So, this work. But i`m not sure, that is best way. > > Attempt increase priority for jailed thread not work, because non > interactive thread (that utilize many cpu) already have small > prioriry(numerical high). > Attempt decrease number ticks in cpu time slice, also not good idea, > because, this increase number context switching on high load. > May be you see other way for do this? > Share you idea. > > Thank. > Original cdjones cpu and memory limit patch > http://wiki.freebsd.org/JailResourceLimits Hello, I can't judge your work / patch as I am not developer nor C programmer. But it is nice to see that someone is working on the resource limits. I am waiting for this feature for a years without success. The original SoC project was never done (not production ready). There were attempts by others to update cdjones patch to newer versions of FreeBSD, but still with some minor problems. The last I remember is "Memory limits on 7.0" by Christopher Thunes (e-mail in archive of this list from 2008-06-24). Unfortunately I had not time to test his patch in times of 7.0 and I am not aware of any newer version of this patch (for 7.1 or upcoming 7.2). It would be nice if independent developers can work together on this subject and do this work production / commit ready. May be you should open PR with you patch, so anybody can find it, test it and help to make it better. Or if you have own web page with this patch + some documentation, I can put the link to http://wiki.freebsd.org/Jails Can you take a look to Memory limits patch and incorporate it in to your patch? Do you have a plan to add jtune? Thanks for your work, I hope I will have time to test it in few weeks. Miroslav Lachman From bzeeb-lists at lists.zabbadoz.net Wed Apr 22 08:35:07 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Wed Apr 22 08:35:14 2009 Subject: Regarding multi-ip Bjoern head patch In-Reply-To: <49EEB93E.9080503@nixcraft.com> References: <49EEB93E.9080503@nixcraft.com> Message-ID: <20090422083240.Q15361@maildrop.int.zabbadoz.net> On Wed, 22 Apr 2009, Vivek Gite wrote: > Hi, > > I'm running FreeBSD 7.1_AMD_P4 release with 4 jails. Recently, our ISP > provided us IPv6 and I'd like to use both multiple IPv4 and IPv6 for my > jails. According to FreeBSD wiki (http://wiki.freebsd.org/Jails) - there is a > patch http://svn.freebsd.org/viewvc/base?view=revision&revision=188281 ; > which is committed to FreeBSD. But I'm not able to use it under said > version. So I'm looking to grab this one and manually patch it up. Is there > any tar-ball to grab a patch? Is it included in FreeBSD 7.2RC1? How do I grab > HEADS UP r185435? Yes, all of FreeBSD 7.2 (BETA, RC1, upcomig RC2 and RELEASE) have and will have it. So if you are going to update your system to any of those versions you'll have it. /bz PS: in case of reply please remove the -virtualization Cc: -- Bjoern A. Zeeb The greatest risk is not taking one. From bzeeb-lists at lists.zabbadoz.net Wed Apr 22 09:50:08 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Wed Apr 22 09:50:14 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <49EE4B6B.5020005@quip.cz> References: <49EE4B6B.5020005@quip.cz> Message-ID: <20090422094447.A15361@maildrop.int.zabbadoz.net> On Wed, 22 Apr 2009, Miroslav Lachman wrote: Hi, > I am running system FreeBSD 7.1-STABLE amd64 GENERIC (Wed Feb 11 09:56:08 CET > 2009) hosting few jails. > The machine has dual core CPU and some jails are set to run only on one core > (core 0 in this example): > > host# cpuset -l 0 -j 25 > > As I tested today, root user inside the jail can change this by the same > command as I am doing it from the host system: > > injail# cpuset -l 0,1 -j 25 > > And from now, jail with JID 25 is running on both cores. > > Is it expected behavior of cpuset to allow user inside the jail change cpuset > of the jail itself or is it a bug? > > It seems to me as undesirable. it is (undesirable) and it seems to be a bug as even if you do host# cpuset -l 0 -r -j 25 you can get back to 0,1 from within the jail. I'll check how/why this is possible. /bz PS: moving this to freebsd-jail@ -- Bjoern A. Zeeb The greatest risk is not taking one. From 000.fbsd at quip.cz Wed Apr 22 10:48:02 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Wed Apr 22 10:48:08 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <20090422094447.A15361@maildrop.int.zabbadoz.net> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> Message-ID: <49EEF5DB.4030408@quip.cz> Bjoern A. Zeeb wrote: > On Wed, 22 Apr 2009, Miroslav Lachman wrote: > > Hi, > >> I am running system FreeBSD 7.1-STABLE amd64 GENERIC (Wed Feb 11 >> 09:56:08 CET 2009) hosting few jails. >> The machine has dual core CPU and some jails are set to run only on >> one core (core 0 in this example): >> >> host# cpuset -l 0 -j 25 >> >> As I tested today, root user inside the jail can change this by the >> same command as I am doing it from the host system: >> >> injail# cpuset -l 0,1 -j 25 >> >> And from now, jail with JID 25 is running on both cores. >> >> Is it expected behavior of cpuset to allow user inside the jail change >> cpuset of the jail itself or is it a bug? >> >> It seems to me as undesirable. > > > it is (undesirable) and it seems to be a bug as even if you do > > host# cpuset -l 0 -r -j 25 > > you can get back to 0,1 from within the jail. > > I'll check how/why this is possible. > > /bz > > PS: moving this to freebsd-jail@ I found this behavior as result of your reply to my e-mail from March http://lists.freebsd.org/pipermail/freebsd-jail/2009-March/000751.html You are suggesting jail__exec_afterstart to use it for cpuset of starting jails, but as I look in to /etc/rc.d/jail, it seems this command is executed inside of the jail: while [ true ]; do eval out=\"\${_exec_afterstart${i}:-''}\" if [ -z "$out" ]; then break; fi jexec "${_jail_id}" ${out} i=$((i + 1)) done So I was confused if cpuset behavior i expected or not and if not, I don't know how to use current rc.d/jail + rc.conf to start jails on choosen cores or in particular set of cpus/cores. That was the reason to my suggestion - write patch for rc.d/jail to support something like: jail__cpuset_list="0,3,5" # start jail on cores 0, 3 and 5 It should be something like: _cpuset="cpuset -l ${_cpuset_list}" eval ${_cpuset} ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \ \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 (I didn't test the example above, so I don't know if it is valid) or something like: if [ -n "$_cpuset_list" ]; then cpuset -l ${_cpuset_list} -j ${_jail_id} fi (^ this seems more simpler) I don't know what is better, or if there is another way to set cpuset of jails from rc.conf But the first problem is as I previously posted - cpuset of jail should not be changed from within jail... Miroslav Lachman From stefan.lambrev at moneybookers.com Wed Apr 22 15:45:08 2009 From: stefan.lambrev at moneybookers.com (Stefan Lambrev) Date: Wed Apr 22 15:45:14 2009 Subject: HEADS UP: multi-IPv4/v6/no-IP jails now in 7-STABLE In-Reply-To: <20090207174104.Y93725@maildrop.int.zabbadoz.net> References: <20090207174104.Y93725@maildrop.int.zabbadoz.net> Message-ID: Hi, Does this allow multiple network interfaces to be used by a single jail instance? On Feb 7, 2009, at 8:18 PM, Bjoern A. Zeeb wrote: > Hi, > > what has started a long time ago with patches from various people, was > started, abandoned, resumed finally found an end. > > I am happy to hereby announce that the multi-IPv4/v6/no-IP jails work > has been merged to 7-STABLE and thus can be used in FreeBSD 7 without > the need to maintain or apply patches from now on. > > This also means that the updated jails will be included in 7.2 > release. > > This update gives you (short selection): > - zero, one or multi-IP jails. > - IPv4 and IPv6 support. > - cpuset support for jails. > - jail names and states to ease administration. - 32bit compat on > 64bit, jail v1 compat, .. > > You'll find a longer summary about all the new features and how to use > them in a posting from December (you should really read it): > http://lists.freebsd.org/pipermail/freebsd-jail/2008-December/000631.html > > Since the above posting, multiple PRs had been addressed and fixes > include > - SIOCGIFADDR ioctl handling which fixes the "samba inside jails > problem" > - no more arp and ndp information disclosure > - updated rc.conf framework (fully backward compatible in 7), see > man 5 rc.conf and /etc/defaults/rc.conf. > - various documentation/man page updates > - ... > > > I'd like to thank everyone who had helped to make this possible! > > > If you like the work, mayhap even use it for your business, or just > want > to support FreeBSD, you may want to visit > http://www.freebsdfoundation.org/ > and help donating some money. > > > Enjoy your new jails! > (and don't try to escape - you sure won't succeed;) > > /bz > > -- > Bjoern A. Zeeb The greatest risk is not taking > one. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " -- Best Wishes, Stefan Lambrev ICQ# 24134177 From 000.fbsd at quip.cz Wed Apr 22 20:26:04 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Wed Apr 22 20:26:11 2009 Subject: HEADS UP: multi-IPv4/v6/no-IP jails now in 7-STABLE In-Reply-To: References: <20090207174104.Y93725@maildrop.int.zabbadoz.net> Message-ID: <49EF7D57.9010307@quip.cz> Stefan Lambrev wrote: > Hi, > > Does this allow multiple network interfaces to be used by a single jail > instance? Yes, I am using it. root@cage ~/# jls -v JID Hostname Path Name State CPUSetID IP Address(es) 25 costa.example.com /vol0/jail/costa ALIVE 2 xxx.yy.105.31 192.168.222.57 root@costa //# ifconfig nfe0: flags=8843 metric 0 mtu 1500 options=19b ether 00:1a:24:bd:e2:0f inet 192.168.222.57 netmask 0xffffffff broadcast 192.168.222.57 media: Ethernet autoselect (100baseTX ) status: active [...] bge1: flags=8843 metric 0 mtu 1500 options=9b ether 00:1a:24:bd:e2:0e inet xxx.yy.105.31 netmask 0xffffffff broadcast xxx.yy.105.31 media: Ethernet autoselect (100baseTX ) status: active Above command (ifconfig) is inside jail, manually stripped other interfaces. (xxx.yy replaces real IP address) bge1 is used for internet connection and nfe0 for access services in LAN Miroslav Lachman From bzeeb-lists at lists.zabbadoz.net Thu Apr 23 14:25:07 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Thu Apr 23 14:25:14 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <49EEF5DB.4030408@quip.cz> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> <49EEF5DB.4030408@quip.cz> Message-ID: <20090423141908.T15361@maildrop.int.zabbadoz.net> On Wed, 22 Apr 2009, Miroslav Lachman wrote: Hi, > Bjoern A. Zeeb wrote: > >> On Wed, 22 Apr 2009, Miroslav Lachman wrote: >> >> Hi, >> >>> I am running system FreeBSD 7.1-STABLE amd64 GENERIC (Wed Feb 11 09:56:08 >>> CET 2009) hosting few jails. >>> The machine has dual core CPU and some jails are set to run only on one >>> core (core 0 in this example): >>> >>> host# cpuset -l 0 -j 25 >>> >>> As I tested today, root user inside the jail can change this by the same >>> command as I am doing it from the host system: >>> >>> injail# cpuset -l 0,1 -j 25 >>> >>> And from now, jail with JID 25 is running on both cores. >>> >>> Is it expected behavior of cpuset to allow user inside the jail change >>> cpuset of the jail itself or is it a bug? >>> >>> It seems to me as undesirable. >> >> >> it is (undesirable) and it seems to be a bug as even if you do >> >> host# cpuset -l 0 -r -j 25 >> >> you can get back to 0,1 from within the jail. >> >> I'll check how/why this is possible. >> >> /bz >> >> PS: moving this to freebsd-jail@ Ok, I am not sure what is going wrong here; well I know but I don't know if it's intended in cpuset. Trying to talk to the right people but they seen to be AWOL atm. If you are brave, you could try: http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff I haven't even compiled it yet. It may work, it may not work, it may make your machine panicing, ... just to warn you. it should still allow you to create further sets within a jail but you should not be able to change the "root set" of the jail from inside the jail anymore (in case it works;) /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From 000.fbsd at quip.cz Thu Apr 23 15:16:22 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Thu Apr 23 15:16:28 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <20090423141908.T15361@maildrop.int.zabbadoz.net> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> <49EEF5DB.4030408@quip.cz> <20090423141908.T15361@maildrop.int.zabbadoz.net> Message-ID: <49F08641.6060607@quip.cz> Bjoern A. Zeeb wrote: > On Wed, 22 Apr 2009, Miroslav Lachman wrote: [...] > Ok, I am not sure what is going wrong here; well I know but I don't > know if it's intended in cpuset. Trying to talk to the right people > but they seen to be AWOL atm. > > > If you are brave, you could try: > > http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff > > I haven't even compiled it yet. It may work, it may not work, it may > make your machine panicing, ... just to warn you. > > it should still allow you to create further sets within a jail but you > should not be able to change the "root set" of the jail from inside > the jail anymore (in case it works;) Thank you, I will try your patch today in Qemu testing environment and report back. Miroslav Lachman From 000.fbsd at quip.cz Thu Apr 23 23:22:12 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Thu Apr 23 23:22:19 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <20090423141908.T15361@maildrop.int.zabbadoz.net> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> <49EEF5DB.4030408@quip.cz> <20090423141908.T15361@maildrop.int.zabbadoz.net> Message-ID: <49F0F81F.8050503@quip.cz> Bjoern A. Zeeb wrote: [...] > Ok, I am not sure what is going wrong here; well I know but I don't > know if it's intended in cpuset. Trying to talk to the right people > but they seen to be AWOL atm. > > > If you are brave, you could try: > > http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff > > I haven't even compiled it yet. It may work, it may not work, it may > make your machine panicing, ... just to warn you. > > it should still allow you to create further sets within a jail but you > should not be able to change the "root set" of the jail from inside > the jail anymore (in case it works;) I did just a quick test. (OK, not so quick, because compilation inside Qemu on my old PC takes 2 hours ;]) It compiles without problems and did what I expect: root@72-rc1 ~/# jls JID IP Address Hostname Path 1 alpha.test /usr/jail/alpha root@72-rc1 ~/# jexec 1 tcsh root@alpha //# cpuset -l 0 -j 1 cpuset: setaffinity: Operation not permitted root@alpha //# cpuset -l 0 -r -j 1 cpuset: setaffinity: Operation not permitted I have no real multicore machine to test it more deeply. (can't test it on production servers and spare machine is blocked by another task) Will this fix be included in 7.2-RELEASE or is it too late to commit this fix? Miroslav Lachman From kagekonjou at gmail.com Fri Apr 24 00:39:06 2009 From: kagekonjou at gmail.com (Kage) Date: Fri Apr 24 00:39:13 2009 Subject: CPU limit for Jails(patch for ULE scheduler) In-Reply-To: <49EE42C5.3010409@quip.cz> References: <49EC926D.6020404@peterhost.ru> <49EE42C5.3010409@quip.cz> Message-ID: I'm definitely interested in this patch. I can see some good work coming from this. However, at this time, I do not have a development FreeBSD server I can heavily test this on. I'm definitely looking forward to a "stable" patch of this, and will make use of it. Please keep us informed! 2009/4/21 Miroslav Lachman <000.fbsd@quip.cz>: > ????????? ?????????? wrote: >> >> Hello all! >> Many users want have limits on resourse for jail, for examle cpu and >> memory limit. >> I`m rewrire original cdjones patch ?for cpu limit for jail under ULE >> scheduler. >> So, ?this work simple. >> We count cpu usage for all jails, and if jail use cpu more than have >> shared cpu, we move his threads to IDLE queue and return to TIMESHARE in >> ?reverse case. >> Jailed thread can use all avaliable cpu time, if ?system has avaliable >> cpu. >> If system under heavy load, jailed thread can`t use cpu long as ratio >> (shared cpu for jail/ all shared cpu) < (estimate usage cpu for jail / >> all usage cpu) . >> Unjailed thread and interactive thread are not subject to this regime. >> Add 2 sysctl >> kern.sched.total_sched_shares - total count shares cpu in system, >> increase if we have more cpu >> kern.sched.flush_estcpu_interval - flush estcpu interval in ticks, >> default is 2560 = 2 * 128 * 10, NCPU*stathz*sec, increase if we have >> more cpu >> For use cpu limit, you need use flag -S NSharedCPU in /usr/sbin/jail >> program. >> My example jail -S100 /usr/jails/root/ root.kostjn.pht ?192.168.0.245 >> /bin/csh >> >> I`m tested this under 10 simultaneous process in jail and in main >> system. test program is infinity cycle an 8 core xeon, use RELENG_7. >> First run process in jail, and after in main system. >> This one process tracking cpu usage > > [...] > >> So we see, that after run in main system, jailed process can`t usage cpu. >> >> Please communicate me ?about all problem in this patch. >> This is initial version, without tune jail parameter in runtime. >> >> So, this work. But i`m not sure, that is best way. >> >> Attempt increase priority for jailed thread not work, because non >> interactive thread (that utilize many cpu) already have small >> prioriry(numerical high). >> Attempt decrease number ticks in cpu time slice, also not good idea, >> because, this increase number context switching on high load. >> May be you see other way for do this? >> Share you idea. >> >> Thank. >> Original cdjones ?cpu and memory limit patch >> http://wiki.freebsd.org/JailResourceLimits > > Hello, > I can't judge your work / patch as I am not developer nor C programmer. But > it is nice to see that someone is working on the resource limits. I am > waiting for this feature for a years without success. The original SoC > project was never done (not production ready). There were attempts by others > to update cdjones patch to newer versions of FreeBSD, but still with some > minor problems. The last I remember is "Memory limits on 7.0" by Christopher > Thunes (e-mail in archive of this list from 2008-06-24). Unfortunately I had > not time to test his patch in times of 7.0 and I am not aware of any newer > version of this patch (for 7.1 or upcoming 7.2). > > It would be nice if independent developers can work together on this subject > and do this work production / commit ready. > > May be you should open PR with you patch, so anybody can find it, test it > and help to make it better. Or if you have own web page with this patch + > some documentation, I can put the link to http://wiki.freebsd.org/Jails > > Can you take a look to Memory limits patch and incorporate it in to your > patch? > > Do you have a plan to add jtune? > > Thanks for your work, I hope I will have time to test it in few weeks. > > Miroslav Lachman > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" > -- ~ Kage http://vitund.com http://hackthissite.org From bugmaster at FreeBSD.org Mon Apr 27 11:06:57 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Apr 27 11:08:25 2009 Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org Message-ID: <200904271106.n3RB6uce002332@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/132092 jail [jail] jail can listen on *:port when jail_socket_unix o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 5 problems total. From bzeeb-lists at lists.zabbadoz.net Mon Apr 27 21:00:11 2009 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Mon Apr 27 21:00:18 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <49F0F81F.8050503@quip.cz> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> <49EEF5DB.4030408@quip.cz> <20090423141908.T15361@maildrop.int.zabbadoz.net> <49F0F81F.8050503@quip.cz> Message-ID: <20090427205719.T15361@maildrop.int.zabbadoz.net> On Fri, 24 Apr 2009, Miroslav Lachman wrote: > Bjoern A. Zeeb wrote: > > [...] > >> Ok, I am not sure what is going wrong here; well I know but I don't >> know if it's intended in cpuset. Trying to talk to the right people >> but they seen to be AWOL atm. >> >> >> If you are brave, you could try: >> >> http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff >> >> I haven't even compiled it yet. It may work, it may not work, it may >> make your machine panicing, ... just to warn you. >> >> it should still allow you to create further sets within a jail but you >> should not be able to change the "root set" of the jail from inside >> the jail anymore (in case it works;) > > I did just a quick test. (OK, not so quick, because compilation inside Qemu > on my old PC takes 2 hours ;]) > It compiles without problems and did what I expect: > ... > I have no real multicore machine to test it more deeply. (can't test it on > production servers and spare machine is blocked by another task) > > Will this fix be included in 7.2-RELEASE or is it too late to commit this > fix? FreeBSD 7/7.2 just got a BUGS entry for the man pages. The patch will not make it; it's still waiting review for HEAD and possibly discussion if a super user inside a jail would still be allowed to further restrict the cpuset (but not extend it). /bz -- Bjoern A. Zeeb The greatest risk is not taking one. From 000.fbsd at quip.cz Mon Apr 27 21:48:34 2009 From: 000.fbsd at quip.cz (Miroslav Lachman) Date: Mon Apr 27 21:48:41 2009 Subject: changing cpuset of jail from inside of jail - is it feature? In-Reply-To: <20090427205719.T15361@maildrop.int.zabbadoz.net> References: <49EE4B6B.5020005@quip.cz> <20090422094447.A15361@maildrop.int.zabbadoz.net> <49EEF5DB.4030408@quip.cz> <20090423141908.T15361@maildrop.int.zabbadoz.net> <49F0F81F.8050503@quip.cz> <20090427205719.T15361@maildrop.int.zabbadoz.net> Message-ID: <49F6282E.8020807@quip.cz> Bjoern A. Zeeb wrote: > On Fri, 24 Apr 2009, Miroslav Lachman wrote: > >> Bjoern A. Zeeb wrote: >> >> [...] >> >>> Ok, I am not sure what is going wrong here; well I know but I don't >>> know if it's intended in cpuset. Trying to talk to the right people >>> but they seen to be AWOL atm. >>> >>> >>> If you are brave, you could try: >>> >>> http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff >>> >>> I haven't even compiled it yet. It may work, it may not work, it may >>> make your machine panicing, ... just to warn you. >>> >>> it should still allow you to create further sets within a jail but you >>> should not be able to change the "root set" of the jail from inside >>> the jail anymore (in case it works;) >> >> >> I did just a quick test. (OK, not so quick, because compilation inside >> Qemu on my old PC takes 2 hours ;]) >> It compiles without problems and did what I expect: >> > ... > >> I have no real multicore machine to test it more deeply. (can't test >> it on production servers and spare machine is blocked by another task) >> >> Will this fix be included in 7.2-RELEASE or is it too late to commit >> this fix? > > > FreeBSD 7/7.2 just got a BUGS entry for the man pages. The patch will > not make it; it's still waiting review for HEAD and possibly > discussion if a super user inside a jail would still be allowed to > further restrict the cpuset (but not extend it). OK, thank you for information. Allowing root inside jail to further restrict the cpuset for some services running inside jail seems useful to me. Just to inform others, this issue has PR number 134050 http://www.freebsd.org/cgi/query-pr.cgi?pr=134050 Miroslav Lachman From stefan.lambrev at moneybookers.com Thu Apr 30 17:31:33 2009 From: stefan.lambrev at moneybookers.com (Stefan Lambrev) Date: Thu Apr 30 17:31:40 2009 Subject: HEADS UP: multi-IPv4/v6/no-IP jails now in 7-STABLE In-Reply-To: <49EF7D57.9010307@quip.cz> References: <20090207174104.Y93725@maildrop.int.zabbadoz.net> <49EF7D57.9010307@quip.cz> Message-ID: <2FFE746D-9F46-4405-9CCE-01B3EF055EA0@moneybookers.com> Hi, On Apr 22, 2009, at 11:25 PM, Miroslav Lachman wrote: > Stefan Lambrev wrote: >> Hi, >> Does this allow multiple network interfaces to be used by a single >> jail instance? > > Yes, I am using it. > - cut - Basically it works, but I found another problem. I have created on two servers jails with 2 IPs on different interfaces. First IP is on "external" interface and second IP is on internal interface. As expected if I send packets from the host (outside jail) their source address match the IP of the interface (from which they are leaving the machine), but if I send packets from jail they always go out with source address equal to the first IP of the jail even when they are going out through the second interface. I do not know if this matters but in my case, internal interface have few vlans and the IP is set on the vlan not directly on the interface. Here is some output from the jail which can be useful: igb0: flags=8843 metric 0 mtu 1500 options=19b ether 00:30:48:9c:3a:0a inet 192.168.3.100 netmask 0xffffffff broadcast 192.168.3.100 media: Ethernet autoselect (100baseTX ) status: active igb1.2: flags=8843 metric 0 mtu 1500 options=3 ether 00:30:48:9c:3a:0b inet 10.35.1.1 netmask 0xffffff00 broadcast 10.35.1.255 media: Ethernet autoselect (1000baseTX ) status: active vlan: 2 parent interface: igb1 And here is the tcpdump from igb1.2 when trying to ping 10.35.1.2 from inside jail: 17:20:04.109972 IP 192.168.3.100 > 10.35.1.2: ICMP echo request, id 28421, seq 0, length 64 17:20:05.110321 IP 192.168.3.100 > 10.35.1.2: ICMP echo request, id 28421, seq 1, length 64 Any idea how this can be fixed? P.S. I know I can rewrite outgoing packets with firewall, but it's not performance wise, and I expect lot of udp multicast through igb1.2, that's why this doesn't look like a proper solution for me. -- Best Wishes, Stefan Lambrev ICQ# 24134177 From alexus at gmail.com Thu Apr 30 18:49:46 2009 From: alexus at gmail.com (alexus) Date: Thu Apr 30 18:49:52 2009 Subject: HEADS UP: multi-IPv4/v6/no-IP jails now in 7-STABLE In-Reply-To: <2FFE746D-9F46-4405-9CCE-01B3EF055EA0@moneybookers.com> References: <20090207174104.Y93725@maildrop.int.zabbadoz.net> <49EF7D57.9010307@quip.cz> <2FFE746D-9F46-4405-9CCE-01B3EF055EA0@moneybookers.com> Message-ID: <6ae50c2d0904301124x6b4ec794v81dc307e52e6e618@mail.gmail.com> thank you so much for all your hard work and all of your time! -- http://alexus.org/