Practical limit to number of jails on a given host?

[Bjoern A. Zeeb]

On 6. Feb 2012, at 20:29 , Doug Barton wrote:

> Howdy,
> 
> Thinking about implementing a poor-man's virtualization solution with
> lots'o'jails, and wondering what people think about the practical limits
> of such a system. I realize that part of the answer is going to depend
> on CPU and RAM, so let's assume for the sake of argument that the answer
> to that bit is, "Lots of both."
> 
> So first question is, is there some sort of hard-coded limit somewhere?
> If not, what is the largest number of jails that you've created
> successfully/reliably on a system, and what are the specs for that system?


Yes, jails provide you 6 9s ... though that's not 99.9999% but 999999 is
the maximum number of jails.  And yes, I have started this many before --
without processes or anything.
It took a couple of days, due to some list handling, which could be
optimized.  You will find that once you get there, you'll have a syscall
which never returns...
You notice once the start loop slows down if you print a . every 100 or 1000.

The machine was a 4 or 8 core amd64 with 8G of memory.

I think I had a slide in
there:  http://www.bsdcan.org/2010/schedule/attachments/130_2010-bz-the-new-vvorld.pdf

I know if using vnets; you can get the 4k (or more) but I also have reports
that it may not scale.

The other limit you'll run into is the number of PIDs.

And eventually scheduling depending on what you want to do.


> And finally, has anyone run into trouble with a large number of IP
> addresses for the jails? ISTR that way back when, the IP addresses
> associated with a particular interface were stored in a linked list, so
> as you added more you would start seeing O(N) slowdown on a lot of
> network stuff in the kernel.

Yeah, we still do list walks here and there.

/bz

-- 
Bjoern A. Zeeb                                 You have to have visions!
   It does not matter how good you are. It matters what good you do!