Question about 'top' values on memory usage

[William LeFebvre]

Artem Kuchin wrote:
> Hello!
> 
> Maybe someone with deeper knowledge of the internals of FreeBSD
> can  clean up something for me (any for many others)^
> 
> Here are lines from my top:
> 
>  PID USERNAME    THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
> 9258 hordelo_ru    1   4    0 40992K  4260K accept 0   0:00  0.00% httpd
> 9257 hordelo_ru    1  44    0 40992K  4296K select 1   0:00  0.00% httpd
> 9259 hordelo_ru    1   4    0 40992K  4292K select 1   0:00  0.00% httpd
> 
> As you see, 'size' is the same for all processes, while RES varies.
> 
> As i understand, the real memory taken by a process is RES and SIZE include
> a bunch of shares .so libs, so, if more httpd's started each will take
> only about 4300K more, so, 100 https will take 430000K to run, right?
> 
> Another question is that is httpd uses threads (as provided by FreeBSD)
> starting a new thread will or will not copy executable copy and data? 
> Basically,
> will a new thread eat another 4300K or just a little bit for its data?
> 


SIZE is the total amount of virtual memory that a process has allocated. 
  This includes text, data, and stack.  It also includes all the stuff 
that's shared with other processes (mostly through the use of shared 
libraries).

RES is the amount of physical memory in use by the process and will only 
include that part of a process's virtual memory space which is currently 
allocated in physical memory.

Unfortunately, freebsd does not appear to track the amount of shared 
virtual memory for each process.  It could be obtained by walking 
through all the pages in a process's vm map, but that would really slow 
top down.  I don't know of any freebsd utility that would give that 
information for an individual process.  But hey, if it's out there 
somewhere where it is easy to grab, I would be very happy to add it to top.

 > All this i need to calculate maximum possible number of https i can run
 > on a box
 > with certain amount of memory and select proper MPM for Apache.
 > Somehow, i could not find any practical info on this regarding FreeBSD.


That's because the answer isn't as straightforward as you want it to be. 
  There are actually two things you need to worry about:  swap space and 
physical memory.  The amount of swap space will place a hard upper bound 
on the number of httpd processes you can run before sbrk (and thus 
malloc) start failing.  In order to determine that amount you are going 
to have to see how much extra swap is gobbled up by each new process. 
But in actuality that swap is only taken when needed, and if a data page 
remains untouched by the process then it won't need to allocate swap 
(backing store) for that page.  At least that's my limited understanding 
of how the freebsd VM model works.  But I would argue that long before 
you hit that hard upper limit you will hit a much more serious practical 
limit in just how much physical memory is adequate for the environment. 
  And that doesn't really limit the number of processes, rather it 
limits how many of them can be active at a given time.  You're not just 
going to be able to sit down and plug numbers in to a formula and say 
"voila!".  You will have to observe how httpd performs in your 
particular environment to see how many page faults per second it 
generates and decide for yourself the point at which X pf/s is too much.

Personally, based on my experience, I would be more concerned with the 
amount of available cpu cycles than memory.  In my experience, once you 
run out of idle time on a web server you have exceeded its capacity to 
serve pages.  In that situation it doesn't matter how many httpd 
processes there are, the system is still not able to keep up with 
demand.  And that will probably happen before the system starts 
thrashing from limited memory.

Bill LeFebvre