Heavy system load by pagedaemon

[Iasen Kostov]

On Wed, 2006-05-10 at 11:18 -0500, Dan Nelson wrote:
> In the last episode (May 10), Iasen Kostov said:
> > On Wed, 2006-05-10 at 15:32 +0300, Iasen Kostov wrote:
> > > On Wed, 2006-05-10 at 15:28 +0300, Iasen Kostov wrote:
> > > > I (probably) have some problem with a hosting server machine
> > > > halting about once a 30 to minute (probably when the free memory
> > > > falls under some threshold). At that moment the machine totally
> > > > halts for about 5 sec then resumes normal work with load average
> > > > around 9-15 (depends on system load) from about 2-3. The system
> > > > is:
> > > > (nice try :()
> > >  FreeBSD 6.1-PRERELEASE #7: Wed Mar 15 17:35:21 EET 2006
> > > 
> > >  ACPI APIC Table: <PTLTD          APIC  >
> > >  Timecounter "i8254" frequency 1193182 Hz quality 0
> > >  CPU: Dual Core AMD Opteron(tm) Processor 270 (2004.56-MHz K8-class CPU)
> > >    Origin = "AuthenticAMD"  Id = 0x20f12  Stepping = 2
> > > 
> > > Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
> > >    Features2=0x1<SSE3>
> > >    AMD Features=0xe2500800<SYSCALL,NX,MMX+,<b25>,LM,3DNow+,3DNow>
> > >  real memory  = 8724152320 (8320 MB)
> > >  avail memory = 8315379712 (7930 MB)
> > >  FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
> > > 
> > 	Hum, I was probably wrong about the free memory threshold - it
> > currently has ~1600MB of free and it is doing it again. When the free
> > memory falls under ~30MB it is just doing it more frequently.
> 
> Are you swapping?  Paste in the output "top -o size".  If you have
> processes that are in the 1gb range, then when one exits you will end
> up with a lot of "free" memory for a short period of time, and it may
> try paging in another process that was completely paged out (if you are
> low enough on RAM for that to have happened).  That could cause
> pagedaemon activity.
> 

last pid: 81499;  load averages:  4.55,  6.48,  7.06
up 39+02:03:35  19:20:24
361 processes: 6 running, 340 sleeping, 15 waiting
CPU states:  7.8% user,  0.0% nice,  5.6% system,  0.6% interrupt, 86.0%
idle
Mem: 3435M Active, 3422M Inact, 462M Wired, 357M Cache, 214M Buf, 63M
Free
Swap: 4096M Total, 1884K Used, 4094M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU
COMMAND
80053 mysql       8  20    0   467M 45904K kserel 0   2:01  0.00% mysqld
79134 nobody      1   4    0   243M  8332K sbwait 3   0:09  0.49% httpd
79402 nobody      1   4    0   243M 58332K sbwait 3   0:10  0.00% httpd
79341 nobody      1   4    0   242M 30156K sbwait 0   0:11  0.00% httpd
79283 nobody      1   4    0   242M 56796K accept 0   0:07  0.10% httpd
79281 nobody      1  -4    0   242M 63012K getblk 1   0:05  0.24% httpd
79917 nobody      1   4    0   242M 59636K sbwait 3   0:10  0.00% httpd
...

httpds are eating 242MB because I'm using eaccelerator which allocates
100MB shared memory (or atleast that's what is in conf :) ) and I have
~400 users.