Heavy system load by pagedaemon

[Iasen Kostov]

On Wed, 2006-05-10 at 11:49 -0500, Dan Nelson wrote:
> In the last episode (May 10), Iasen Kostov said:
> > On Wed, 2006-05-10 at 11:18 -0500, Dan Nelson wrote:
> > > In the last episode (May 10), Iasen Kostov said:
> > > > On Wed, 2006-05-10 at 15:32 +0300, Iasen Kostov wrote:
> > > > > On Wed, 2006-05-10 at 15:28 +0300, Iasen Kostov wrote:
> > > > > > I (probably) have some problem with a hosting server machine
> > > > > > halting about once a 30 to minute (probably when the free
> > > > > > memory falls under some threshold). At that moment the
> > > > > > machine totally halts for about 5 sec then resumes normal
> > > > > > work with load average around 9-15 (depends on system load)
> > > > > > from about 2-3. The system is:
> > > > > > (nice try :()
> > > > >  FreeBSD 6.1-PRERELEASE #7: Wed Mar 15 17:35:21 EET 2006
> > > > > 
> > > > >  ACPI APIC Table: <PTLTD          APIC  >
> > > > >  Timecounter "i8254" frequency 1193182 Hz quality 0
> > > > >  CPU: Dual Core AMD Opteron(tm) Processor 270 (2004.56-MHz K8-class CPU)
> > > > >    Origin = "AuthenticAMD"  Id = 0x20f12  Stepping = 2
> > > > > 
> > > > > Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
> > > > >    Features2=0x1<SSE3>
> > > > >    AMD Features=0xe2500800<SYSCALL,NX,MMX+,<b25>,LM,3DNow+,3DNow>
> > > > >  real memory  = 8724152320 (8320 MB)
> > > > >  avail memory = 8315379712 (7930 MB)
> > > > >  FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
> > > > > 
> > > > 	Hum, I was probably wrong about the free memory threshold - it
> > > > currently has ~1600MB of free and it is doing it again. When the
> > > > free memory falls under ~30MB it is just doing it more
> > > > frequently.
> > > 
> > > Are you swapping?  Paste in the output "top -o size".  If you have
> > > processes that are in the 1gb range, then when one exits you will
> > > end up with a lot of "free" memory for a short period of time, and
> > > it may try paging in another process that was completely paged out
> > > (if you are low enough on RAM for that to have happened).  That
> > > could cause pagedaemon activity.
> > > 
> > 
> > last pid: 81499;  load averages:  4.55,  6.48,  7.06
> > up 39+02:03:35  19:20:24
> > 361 processes: 6 running, 340 sleeping, 15 waiting
> > CPU states:  7.8% user,  0.0% nice,  5.6% system,  0.6% interrupt, 86.0% idle
> > Mem: 3435M Active, 3422M Inact, 462M Wired, 357M Cache, 214M Buf, 63M Free
> > Swap: 4096M Total, 1884K Used, 4094M Free
> 
> Well, you're not swapping, which is good.
>  
> >   PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
> > 80053 mysql       8  20    0   467M 45904K kserel 0   2:01  0.00% mysqld
> > 79134 nobody      1   4    0   243M  8332K sbwait 3   0:09  0.49% httpd
> > 79402 nobody      1   4    0   243M 58332K sbwait 3   0:10  0.00% httpd
> > 79341 nobody      1   4    0   242M 30156K sbwait 0   0:11  0.00% httpd
> > 79283 nobody      1   4    0   242M 56796K accept 0   0:07  0.10% httpd
> > 79281 nobody      1  -4    0   242M 63012K getblk 1   0:05  0.24% httpd
> > 79917 nobody      1   4    0   242M 59636K sbwait 3   0:10  0.00% httpd
> > ...
> > 
> > httpds are eating 242MB because I'm using eaccelerator which allocates
> > 100MB shared memory (or atleast that's what is in conf :) ) and I have
> > ~400 users.
> 
> I would have expected RES to be at least 100MB to account for that
> shared memory, but I could be wrong.  I don't have any suggestions on
> your paging problem, though.
> 

	OK, if somebody can tell me - if this is normal for pagedaemon to lock
the machine under heavy fragmentation and ~8G RAM or it's just FreeBSD
pagedaemon is Crappy (or the whole VM ?) and can't handle it and I
should explore other options (Linux, Solaris, older FreeBSD, other
BSDs ?) or it is just a bug (it's ~2 month before release but it should
be stable right ?).