FreeBSD 6.0 - amount of total visible memory is decreasing in time

Ilya E Veretenkin ilyaver at mail.ru
Sat Jan 7 16:05:29 PST 2006


Problem: Total amount of visible memory is decreasing while system is
running
(total amount = sum of all memory fields in 'top' utility output, 
in other words: Active+Inact+Wired+Cache+Buf+Free)
OS: FreeBSD 6
Server: 2xAmd64bit, 4GB memory, tyan motherboard

hw.physmem: 3478589440
hw.realmem: 3488743424
These values don't change in time.

Just after server boot, the sum of memory fields in 'top' utility output is
3405M (this is correct: hw.physmem is the same). 
But this sum(i.e. total visible memory for OS) is decreasing in time, 
according to top's output. After all, server begins to swap and needs to be 
rebooted again.

Here goes monitoring log. It was created by hand-written perl script,
which is called every minute and which summarizes 'top' utility's 6
memory fields. Total memory(Active+Inact+Wired+Cache+Buf+Free)
is in the second column in the following log:

Sat Jan 7 20:37:00 MSK 2006 3224
Sat Jan 7 20:38:00 MSK 2006 3224 
Sat Jan 7 20:39:00 MSK 2006 3224 
Sat Jan 7 20:40:00 MSK 2006 3225 
Sat Jan 7 20:41:00 MSK 2006 3223 
Sat Jan 7 20:42:00 MSK 2006 3224 
Sat Jan 7 20:43:00 MSK 2006 3222 
Sat Jan 7 20:44:00 MSK 2006 3223 
Sat Jan 7 20:45:00 MSK 2006 3221 
Sat Jan 7 20:46:00 MSK 2006 3218 
Sat Jan 7 20:47:00 MSK 2006 3217 
Sat Jan 7 20:48:00 MSK 2006 3218 
Sat Jan 7 20:49:00 MSK 2006 3213 
Sat Jan 7 20:50:00 MSK 2006 3212 
Sat Jan 7 20:51:00 MSK 2006 3212 
Sat Jan 7 20:52:00 MSK 2006 3212 
Sat Jan 7 20:53:00 MSK 2006 3211 
Sat Jan 7 20:54:00 MSK 2006 3211 
Sat Jan 7 20:55:00 MSK 2006 3210 
Sat Jan 7 20:56:00 MSK 2006 3211 
Sat Jan 7 20:57:00 MSK 2006 3209 
Sat Jan 7 20:58:00 MSK 2006 3211 
Sat Jan 7 20:59:00 MSK 2006 3210 
Sat Jan 7 21:00:00 MSK 2006 3209 
Sat Jan 7 21:01:00 MSK 2006 3208 
Sat Jan 7 21:02:00 MSK 2006 3207 
Sat Jan 7 21:03:00 MSK 2006 3205 
Sat Jan 7 21:04:00 MSK 2006 3206 
Sat Jan 7 21:05:00 MSK 2006 3206 
Sat Jan 7 21:06:00 MSK 2006 3204 
Sat Jan 7 21:07:00 MSK 2006 3203 
Sat Jan 7 21:08:00 MSK 2006 3204 
Sat Jan 7 21:09:00 MSK 2006 3202 
Sat Jan 7 21:10:00 MSK 2006 3202 
Sat Jan 7 21:11:00 MSK 2006 3204 
Sat Jan 7 21:12:00 MSK 2006 3203 
Sat Jan 7 21:13:00 MSK 2006 3203 
Sat Jan 7 21:14:00 MSK 2006 3203 
Sat Jan 7 21:15:00 MSK 2006 3202 
Sat Jan 7 21:16:00 MSK 2006 3202 
Sat Jan 7 21:17:00 MSK 2006 3202 
Sat Jan 7 21:18:00 MSK 2006 3201 
Sat Jan 7 21:19:00 MSK 2006 3201 
Sat Jan 7 21:20:00 MSK 2006 3201 
Sat Jan 7 21:21:00 MSK 2006 3202 
Sat Jan 7 21:22:00 MSK 2006 3201 
Sat Jan 7 21:23:00 MSK 2006 3200 
Sat Jan 7 21:24:00 MSK 2006 3203 
Sat Jan 7 21:25:00 MSK 2006 3202 
Sat Jan 7 21:26:00 MSK 2006 3201 
Sat Jan 7 21:27:00 MSK 2006 3203 
Sat Jan 7 21:28:00 MSK 2006 3203 
Sat Jan 7 21:29:00 MSK 2006 3203 
Sat Jan 7 21:30:00 MSK 2006 3202 
Sat Jan 7 21:31:00 MSK 2006 3203 
Sat Jan 7 21:32:00 MSK 2006 3203 
Sat Jan 7 21:33:00 MSK 2006 3202 
Sat Jan 7 21:34:00 MSK 2006 3202 
Sat Jan 7 21:35:00 MSK 2006 3201 
Sat Jan 7 21:36:00 MSK 2006 3201 
Sat Jan 7 21:37:00 MSK 2006 3201 
Sat Jan 7 21:38:00 MSK 2006 3200 
Sat Jan 7 21:39:00 MSK 2006 3201 
Sat Jan 7 21:40:00 MSK 2006 3199 
Sat Jan 7 21:41:00 MSK 2006 3201 
Sat Jan 7 21:42:01 MSK 2006 3201 
Sat Jan 7 21:43:00 MSK 2006 3200 
Sat Jan 7 21:44:00 MSK 2006 3200 
Sat Jan 7 21:45:00 MSK 2006 3200 
Sat Jan 7 21:46:00 MSK 2006 3200 
Sat Jan 7 21:47:00 MSK 2006 3199 
Sat Jan 7 21:48:00 MSK 2006 3199 
Sat Jan 7 21:49:00 MSK 2006 3199 
Sat Jan 7 21:50:00 MSK 2006 3198.544 
Sat Jan 7 21:51:00 MSK 2006 3199 
Sat Jan 7 21:52:00 MSK 2006 3199 
Sat Jan 7 21:53:00 MSK 2006 3198 
Sat Jan 7 21:54:00 MSK 2006 3197 
Sat Jan 7 21:55:00 MSK 2006 3198 
Sat Jan 7 21:56:00 MSK 2006 3197 
Sat Jan 7 21:57:00 MSK 2006 3198 
Sat Jan 7 21:58:00 MSK 2006 3197 
Sat Jan 7 21:59:00 MSK 2006 3197 
Sat Jan 7 22:00:00 MSK 2006 3198 
Sat Jan 7 22:01:00 MSK 2006 3197 
Sat Jan 7 22:02:00 MSK 2006 3197 
Sat Jan 7 22:03:00 MSK 2006 3197 
Sat Jan 7 22:04:00 MSK 2006 3196 
Sat Jan 7 22:05:00 MSK 2006 3196 
Sat Jan 7 22:06:00 MSK 2006 3195 
Sat Jan 7 22:07:00 MSK 2006 3195.62 
Sat Jan 7 22:08:00 MSK 2006 3196 
Sat Jan 7 22:09:00 MSK 2006 3196 
Sat Jan 7 22:10:00 MSK 2006 3196 
Sat Jan 7 22:11:00 MSK 2006 3194 
Sat Jan 7 22:12:00 MSK 2006 3195 
Sat Jan 7 22:13:00 MSK 2006 3196 
Sat Jan 7 22:14:00 MSK 2006 3195 
Sat Jan 7 22:14:00 MSK 2006 3195 

(+-2 megabyte fluctuations here is a result of rounding)

As you can see, the memory, which was visible from OS's point of view, decreased
from 3224 to 3195 Megabytes in less than 2 hours, according to 'top' utility's output.
And this process will continue to the point, where swapping begins.
Server during this monitoring was running MySql 4.1.15 + Apache 1.3 under heavy
load.
I'd like to mention one more time: _total_ memory decreases (not free memory).
The problem seems very strange for me.

How to get rid of such memory loss? And what is the reason of such
memory loss?




More information about the freebsd-hackers mailing list