conf/118255: savecore never finding kernel core dumps (rcorder problem)

[Jeremy Chadwick]

On Sat, Dec 01, 2007 at 10:36:08PM +0300, Mike Makonnen wrote:
> On Sat, 2007-12-01 at 11:10 +0000, Jeremy Chadwick wrote:
> >  This is great information; thanks for providing it! I found it quite
> >  educational/informational.  The documentation Antony provided seems to
> >  indicate two things:
> >  
> >  1) That savecore(8) should really be run before swapon(8) -- I don't see
> >  any indication that swap needs to be made available prior to mounting
> >  filesystems, which is what Doug B. was stating was a necessity.
> 
> The dependency if I recall correctly is: savecore needs file systems
> mounted, which need to be fsck(8)ed, which may need swap depending on
> the size of the file systems. The real problem is fsck(8). It can be
> a real memory hog in some situations (in which case it will need swap
> to function properly). If at all possible I would recommend increasing
> the size of the swap partition as a work-around.

Ah, this explains the need.  Hmm, definitely a chicken-and-egg problem.

> >  2) That even regardless of Item #1, savecore(8) should be working
> >  (assuming that kernel dumps are still written from the end of the swap
> >  device to the front (e.g. backwards)), and that swapon(8) shouldn't be
> >  stomping on kernel dumps.
> 
> Yes, if the size of the swap partition is big enough that the two
> don't collide. Apparently, in this case the swap space probably isn't
> large enough.

The swap partition on the system in question is 8GB:

$ swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/ad0s1b       8388608        0  8388608     0%

The machine itself contains 3GB of RAM.

$ dmesg | grep memory
real memory  = 3211657216 (3062 MB)
avail memory = 3141939200 (2996 MB)

I've always been of the "make swap at least 2x larger than the amount of
memory you have / plan to expand to" mentality, so I don't think swap
size is the problem in this case.

However, I'm not sure how much swap was being used at the time either of
the panics occured.  For sake of argument, this machine has rarely used
swap throughout the years (and it's our production web server).  I
monitor memory utilisation using symux (from ports), but sadly the swap
used/free byte counters in symux appear to be incorrect/false, so I
exclude them from monitoring.  :-(

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |