DEBUG - analysing core dumps

[Damien Fleuriot]

Hello list,



We've got these boxes at work running FreeBSD 8.1-STABLE amd64 and
serving as firewalls and openvpn gateways.

We use CARP interfaces to provide an active-passive fault tolerant system.


Today, we received a nagios alert from the master box saying it's
rsyslogd process had crashed.

I logged on to it and tried to relaunch it, to no avail:
pid 2303 (rsyslogd), uid 0: exited on signal 11 (core dumped)




I would like advice on how to debug the output from the core dump.

This is what I get from gdb:

# gdb
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd".
(gdb) core rsyslogd.core
Core was generated by `rsyslogd'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000004258ec in ?? ()




Sadly, getting a backtrace with "bt" gives me more lines with "??",
which is totally not helpful:
[SNIP]
#13 0x00007fffff1f9d70 in ?? ()
#14 0x0000000000000000 in ?? ()
#15 0x6f70732f7261762f in ?? ()
#16 0x6c737973722f6c6f in ?? ()
#17 0x5f6e70766f2f676f in ?? ()
#18 0x746174732e676f6c in ?? ()
#19 0x0000000000000065 in ?? ()
#20 0x0000000000000000 in ?? ()
[SNIP]

I am not sure what steps I should follow to get more information ?



Also, I believe that often, core dumps with signal 11 = RAM problems and
I would like a confirmation here.

I am concerned because rsyslogd is the only process that crashes in this
way, even after I rebooted the firewall.



Thanks for your input :)


--
dfl