HEADS UP: DDB output capture, scripting, and textdumps imported
Robert Watson
rwatson at FreeBSD.org
Wed Dec 26 03:54:18 PST 2007
Dear all:
Per the prior e-mail thread, I've added several new facilities to DDB(4):
- DDB output capture, in which the input and output to DDB can be captured
selectively to a buffer for later inspection using a sysctl (after
continue), or in a textdump or kernel dump.
- DDB scripting, which allows simple scripts to be defined and executed,
either by hand, or as a result of a DDB event such as a panic, break key, or
watchdog firing.
- Kernel textdumps, which can store DDB output, the kernel message buffer,
kernel configuration, and other debugging information in a compact form in a
dump partition, providing an alternative to full memory dumps where only DDB
output is required.
I've included the key commit messages below, and have updated ddb(4) and
textdump(4) to reflect these new facilities.
One idea that has been bounced around a bit is adding an automatic kernel dump
reporting facility that could be enabled by a system administrator to submit
textdumps to a specific e-mail address for inspection. I find that most
textdumps I generate, assuming I include the kernel message buffer and a
reasonable set of information-gathering commands, are around 90k-100k, so
quite manageable and portable compared to regular dumps, and not dependent on
having an exactly synchronized set of kernel sources and debugging symbols.
I've not implemented this, but it is (as they say) a small matter of shell
scripting if someone is interested in giving this a spin.
Adding new information sources to textdumps is relatively easy (and can be
made easier), should there be other information that people want to capture,
such as $FreeBSD$ version tags, etc.
Many thanks to the various people who've given me feedback on this project,
including Antoine Brodin, Peter Wemm, Wojciech Koszek, Nikolay Pavlov, and
Michael Bushkov.
Robert N M Watson
Computer Laboratory
University of Cambridge
---------- Forwarded message ----------
Date: Tue, 25 Dec 2007 23:06:51 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/sys/conf files src/sys/ddb db_capture.c
db_command.c db_input.c db_main.c db_output.c ddb.h
rwatson 2007-12-25 23:06:51 UTC
FreeBSD src repository
Modified files:
sys/conf files
sys/ddb db_command.c db_input.c db_main.c
db_output.c ddb.h
Added files:
sys/ddb db_capture.c
Log:
Add a new DDB(4) facility, output capture. Input and output from DDB may be
captured to a memory buffer for later inspection using sysctl(8), or in the
future, to a textdump.
A new DDB command, "capture", is added, which accepts arguments "on", "off",
"reset", and "status".
A new DDB sysctl tree, debug.ddb.capture, is added, which can be used to
resize the capture buffer and extract buffer contents.
MFC after: 3 months
Revision Changes Path
1.1258 +1 -0 src/sys/conf/files
1.1 +301 -0 src/sys/ddb/db_capture.c (new)
1.74 +1 -0 src/sys/ddb/db_command.c
1.37 +1 -0 src/sys/ddb/db_input.c
1.6 +3 -0 src/sys/ddb/db_main.c
1.38 +8 -0 src/sys/ddb/db_output.c
1.44 +15 -0 src/sys/ddb/ddb.h
---------- Forwarded message ----------
Date: Tue, 25 Dec 2007 23:25:04 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/share/man/man4 ddb.4
rwatson 2007-12-25 23:25:04 UTC
FreeBSD src repository
Modified files:
share/man/man4 ddb.4
Log:
Document DDB capture facility.
MFC after: 3 months
Revision Changes Path
1.43 +34 -1 src/share/man/man4/ddb.4
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 09:33:19 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/sys/conf files src/sys/ddb db_command.c
db_command.h db_lex.c db_lex.h db_main.c db_script.c ddb.h
rwatson 2007-12-26 09:33:19 UTC
FreeBSD src repository
Modified files:
sys/conf files
sys/ddb db_command.c db_command.h db_lex.c
db_lex.h db_main.c ddb.h
Added files:
sys/ddb db_script.c
Log:
Add a simple scripting facility to DDB(4), allowing the user to
define a set of named scripts. Each script consists of a list of DDB
commands separated by ";"s that will be executed verbatim. No higher
level language constructs, such as branching, are provided for:
scripts are executed by sequentially injecting commands into the DDB
input buffer.
Four new commands are present in DDB: "run" to run a specific script,
"script" to define or print a script, "scripts" to list currently
defined scripts, and "unscript" to delete a script, modeled on shell
alias commands. Scripts may also be manipulated using sysctls in the
debug.ddb.scripting MIB space, although users will prefer to use the
soon-to-be-added ddb(8) tool for usability reasons.
Scripts with certain names are automatically executed on various DDB
events, such as entering the debugger via a panic, a witness error,
watchdog, breakpoint, sysctl, serial break, etc, allowing customized
handling.
MFC after: 3 months
Revision Changes Path
1.1259 +1 -0 src/sys/conf/files
1.75 +31 -5 src/sys/ddb/db_command.c
1.14 +1 -0 src/sys/ddb/db_command.h
1.23 +30 -1 src/sys/ddb/db_lex.c
1.15 +7 -4 src/sys/ddb/db_lex.h
1.7 +3 -0 src/sys/ddb/db_main.c
1.1 +564 -0 src/sys/ddb/db_script.c (new)
1.45 +29 -0 src/sys/ddb/ddb.h
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 09:38:22 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/sbin Makefile src/sbin/ddb Makefile ddb.8 ddb.c
ddb.h ddb_script.c
rwatson 2007-12-26 09:38:22 UTC
FreeBSD src repository
Modified files:
sbin Makefile
Added files:
sbin/ddb Makefile ddb.8 ddb.c ddb.h ddb_script.c
Log:
Add command-line tool ddb(8), which allows DDB(4) scripts to be
managed from userspace. It is largely a wrapper for sysctl()
calls, but because the sysctls for adding and removing scripts
are awkward to use directly, this provides an easier-to-use
interface.
MFC after: 3 months
Revision Changes Path
1.169 +1 -0 src/sbin/Makefile
1.1 +8 -0 src/sbin/ddb/Makefile (new)
1.1 +106 -0 src/sbin/ddb/ddb.8 (new)
1.1 +67 -0 src/sbin/ddb/ddb.c (new)
1.1 +37 -0 src/sbin/ddb/ddb.h (new)
1.1 +160 -0 src/sbin/ddb/ddb_script.c (new)
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 09:51:38 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/share/man/man4 ddb.4
rwatson 2007-12-26 09:51:38 UTC
FreeBSD src repository
Modified files:
share/man/man4 ddb.4
Log:
Add SCRIPTING section to describe new DDB scripting facilities.
Update copyright.
Revision Changes Path
1.45 +147 -2 src/share/man/man4/ddb.4
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 11:32:33 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/sys/conf files src/sys/ddb db_capture.c
db_command.c db_textdump.c ddb.h src/sys/kern kern_shutdown.c
rwatson 2007-12-26 11:32:33 UTC
FreeBSD src repository
Modified files:
sys/conf files
sys/ddb db_capture.c db_command.c ddb.h
sys/kern kern_shutdown.c
Added files:
sys/ddb db_textdump.c
Log:
Add textdump(4) facility, which provides an alternative form of kernel
dump using mechanically generated/extracted debugging output rather than
a simple memory dump. Current sources of debugging output are:
- DDB output capture buffer, if there is captured output to save
- Kernel message buffer
- Kernel configuration, if included in kernel
- Kernel version string
- Panic message
Textdumps are stored in swap/dump partitions as with regular dumps, but
are laid out as ustar files in order to allow multiple parts to be stored
as a stream of sequentially written blocks. Blocks are written out in
reverse order, as the size of a textdump isn't known a priori. As with
regular dumps, they will be extracted using savecore(8).
One new DDB(4) command is added, "textdump", which accepts "set",
"unset", and "status" arguments. By default, normal kernel dumps are
generated unless "textdump set" is run in order to schedule a textdump.
It can be canceled using "textdump unset" to restore generation of a
normal kernel dump.
Several sysctls exist to configure aspects of textdumps;
debug.ddb.textdump.pending can be set to check whether a textdump is
pending, or set/unset in order to control whether the next kernel dump
will be a textdump from userspace.
While textdumps don't have to be generated as a result of a DDB script
run automatically as part of a kernel panic, this is a particular useful
way to use them, as instead of generating a complete memory dump, a
simple transcript of an automated DDB session can be captured using the
DDB output capture and textdump facilities. This can be used to
generate quite brief kernel bug reports rich in debugging information
but not dependent on kernel symbol tables or precisely synchronized
source code. Most textdumps I generate are less than 100k including
the full message buffer. Using textdumps with an interactive debugging
session is also useful, with capture being enabled/disabled in order to
record some but not all of the DDB session.
MFC after: 3 months
Revision Changes Path
1.1260 +1 -0 src/sys/conf/files
1.2 +55 -3 src/sys/ddb/db_capture.c
1.76 +1 -0 src/sys/ddb/db_command.c
1.1 +555 -0 src/sys/ddb/db_textdump.c (new)
1.46 +23 -0 src/sys/ddb/ddb.h
1.187 +9 -1 src/sys/kern/kern_shutdown.c
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 11:35:07 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/share/man/man4 Makefile ddb.4 textdump.4
rwatson 2007-12-26 11:35:07 UTC
FreeBSD src repository
Modified files:
share/man/man4 Makefile ddb.4
Added files:
share/man/man4 textdump.4
Log:
Add textdump(4) man page to describe the textdump facility and provide
some stock formulas for use.
Update ddb(4) to reference the textdump(4) page, list the textdump
commands, and suggest using them with scripts and output capture.
Update HISTORY section.
Hook up textdump(4) to build.
MFC after: 3 months
Revision Changes Path
1.401 +1 -0 src/share/man/man4/Makefile
1.46 +31 -0 src/share/man/man4/ddb.4
1.1 +162 -0 src/share/man/man4/textdump.4 (new)
---------- Forwarded message ----------
Date: Wed, 26 Dec 2007 11:42:10 +0000 (UTC)
From: Robert Watson <rwatson at FreeBSD.org>
To: src-committers at FreeBSD.org, cvs-src at FreeBSD.org, cvs-all at FreeBSD.org
Subject: cvs commit: src/sbin/savecore savecore.8 savecore.c
rwatson 2007-12-26 11:42:10 UTC
FreeBSD src repository
Modified files:
sbin/savecore savecore.8 savecore.c
Log:
Teach savecore(8) how to extract textdump(4) dumps.
Update savecore(8) man page to reflect textdump additions.
MFC after: 3 months
Revision Changes Path
1.26 +11 -0 src/sbin/savecore/savecore.8
1.79 +160 -79 src/sbin/savecore/savecore.c
More information about the freebsd-current
mailing list