Does FreeBSD support sparse kernel crash dumps?

[Peter Wemm]

On Friday 07 April 2006 04:01 pm, Paul Marciano wrote:
> --- Peter Wemm <peter at wemm.org> wrote:
> > On Friday 07 April 2006 12:47 pm, Paul Marciano
> >
> > wrote:
> > > Hello.  I read a while back about someone working
> >
> > on
> >
> > > supporting sparse kernel crash dumps (dumping only
> >
> > the
> >
> > > active kernel pages to the dump device as opposed
> >
> > to
> >
> > > all physical memory - for machines where the phys
> >
> > mem
> >
> > > is greater than the dump dev space.)
> > >
> > > Does anyone know the status of that project?  Was
> >
> > it
> >
> > > committed, or are there plans to commit it?
> >
> > I have a working prototype as of last night.  There
> > should be something
> > committable in the next week or two.
> >
> > When I boot my 2GB machine and force a crash dump
> > from single user mode, the
> > fully debuggable vmcore file is in the 40-50MB
> > range.  A busy machine with
> > 12GB ram took about 150MB to dump.
> >
> > There are still some things to work out.  It was
> > written for the amd64 kernel,
> > but can be ported to i386.
> >
> > -Peter
>
> That's very timely news Peter.  Do you think your code
> is easily back-portable to 5.4?  Are the changes
> limited to dump_machdep.c or otherwise not dependent
> on a great deal of -current updates?

My WIP is relative to 6.x.  It will port forwards to current fairly easily, 
and will port to i386 trivially.  The code would be simpler on i386 because 
direct map data doesn't exist there.  But on the other hand, i386 has to deal 
with PAE mode and the dump code would be significantly impacted by PAE mode

Changes are mostly in sys/amd64/amd64/dump_machdep.c (a rewrite actually), an 
8 line addition to sys/vm/vm_page.c and some trivial patches to some other MD 
pmap files.  libkvm is affected as a complete rewrite of 
lib/libkvm/kvm_amd64.c.  (Porting libkvm changes to i386 will be easy and 
relatively immune to PAE kernel effects). 

> Do you compress the data stream at all (e.g. gzip)?

No, but it could be done in theory.. if you were willing to set aside some 
memory for the compression algorithm to use.  Or just do some sort of simple 
compression (run length encoding?)  The problem is that the dump code cannot 
allocate memory after the machine has crashed.  It has to be able  to run as 
isolated from the rest of the kernel as possible in order to give a true 
snapshot of the undisturbed state.

> I have a specific need on a CompactFlash based system.
>  I have a 256MB IDE mode card on a machine with 512MB
> physical memory.  I can probably commit 128MB of the
> card as a dumpdev but I can't go beyond that.

I truely do not know what to expect.  Without compression, check the 
difference between sysctl vm.kvm_free and vm.kvm_size to know how much an 
uncompressed minidump would take.  I don't recall if those are on 5.4 or not 
(the code for those sysctl's would backport from pmap.c really easily).

> Good luck with it.
>
> Regards,
> Paul.

-Peter