Thursday, April 1, 2010

Linux Crash debug tips - I have a partial or "incomplete" vmcore, what can I find out ?

In some circumstances when attempting to debug a vmcore from Linux, you may have only been able to get part of the vmcore, either due a technical issue or the machine being forcebly rebooted before completion.

When loading the vmcore in crash you may find something similar to the message below:

WARNING: yourvmcorefilename: may be truncated or incomplete
         PT_LOAD p_offset: 3358304032
                 p_filesz: 13690204160
           bytes required: 17048508192
            dumpfile size: 5412478976

crash: read error: kernel virtual address: ffff81042ff0eac0  type: "cpu_pda entry"
crash: read error: kernel virtual address: ffff810230005458  type: "pglist node_id"

A workaround to get some data in this case is to use a recent crash and use the --minimal parameter.

crash --minimal vmcorefilehere vmlinux

 --snip --

NOTE: minimal mode commands: log, dis, rd, sym, eval and exit

This will present you with a limited set of commands that you can use that will allow you to get basic data from the vmcore.

It is important to note that backtraces are not entirely accurate and that when you inspect memory it may not be available.  Crash  usually throws an error when this is the case.

This is not an april fools joke.

No comments: