Tuesday, March 16, 2010

Linux Crash debug tips - Which process is on which CPU ?

I'm going to start miniblogging (which is larger than twitters microblogging) but not quite enough to be a blog.  For tips that I learn using the crash program.

Say you've had a soft lockup, the kernel has paniced either by your setting or a forced coredump, how can you tell which process is on which CPU.

# crash ./vmcore
>ps 
   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
>     0      0   0  ffffffff802e5ae0  RU   0.0       0      0  [swapper]
>     0      1   1  ffff81083006e100  RU   0.0       0      0  [swapper]
>     0      1   2  ffff81010eb26080  RU   0.0       0      0  [swapper]
>     0      1   3  ffff81083e11a080  RU   0.0       0      0  [swapper]
      0      1   4  ffff8104300787a0  RU   0.0       0      0  [swapper]
>     0      1   5  ffff810c3e1d87e0  RU   0.0       0      0  [swapper]
>     0      1   6  ffff81043e15a820  RU   0.0       0      0  [swapper]
>     0      1   7  ffff810c3e200860  RU   0.0       0      0  [swapper]
>     0      1   8  ffff81043e209860  RU   0.0       0      0  [swapper]
>     0      1   9  ffff810c3e2727e0  RU   0.0       0      0  [swapper]
>     0      1  10  ffff81043e21c7e0  RU   0.0       0      0  [swapper]
>     0      1  11  ffff810c3e275860  RU   0.0       0      0  [swapper]
>     0      1  12  ffff81043e2a8820  RU   0.0       0      0  [swapper]
>     0      1  13  ffff810c3e2e87e0  RU   0.0       0      0  [swapper]
>     0      1  14  ffff81043e2b17a0  RU   0.0       0      0  [swapper]
>     0      1  15  ffff810c3e2eb860  RU   0.0       0      0  [swapper]
      1      0  11  ffff81083006b040  UN   0.0   10344    684  init

The tasks that have a > to the left are the processes that are in the RUN state (that means on the CPU right now).

The CPU column aligned to the task is the surprisingly the CPU that the process was on (or was last on).   Too easy.