Scribd
Upload
171 views14 pages

05 Solaris

This document discusses crash dump analysis on the Solaris operating system. It provides an overview of Solaris, describes different types of system problems and the corresponding debugging tools. These include crashes producing core files or crash dumps analyzed using mdb, dbx, gdb and CAT. Reasons for crashes like bad traps or panics are explained. Methods for handling performance issues and hangs using tools like DTrace, mdb and command line utilities are also summarized.

Uploaded by

Yc Rdml
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
171 views14 pages

05 Solaris

This document discusses crash dump analysis on the Solaris operating system. It provides an overview of Solaris, describes different types of system problems and the corresponding debugging tools. These include crashes producing core files or crash dumps analyzed using mdb, dbx, gdb and CAT. Reasons for crashes like bad traps or panics are explained. Methods for handling performance issues and hangs using tools like DTrace, mdb and command line utilities are also summarized.

Uploaded by

Yc Rdml
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Crash Dump Analysis

Solaris

Jakub Jermář
Martin Děcký
Solaris overview
● UNIX derivative (System V Release 4)
● 32­bit and 64­bit
▪ IA­32, AMD64, SPARC (V7 ­ V9)
● Unofficial PowerPC port
▪ Solaris 8, 9, 10
▪ Solaris Nevada
● OpenSolaris
● Illumos, OpenIndiana
● Solaris 11 Express
Crash Dump Analysis – MFF UK – Solaris 2
System debugging
● Variety of problems → variety of tools
● Problems ● Tools
▪ Crashes ▪ mdb, kmdb, Solaris 
● Crash dumps CAT
● Core dumps ▪ dbx, gdb
▪ Bad behavior ▪ Debug kernels
● Performance ▪ DTrace
● Hangs
▪ Command line tools
● Oddities
Crash Dump Analysis – MFF UK – Solaris 3
Crashes
● Crashing application can produce a core file
▪ Configure using the coreadm utility
● Crashing kernel produces a crash dump
▪ configure using the dumpadm utility
● Both types contain the snapshot of the 
application/kernel, memory and other essential 
state information for later post­mortem 
debugging session

Crash Dump Analysis – MFF UK – Solaris 4
Crashes (2)
● If the application does not crash, we can still get 
a core file of the running process
▪ Use the gcore utility
● If the system does not crash, we can still get a 
crash dump of the (running) kernel
▪ Use the savecore utility

Crash Dump Analysis – MFF UK – Solaris 5
Crashes (3)
● Core file analysis tools
▪ mdb
● Assembly­level debugger
● Standard part of Solaris distributions
● http://opensolaris.org/os/community/mdb/
▪ dbx
● Source­level debugger similar to gdb
● Part of Sun Studio
● http://developers.sun.com/sunstudio/downloads/

Crash Dump Analysis – MFF UK – Solaris 6
Crashes (4)
● Core file analysis tools
▪ gdb
● Source­level debugger similar to dbx
● Standard part of Solaris distributions
– Need to install a package on OpenSolaris
● http://www.gnu.org/software/gdb/

Crash Dump Analysis – MFF UK – Solaris 7
Crashes (5)
● Crash dump analysis tools
▪ mdb
▪ Solaris CAT
● Solaris Crash Ananlysis Tool (a.k.a. S­CAT)
● “Heuristic” and automated analysis of crash dumps
● http://blogs.sun.com/solariscat/
▪ Debug kernels
● special support for debugging
– E.g. TRAPTRACE – tracing of all traps

Crash Dump Analysis – MFF UK – Solaris 8
Reasons for crash
● BAD TRAP (or other bad trap)
▪ registers from the time of the crash saved in the 
trap frame
● Failed assertions
● Panic states
▪ cmn_err() 
● XIR / NMI
● Voluntary crashes initiated by the user
Crash Dump Analysis – MFF UK – Solaris 9
Reasons for dumping core
● Bad memory access
▪ SIGSEG, SIGBUS
● Bad operation
▪ SIGFPE (Arithmetic Exception)
● Application abort()
▪ SIGABRT
● User request via gcore

Crash Dump Analysis – MFF UK – Solaris 10
Bad behavior
● Performance issues
▪ DTrace
● Dynamic tracing framework
● Can provide insight into what's going on in the software
● Standard part of recent Solaris distributions
● http://opensolaris.org/os/community/dtrace/

Crash Dump Analysis – MFF UK – Solaris 11
Bad behavior (2)
▪ Command line tools
● vmstat
– Monitoring memory load
● iostat
– Monitoring I/O load
● mpstat
– Monitoring CPU load
● ps, prstat, top
– Monitoring process activities

Crash Dump Analysis – MFF UK – Solaris 12
Bad behavior (3)
● Hangs
▪ “No or diminishing forward progress”
▪ mdb ­k
● mdb attached to the live system
▪ mdb ­K
● Single stepping using kmdb
▪ mbd, dbx, gdb
● Attach to a running process

Crash Dump Analysis – MFF UK – Solaris 13
Bad behavior (4)
▪ Command line tools
● pstack
– Display the user stack of a running process
● Oddities
▪ “Software does something else”
▪ DTrace
▪ mbd ­k, mdb ­K
▪ Command line utilities
● truss
Crash Dump Analysis – MFF UK – Solaris 14

You might also like