How do I analyze and debug core files on AIX.

Sometimes, applications stop working and crash for any reason. A core file appears in scene.

First, we’re going  to set a PATH for all cores files on the system  using syscorepath command.

In order to do this, We’re going to:

1- Create and mount a new filesystem /corefiles

1.1- Create the new filesytem:

# crfs -v jfs2 -g volgrp2 -m /home/pipe -A yes -a size=1G
File system created successfully.
1048340 kilobytes total disk space.
New File System size is 2097152

1.2- Mount the filesystem:

# mount /corefiles

2- Using syscorepath command, we’ll set a PATH where all these cores files will be dumped..

2.1- Set the directory for all cores files:

# syscorepath -p /corefiles

2.2- To check if the core’s PATH was correctly set, running:

# syscorepath -g /corefiles
# /corefiles

3- Analize the core file using dbx command.

Once you get all cores in /corefiles, we’re going to analyse them using dbx command. Follow these steps:

3.1- Get the cause of the core file by running check_core command. In the example ksh93 is the victim:

# /usr/lib/ras/check_core /corefiles/core |tail -1
ksh93

3.2- Debugging with dbx (Provides an environment to debug and run programs) command:

# dbx /usr/bin/ksh93 /corefiles/core
Type 'help' for help.
[using memory image in /corefiles/core]
reading symbolic information ...warning: no source compiled with -g

Segmentation fault in strlen at 0xd0105880
0xd0105880 (strlen)    89030000         lbz   r8,0x0(r3)

3.3- Once your are in dbx shell, type ‘where’

(dbx) where
strlen() at 0xd0105880
varsub() at 0x100475fc
copyto(??, ??) at 0x100494c8
sh_mactrim(??, ??) at 0x1004a5a0
word_trim(??, ??) at 0x1000e340
sh_exec(??, ??) at 0x1000ff88
sh_exec(??, ??) at 0x1001043c
sh_exec(??, ??) at 0x1000ff50
sh_exec(??, ??) at 0x10010470
sh_exec(??, ??) at 0x1000fdcc
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x100106d8
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fd94
b_dot_cmd(??, ??, ??) at 0x10066e20
sh_funct(??, ??, ??, ??, ??) at 0x1000e500
sh_exec(??, ??) at 0x1000ee88
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x10010370
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe98
exfile(??, ??) at 0x10000ed8
sh_main(??, ??, ??) at 0x100015dc
pmain.main(??, ??) at 0x10000370

Or you can use others dbx’s commands. Please see man dbx.

</pre>
(dbx) thread
(dbx) map
(dbx) func
(dbx) dump
(dbx) quit
<pre>

Just thanks if the post was helpful 🙂

Advertisements

About igalvarez

More than 20 years experiences on IT industry. 20+ years in Unix experience : IBM-AIX, HMC/SDMC/IVM, SVC, Protectier, PureFlex Systems, VIOservers, IBM Bladecenters, IBM System Power, RedHat Linux, SuSE Linux, Debian/Ubuntu Linux. Solaris, SCO Unix, Tru64 Unix, Linux Virtualization: XEN, KVM, databases sb2, oracle, postgreSQL, Find more information here: http://en.gravatar.com/igalvarez
This entry was posted in AIX, CORES, DBX, DEBUG and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s