DISK OPERATION ERROR in AIX

Last month, we noted errpt showing an error:

[root@aix:/] errpt -a
IDENTIFIER TIMESTAMP  T C RESOURCE_NAME  DESCRIPTION
DCB47997   1122091816 T H hdisk29        DISK OPERATION ERROR
F31FFAC3   1122091816 I H hdisk29        PATH HAS RECOVERED
DE3B8540   1122091816 P H hdisk29        PATH HAS FAILED
DCB47997   1122075216 T H hdisk26        DISK OPERATION ERROR
F31FFAC3   1122075216 I H hdisk26        PATH HAS RECOVERED
DE3B8540   1122075216 P H hdisk26        PATH HAS FAILED
DCB47997   1122070516 T H hdisk31        DISK OPERATION ERROR
F31FFAC3   1122070516 I H hdisk31        PATH HAS RECOVERED
DE3B8540   1122070516 P H hdisk31        PATH HAS FAILED

A way to find what’s causing this error is using ‘lsmpio’ command to display detailed device statistics for the hdisks devices.
NOTE: lsmpio command was introduce in AIX 7.1 TL3 and 6.1 TL9.

[root@aix:/] lsmpio -Sdl hdisk29
Disk: hdisk29
    Path statistics since Fri Dec 09 12:56:32 CET 2016
    Path 0: (fscsi0:500a09829af63ecf,7000000000000)
        Path Selections:                              1282755
        Adapter Errors:                                     0
            Software:                        0
            Hardware:                        0
            Transport Dead:                  0
            Transport Busy:                  0
            Transport Fault:                 0
            No Device Response:              0
            Target Port ID Changed:          0
        Command Timeouts:                                   0
        Reservation Conflicts:                              0
        SCSI Queue Full:                                    0
        SCSI Busy:                                          0
        SCSI ACA Active:                                    0
        SCSI Task Aborted:                                  0
        SCSI Aborted Command:                               0
        SCSI Check Condition:                               0
            Medium Error:                    0
            Hardware Error:                  0
            Not Ready:                       0
            Other:                           0
        Last Error:                                       N/A
        Last Error Time:                                  N/A
        Path Failure Count:                                 0
            Due to Adapter Error:            0
            Due to I/O Error:                0
            Due to Health Check:             0
            Due to SCSI Sense:               0
            Due to Qualifier Bit:            0
            Due to Opening Error:            0
        Last Path Failure:                                N/A
        Last Path Failure Time:                           N/A
Disk: hdisk29
    Path statistics since Fri Dec 09 12:56:31 CET 2016
    Path 1: (fscsi1:500a09828af63ecf,7000000000000)
        Path Selections:                              1254441
        Adapter Errors:                                     3
            Software:                        0
            Hardware:                        0
            Transport Dead:                  0
            Transport Busy:                  0
            Transport Fault:                 0
            No Device Response:              0
            Target Port ID Changed:          0
        Command Timeouts:                                   3
        Reservation Conflicts:                              0
        SCSI Queue Full:                                    0
        SCSI Busy:                                          0
        SCSI ACA Active:                                    0
        SCSI Task Aborted:                                  0
        SCSI Aborted Command:                               0
        SCSI Check Condition:                               0
            Medium Error:                    0
            Hardware Error:                  0
            Not Ready:                       0
            Other:                           0
        Last Error:                                  SCSI Command Timeout
        Last Error Time:                             Mon Dec 12 09:12:08 CET 2016
        Path Failure Count:                                 0
            Due to Adapter Error:            0
            Due to I/O Error:                0
            Due to Health Check:             0
            Due to SCSI Sense:               0
            Due to Qualifier Bit:            0
            Due to Opening Error:            0
        Last Path Failure:                                N/A
        Last Path Failure Time:                           N/A

As you can see in the output, the cause of the issue is “SCSI Command Timeout” on FC adapter.
Please, check FC network (VIOS, NPIV, SAN zonning).

Advertisements

About igalvarez

More than 20 years experiences on IT industry. 20+ years in Unix experience : IBM-AIX, HMC/SDMC/IVM, SVC, Protectier, PureFlex Systems, VIOservers, IBM Bladecenters, IBM System Power, RedHat Linux, SuSE Linux, Debian/Ubuntu Linux. Solaris, SCO Unix, Tru64 Unix, Linux Virtualization: XEN, KVM, databases sb2, oracle, postgreSQL, Find more information here: http://en.gravatar.com/igalvarez
This entry was posted in AIX and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s