Disk I/O errors on Adaptec ASR8805 raid controller

We have an Adaptec ASR8805 controller on one of the servers we manage. For various reasons we need to shrink a logical volume that is sitting on a RAID 6 logical device created and exposed by this controller, but we can’t because we’re getting seek errors:

Buffer I/O error on device dm-2, logical block 3330419721
sd 6:0:1:0: [sdb]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 6:0:1:0: [sdb]  Sense Key : Hardware Error [current] 
sd 6:0:1:0: [sdb]  Add. Sense: Internal target failure
sd 6:0:1:0: [sdb] CDB: Read(16): 88 00 00 00 00 06 34 11 69 00 00 00 01 00 00 00
end_request: critical target error, dev sdb, sector 26643360000
sd 6:0:1:0: [sdb]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE

From what the controller is reporting, the RAID 6 is healthy, and all the physical drives SMART information seems ok(ish).

It turns out, no background checking of the RAID 6 parity has been enabled, and that is probably the problem, as reported by this article.

To get a “quick” fix (it’s a 24T array), I started:

# arcconf task start 1 logicaldrive 1 verify_fix

when it’ll be finished, I’ll enable the background check with:

# arcconf consistencycheck 1 on

I really hope this saves time to some fellow admin out there :)

Adaptec Series 6 – 6G SAS/PCIe 2 (rev 01) [9005:028b] tools

Posting this in hope to save time to some fellow admins out there.

I had this on a server:

# lspci | grep -i adaptec
05:00.0 RAID bus controller: Adaptec Series 6 - 6G SAS/PCIe 2 (rev 01)
# lspci -n | grep '05:00'
05:00.0 0104: 9005:028b (rev 01)

Seems impossible to see the model number, not even lshw reports it. Luckily, the tools are the same for all the controllers, you can find them going to the Microsemi Adaptec Series 6 support page, clicking any controller, clicking the Storage Manager download link and then the Microsemi Adaptec ARCCONF Command Line Utility.

This is the link to the Microsemi Adaptec ARCCONF Command Line Utility Download Page at the moment of this writing.

Once installed, the tool will be in /usr/Arcconf, I created a symlink in /usr/bin

# arcconf getconfig 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
   Controller Status                        : Optimal
   Channel description                      : SAS/SATA
   Controller Model                         : Adaptec 6405E
...

There’s more information about how to use the tools on these pages: