[GRLUG] RAID recovery

[Greg Folkert]

On Wed, 2014-03-05 at 23:08 -0600, L. V. Lammert wrote:
> Has a power failure at a site today and the UPS batteries ran out. Had
> some strange problems on restart, and I realize two disks are missing
> from the RAID6 volume:
> 
>     Number   Major   Minor   RaidDevice State
>        0       8        2        0      active sync   /dev/sda2
>        1       8       34        1      active sync   /dev/sdc2
>        2       0        0        2      removed
>        3       0        0        3      removed
> 
> 
> The weird part is that all four disks show active (sda2, sdb2, sdc2,
> sdd2) in fdisk, but when I try to add one of the missing disks to the
> array, it shows all three busy:
> 
> mdadm --assemble --run --force --update=resync /dev/md0 /dev/sda2
> /dev/sdc2 /dev/sdb2
> 
>           State : active, degraded
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>   Spare Devices : 0
> 
> Any pointers on the correct ray to resync the array?

You are trying to assemble an array that is currently broken. Never
work.

Assuming the drives missing are in good shape, and are properly
partitioned. That is before what you asked for and you've already said
they are "active", so I assume you've double checked that and verified
they are good to go. On that...

Do this first to see the status of /dev/md0 (and any other md raid
devices):

        cat /proc/mdstat

This will tell you the status of the Raid6 array... I prefer this as the
pretty tools just confuse things.

The once you've determined indeed truly things are as  you say...

First make sure the drives are indeed failed and removed from the array:

        mdadm --manage /dev/md0 --fail /dev/sdb2
        mdadm --manage /dev/md0 --remove /dev/sdb2
        mdadm --manage /dev/md0 --fail /dev/sdd2
        mdadm --manage /dev/md0 --remove /dev/sdd2
        
You have to re-add the drives back into the array, first.

        mdadm --manage /dev/md0 --add /dev/sdb2
        mdadm --manage /dev/md0 --add /dev/sdd2

Then you can "watch cat /proc/mdstat" until things finish.

If you want better/higher rebuild rates... Google "LINUX MD RAID REBUILD
SPEED" and pick and apropos one.
-- 
greg at gregfolkert.net
PGP key 1024D/B524687C 2003-08-05
Fingerprint: E1D3 E3D7 5850 957E FED0 2B3A ED66 6971 B524 687C
"Friendship multiplies the good of life and divides the evil."
    -- Baltasar Gracian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <http://shinobu.grlug.org/pipermail/grlug/attachments/20140306/15f8f395/attachment.pgp>