[SOLVED]: URGENT 810 Software RAID failed after power outage

[Beeker]

No I have 4 x HDD's with one hot spare so it should be b,c,d & e with f as a hot spare so may be either the power or the sata lead may have come out, I will check these when I get home and let you know

[Crumble]

oops didn't see your post.  so yeah that is a good start.  once you got d showing again, give these two commands a go. assuming it doesnt rebuild itself on its own.


mdadm --stop /dev/md1
mdadm --assemble --force /dev/md1 /dev/sd[b,c,d,e,f]  

[Beeker]

I got home and checked all the HDD's and found with one making a slow beeping noise which I can only assume is sdd1 so a couple of questions

Should I leave it plugged in and try to rebuild the RAID with the commands you gave me and hopefully the spare drive will now form part of the RAID though I think the spare drive is just that a "spare drive" and not a hot spare so the other choice would be to unplug sdd1 and plug that into sdf and then reassemble the RAID with your commands and hopefully the data will come back

I really appreciate all you help with trying to help me fix this thanks very much

Best regards
Beeker

[tschak909]

Once you're back up and running, we need to make a ticket so that we can improve the RAID UI in the web admin, to help deal with these issues.

-Thom

[Beeker]

Will do happy to provide any information or help

[tschak909]

please make a ticket @ http://svn.linuxmce.org/ ... thanks

-Thom

[Beeker]

Done Thom..........excuse any mistakes as its the first ticket I have ever created so I hope its correct 

Please let me know if I need to make any changes

Beeker

[Crumble]

Well, I do not recommend setting up the RAID again with out a spare.  Unless you back up the data.  Remember RAID 5 will survive one drive failure.  If more than that fails that it is it, donezo.  All that data is gone forever.  I recommend replacing the drive.  I know it sucks, but better safe than sorry.  If you do not want to heed this warning.  I would leave it as is and run those commands to see if it will rebuild.  I really stress being patient if you need another drive and can't get one right away though.

[Beeker]

Just so I fully understand could I clarify a couple of your suggestions as you point out I rather not lose all the data.

As the original RAID was set up with 4 x HDD's and the 5th one Linuxmce automatically marked as spare can I buy another 1.5TB HDD and replace the failed one in the raid and hopefully it will rebuild as from my understanding that you are saying if I replace the HDD I can't just have 4 hDD's I need it to have the spare as well and if that is correct once I replace the faulty HDD then it will it rebuild or will I have to use one of your commands that you sent.

I have a backup of some of the data though as its a RAID array I assume I can't take out each HDD and put it in my QNAP and copy it

Really appreciate all your help and sorry for the dumb questions I just need to recover the family photos or life won't be worth living I certainly have learned a lesson here about backing data up

Regards
Beeker

[Crumble]

RAID does not like changes being made.  Problems can arise in linux from just trying to rebuild the array with one disk missing.  If the drive is there and say linux sees a write error or read error it will mark that dirty and should start using the spare.  In your case it knows that f is part of the array.  If d just isn't there this could create a problem.  Even though it sees f and knows its a spare.  I don't know why honestly.  Just my experience.  Your best bet is to replace d, leave f where it is.  Then run the very first command you posted to verify it is rebuilding.  If it does not automagically rebuild you will need to run the two commands i posted to force it to rebuild.  That would be the smartest way of doing this.  Although i should mention, there is always a chance that something else could go very wrong.  RAID 5 is a piece of garbage.  I do not understand why it was so popular.  It puts the data on the disks like a tic tac toe game making it difficult to retrieve data.  There are tools that can do this though.  Knoppix has one.  What you do is make images of the disks.  Put the images on four more disks then have the tools try and rebuild the data.  In your case i don't think that will work.  Usually a four drive RAID 5 the fourth disk is the parity drive.  Where the three disk RAID 5 parity is usually broken up between all three drives along with the data.  So, on second thought backup probably isn't an option for you.  You could try, there is no harm in making images of the drives before you try and rebuild it.  At least that way, you can always send the images off for recovery if your significant other is making death threats.  Then just blame it on the company, they screwed up, act really mad.   

[Beeker]

Thanks I will purchase a new Seagate 1.5TB HDD and see what it does and let you know............its going to take a couple of days to organise a new HDD

Thanks again for your patience and assistance with this issue........just confirming I should first try su mdadm -D /dev/md1 then if that does nothing then use the two cmd's that you suggested 

[Crumble]

Yes, that is exactly right.  Good Luck Beeker. 

[Beeker]

Thanks I think I am going to need it 

[Crumble]

how did you make out beeker?

[Beeker]

Hi Crumble,
I finally found a used Seagate 1.5TB on eBay for the right price of $61 and its on its way so I should have it next week so I am hoping that when I plug it in it will automatically rebuild the software RAID then I can copy all the content over to my new QNAP NAS that is ready to go with RAID6 and 19TB

I will have everything crossed next week when I put the replacement HDD in, so wish me luck and I will definitely let you know how it goes as you have been a fantastic help and if it all goes according to plan than I wont end up being killed if I can recover the wedding photos and all the other data 

Thanks again

Kind regards
Beeker