[SOLVED]: URGENT 810 Software RAID failed after power outage

[Crumble]

oh, maybe it was e.  can you run fdisk -l again and post

[Beeker]

Here you go after formatting sdd1 with ext3

dcerouter_1024641:/home/bruce# fdisk -l

Disk /dev/sda: 750.1 GB, 750156374016 bytes
255 heads, 63 sectors/track, 91201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x39686ed2

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1       90446   726507463+  83  Linux
/dev/sda2           90447       91201     6064537+   5  Extended
/dev/sda5           90447       91201     6064506   82  Linux swap / Solaris

Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0006a5c8

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1      182401  1465136001   83  Linux

Disk /dev/sdc: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xb217d64b

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1      182401  1465136001   83  Linux

Disk /dev/sdd: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xc8efaf55

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x43b7a284

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1      182401  1465136001   83  Linux

Disk /dev/sdf: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2848762e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1      182401  1465134080   83  Linux
dcerouter_1024641:/home/bruce#

[Crumble]

if it was the replacement drive it probably was e. 

[Crumble]

woohoo!  ok safe, didn't mean to scare you Beeker would be a shame to get this far and then format your data off LOL.

[Beeker]

Phew sdd1 is definitely the replacement HDD that is currently showing no valid partition

[Crumble]

ok, looks like you got f working again.  this is probably why the drive assignment changed and made me pee myself a little when you formatted d.  good stuff.  ok lets work on the superblocks.  gimme a couple to get the right strategy going.  working at the moment too.  

[Beeker]

Thanks much appreciated 

[Crumble]

Hey Beeker, sorry I haven't replied in a while.  I have been very busy at work and had no internet at the home.  If you still need to fix that RAID follow these instructions on this link for /dev/sdb.

first run this command just to be safe

mdadm --stop /dev/md1

then follow this
http://linuxexpresso.wordpress.com/2010/03/31/repair-a-broken-ext4-superblock-in-ubuntu/

your using ext3 i think?
Once you have replaced the bad superblock with a backup run


mdadm --assemble --force /dev/md1 /dev/sd[b,c,d,e,f] 


let me know if you have any problems
if you do get the RAID up and running again you will need to run fsck
but NOT until it is rebuilt and ready to use/online

[Crumble]

On second thought don't run fsck.  Backup your data once you get it running and then run fsck if you want to use that RAID.  Sometimes fsck will throw stuff in the lost and found folder.  Better to keep your organization the way it is for photos i am assuming.

[Beeker]

Thanks Crumble,
Just in the car stuck in traffic and will be home
in about 40mins so I will let you know how it
goes and thanks again:)

[Beeker]

Hey Crumble,
Reading the wordpress documentation it says to run

sudo fsck.ext3 -v /dev/sdd

Is it ok to run this cmd ?

Regards
Beeker

[Crumble]

you want    sudo fsck.ext3 -v /dev/sdb

do everything to /dev/sdb

If i am correct that is where the bad superblock was

make sure to run mdadm --stop /dev/md1  before anything

[Beeker]

I have stopped it already and I am sure it was sdd as that was the HDD I replaced so I assume it wont hurt if it the wrong HDD and if it comes back ok then I will check sdb

[Crumble]

sdd should be blank from the formatting

dcerouter_1024641:/home/bruce# mdadm --stop /dev/md1
mdadm: stopped /dev/md1
dcerouter_1024641:/home/bruce# mdadm --assemble --force /dev/md1 /dev/sd[b,c,d,e,f]
mdadm: no recogniseable superblock on /dev/sdb
mdadm: /dev/sdb has no superblock - assembly aborted
dcerouter_1024641:/home/bruce#

[Beeker]

I started checking sdd and currently it says this

dcerouter_1024641:/home/bruce# fsck.ext3 -v /dev/sdd
e2fsck 1.41.3 (12-Oct-2008)
/dev/sdd has gone 1270 days without being checked, check forced.
Pass 1: Checking inodes, blocks, and sizes

Should I cancel it and check sdb