Trouble installing with drives over 2TB?

[nosebreaker]

After numerous problems with software RAID5 and failing, I gave up and bought a 3Ware 9650SE (4-port version).  I chose this because I have the 8-port version in another system with Ubuntu 6.10 and it worked perfectly.

I had trouble getting LinuxMCE to install onto 3x 1.5TB drives in a raid5 setup before, such that I had to use the cd's and a manual install.  Now it appears that the LinuxMCE dvd can't install onto a drive this large (3TB) and it fails creating the partitions.  I'll install kubuntu 7.10 again if necessary, but I was hoping there might be a boot flag or something I could pass to it.  Any ideas?

[colinjones]

Well it is probably because you aren't using GPT partitions. The DVD uses the older style, partitions, same as Windows and most other OS's. But those partition tables can only describe a partition of 2TB maximum. So if the DVD is trying to define the entire drive as the / (root) partition, which is typically what happens (certainly my 500GB system drive has / as almost the entire disk, less a small extended partition containing 2 small logical partitions) then it will be trying to define a partition of nearly 3TB which is too large for these partition tables. Only a GPT partition could handle this.

My first recommendation would be not to make such a large disk your LMCE core system drive. I would use a much smaller disk (realistically you only need 40+GB) for your system, and use the big volume for your media, ideally on a separate system over the internal network - this has a range of advantages. Don't get hung up on RAIDing your system drive, it definitely is not worth it!

[nosebreaker]

It is worth it to RAID the system, for this reason alone:
If there is a hardware failure, I won't have time to repair the system.  It will be broken probably permanently.  I plan to use VoIP as well, so this will mean the phone system is broken as well.

I have tried making a 500MB /boot, 9GB swap and 100GB / for root and it still won't install.  It gives the grub-install error.  I have tried so many things, at this point I don't know what I'm going to end up doing (especially since I bought a raid card now).

[nosilla99]

@nosebreaker

I am not familiar with that Raid card, but most will allow you to create separate logical disks.  If you can do this from the raid card bios create 2 separate logical disks.

I would recommend atleast 40GB for Disk 1 and use the rest for disk 2.  Then let LMCE partition the first hard disk but tell it not to use the 2nd disk (you can partition and mount this manually after installation)

NOS

[nosebreaker]

I've never noticed volume creation in the RAID bios, typically that's handled by the OS.  I can create the different volumes, but it won't install grub.  I'll see if I can create multiple drives in the RAID bios.

[nosilla99]

@nosebreaker

The raid bios allows you to create logical disks whereas the OS deals with partitions.  Basically after configuring the raid bios, your OS will think you have mutliple physical hard disks present.

If you are using the DVD install it also creates a recovery partition of about 8GB and copies the DVD contents to that partition hence my recommendation of about 40GB for disk1

NOS

[colinjones]

Please note, you cannot use the DVD for installation unless you create RAID volumes/containers in the RAID BIOS like nos describes (read my other post to you in the thread on 32v64bit).

Also, my advice not to RAID the system was because, if the RAID itself breaks (in my experience, more common than HDD failing!) you will be unable to boot at all - if you cannot fix that broken RAID set, then you loose everything. If a HDD fails, more often than not you can easily recover the data... your choice, but remember reinstalling is easy and you will find that you probably end up doing this quite a few times for various reasons - the system is almost "stateless" bar a little config, media is king!

[nosilla99]

Colin,

I agree with you 100% if you are referring to software raid, but h/w raid should be totally reliable and has obviously demonstrated its stability within buisness environments as very few people would opt for a non raid soultion.

I am lucky enough to have picked up an ATA Raid card with 8 ports from a server which was being decommisioned and have used this with no failures since the very early days of pluto.

NOS

[Zaerc]

Other alternatives would be the CD installer, or creating the patitions in advance as decribed in: http://wiki.linuxmce.org/index.php/DVD_Install_And_Custom_Partitions

[colinjones]

Colin,

I agree with you 100% if you are referring to software raid, but h/w raid should be totally reliable and has obviously demonstrated its stability within buisness environments as very few people would opt for a non raid soultion.

I am lucky enough to have picked up an ATA Raid card with 8 ports from a server which was being decommisioned and have used this with no failures since the very early days of pluto.

NOS

It is in fact hardware RAID's in enterprise environments I am talking about! People set far too much stock in these. Until recently I was the Regional Operations Manager of a multinational for the Asia Pacific region, covering around 6000 seats, running over 350 servers in everything from DELL PERC RAIDs, IBM ones and EMC DMX/Clarrion SANs, DAS, NAS, iSCSI and Centera's. I was ultimately responsible for all failures in the region. Believe me, they fail (broken RAID sets, split hypervolumes, failed RAID backplane hardware, spontaneous degrading, and inconsistencies between the RAID container metadata and RAID BIOS metadata during reboots that cause a RAID config to be completely lost - if you don't catch this one before any new data is written to the drive, it is almost unrecoverable!) far more often than people think, or than they should!

I'm certainly not advocating businesses not use RAID (HW), I'm just saying with the issues above combined with endemic oversight of RAID health monitoring with IT engineers, often RAIDs merely increase availability/MTBF rather than actually provide real "redundancy"! (its amazing how often an engineer came to me with a credulous look on his face and explained that 2 disks failed at exactly the same time - to which I always replied "bullshit!" show me your RAID health monitoring for that set for the last 3 months - of course they couldn't!) Further, I'm certainly not making a case for HW vs SW or the other way around for home systems, but I will say that some of the only systems I have worked with that actually provide real "redundancy" were software systems such as those employed on the EMC Centera, which are truly able to "self heal" rather than dumb availability from RAID - but those are insanely expensive (very clever tho!)

btw, in a business environment you should never use RAID5 anyway, it is most definitely the "ugly sister" of the RAID family, very slow and unreliable for the sake of saving a few bucks. RAID 10 is usually the way to go, and if you can hyper/meta-volume them as well, all the better.

[nosebreaker]

The case I bought can really only fit 3 drives, so I am stuck with raid5.  I have a program one of my contractors wrote that can read the raw data stripes and recombine them if the controller ever fails anyway.

The controller has a "boot volume" option that lets you create a separate volume, and it has an option to split every X bytes automatically after that.  So I made a 200gb partition, put vista in half of it and kubuntu in the other half, and it worked the first try.  I then used the CD's to install LMCE 0710, and aside from having some trouble with 1080 plasma scaling text incredibly small (like 4 pixels for a character, but it takes up many actual pixels on the screen).

[colinjones]

The text size is a common problem when X is unable to get the correct DPI for your screen. Easy to fix and plenty of how tos on the wiki and forums for this.

You will still have the partition problem for partitions over 2TB unless you use GPT partitions.

Have fun!

[Zaerc]

There is nothing wrong with raid5, but plenty with people who leave things unchecked for months on end and don't backup their valuable data.  Raid is no alternative for making proper backups, anybody with half a brain knows this.

[colinjones]

There is nothing wrong with raid5, but plenty with people who leave things unchecked for months on end and don't backup their valuable data.  Raid is no alternative for making proper backups, anybody with half a brain knows this.

Absolutely agreed on the backup! Any suggestions on a cost effective hardware backup solution? I have nearly a TB of media and no easy way of backing up - the only options I have come up with so far is an external HDD (not very scalable), LTO4 (very expensive), and some kind of removable disk/cartridge solution (can't remember what it was called, but the disks were only about 200GB and cost like $100+ each and the drive was around $400, so not really cost effective)....

[Zaerc]

Well I simply don't consider ordinary media files to be valuable enough, and rumour has it that most media nowadays comes on these shiny plastic discs that you can store and keep as a backup. 

I keep my important business-data backed up on an USB stick as well as an external server in a colo somewhere, to be honest I don't have that much and I'm very selective.