Author Topic: Media Storage Idea  (Read 3424 times)

Sparkie

  • Guest
Media Storage Idea
« on: January 03, 2008, 05:52:42 am »
I haven't had a straight answer for how to setup Data Storage for a house with a few media directors.
I have 2 Terabytes of media and its growing... everything is scattered among many drives on various computers.

Here are the basic requirements...

I'd like it all to be at one source
Fail-safe solid reliability
Easy to setup and maintain
Easy to increase storage size
Cross platform (XP/Ubuntu/Vista/Mac) readable/writable
Accessible to steam over a network.
Fast performance
A price below $2000
*Has to include parts to build the Core Server*
How should i set this up?
Specs?

Please help!
« Last Edit: January 04, 2008, 03:28:00 am by Sparkie »

posde

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 2846
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Media Storage Idea
« Reply #1 on: January 03, 2008, 08:55:07 am »
Are you looking for a totally new computer, or just ideas for implementing storage within an existing system?

If it is the later one, I suggest a good quality 8-Port SATA-II RAID controller (about 450€), like the TEKRAM (ARECA) ARC-1220 or TEKRAM (ARECA) ARC-1120. Start with three 1TB drives (ea 350€), get a good SATA backplane (abut 100€) and you are set. With a little bit of luck you will end up below 1500€. I assume pricing in USD will be similar. You save some money by getting a 3WARE RAID controller, but those have some performance problems (or so I have heard) with bigger files.

BTW: I would not put all my files onto a single partition, because of problems with large file system sizes.

rgds
Oliver

posterberg

  • Veteran
  • ***
  • Posts: 82
    • View Profile
Re: Media Storage Idea
« Reply #2 on: January 03, 2008, 01:57:59 pm »
I've been using LVM and one partition for about 5 year on a continuously growing filesystem  (currently 2 TB). But I am going to split it up now, I agree with Oliver - there are problems.

I am thinking about using simple 2 disk NAS cases and have 1 TB in each case.

posde

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 2846
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Media Storage Idea
« Reply #3 on: January 03, 2008, 02:01:24 pm »
After playing with some network storage together with LMCE I will not go that route. Maybe it should work, but my experience has been bad when using cifs or nfs mounted drives with LMCE. So, my installation(s) will not use NAS, but local storage. Maybe iSCSI is another route.

rgds
Oliver

Misogynist

  • Newbie
  • *
  • Posts: 12
    • View Profile
Re: Media Storage Idea
« Reply #4 on: January 03, 2008, 02:33:24 pm »
In the system I'm going to set up, I was planning on using a Fedora 8 Xen Dom0, I want to set up one Xen DomU using Openfiler as my storage system (crypted with truecrypt), and share the storage in a virtual NAS (and a real NAS to the MDs) - the LinuxMCE will run in a DomU as well.

So this topic is quite interresting to me. Especially the one parts about big partitions - because I'm planning on putting 4 TB drives into a raid 5 - and using it as one big partition.
The other thing I'm interested in, is the 3ware controller. I planned on getting a 3WARE 9550SXU-12.

Could you guys be more specific about those problems you were talking about? Since that could throw over a big part of my project...


posde

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 2846
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Media Storage Idea
« Reply #5 on: January 03, 2008, 03:00:21 pm »
Regarding the 3Ware performance problem. http://forums.storagereview.net/index.php?showtopic=25923 this is what I have found.

rgds
Oliver

Matthew

  • Douchebag
  • Addicted
  • *
  • Posts: 567
    • View Profile
Re: Media Storage Idea
« Reply #6 on: January 03, 2008, 04:13:55 pm »
750GB drives are currently about $165ea, so 2.25Tb is probably $525 shipped inside the US; 500GB drives are about $95 shipped, so 2TB is about $400. LMCE includes SW RAID that's probably just as good for home use as what you'd get from a SATA RAID card. You can get servers with 4xSATA for under $250. So $650-800 will get what you want with new HW. A cheap (but perfectly adequate, with LMCE SW RAID) 2xSATA card is something like $15, so an existing cheap PC with 2 builtin SATA ports will do. In either case you can probably also add an Orbiter to that NAS PC without lacking performance for either task.

rrambo

  • Guru
  • ****
  • Posts: 221
    • View Profile
Re: Media Storage Idea
« Reply #7 on: January 03, 2008, 04:25:25 pm »
After playing with some network storage together with LMCE I will not go that route. Maybe it should work, but my experience has been bad when using cifs or nfs mounted drives with LMCE. So, my installation(s) will not use NAS, but local storage. Maybe iSCSI is another route.

rgds
Oliver

I built a FreeNas setup for my storage....  very low hardware requirements and of course the software is free....  I currently only have about 1TB of media so can't comment on problems with larger filesystems...  I had absolutely no problems using this NAS with lmce 7.04...  after installing the 7.10 beta2 I've had some problems, but it could be bugs with the beta or I could have just had a problem with my setup....  (shares disappearing but coming back after rebooting the core/hybrid)

Sparkie

  • Guest
Re: Media Storage Idea
« Reply #8 on: January 04, 2008, 03:32:08 am »
Wow. You guys are really good! I modified my original post.

I forgot that I need to build a Core into this price. Cost has gone up to $2000. (Don't tell my girlfriend 8))

HAHA! Changes things a bit, I know...

Everyones input is appreciated! I'm sure there's tons of others looking for the same answers...

More ideas please!

posde

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 2846
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Media Storage Idea
« Reply #9 on: January 04, 2008, 01:10:39 pm »
After playing with some network storage together with LMCE I will not go that route.
[..]
[..]after installing the 7.10 beta2 I've had some problems, but it could be bugs with the beta or I could have just had a problem with my setup....  (shares disappearing but coming back after rebooting the core/hybrid)

This is exactly the problem I encountered. But it might have to do with me accessing the Samba server on the outside network.

rgds
Oliver

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5476
  • DOES work for LinuxMCE.
    • View Profile
Re: Media Storage Idea
« Reply #10 on: January 05, 2008, 06:01:35 am »
dsk: what specifically?

-Thom

Matthew

  • Douchebag
  • Addicted
  • *
  • Posts: 567
    • View Profile
Re: Media Storage Idea
« Reply #11 on: January 05, 2008, 07:42:13 pm »
This subject is offtopic to this thread. So I have posted it in a new topic, called "Ubuntu Home Server", copying this post. Please continue discussion of it in that other topic.


This project is coming along nicely:
http://www.ubuntuhomeserver.org/

I hope that both of these projects do not try to do to much (focus on getting the things they can perfect) but in the future could work together to make one hell of a system.  I like the security of the UHS, the GPL, and its data storage capabilites.  I like LMCE to play media and automate the house.  I worry about security in LMCE.

That UHS project is designed to make Ubuntu 0804 include more features for the home, including simpler GUIs for administering services for a networked multimedia home. Since LMCE is supposed to be a further packaging on top of Ubuntu, there's no reason they can't work together, if designed properly modularly. They each need to be able to omit their specific chosen components in favor of workalike substitutes that meet the same API. And their installers need to be complementary.

So UHS needs to be able to use the LMCE component where the two projects present alternatives. And LMCE needs to be able to install as packages onto UHS the way it currently does on Kubuntu, but also select (and substitute) between packages where they conflict between the two. The Debian/Ubuntu APT installation system accommodates that requirement well, so if it's used properly we can do it. But the main upgrade to LMCE is just being able to upgrade an LMCE's underlying Ubuntu against the Ubuntu repository systems, including a new Ubuntu version, without trashing the rest of LMCE. LMCE 0710 is supposed to allow that. If it does, integrating the two complementary systems will be fairly straightforward.

And beneficial. Mainly because there would be a larger, unified community of developers, who could work together on the unified set of tasks. And because the "conflicts" in each project's selections for functions also offer more choices, therefore appeal to a larger group of users who might prefer one or another choice. And also because the requirements for the integration would improve the overall robustness of each project, more integrable not only with each other, but also with other possible systems, either current or to come in the future.

I recommend people participate in this discussion in each project's forums. The UHS project started in reference to LMCE back in May, so we're catching up with them in cooperation. Even if the UHS project dies, it's worth looking at for more features and robustness as a home solution. Thanks for bringing it to our attention.

Zaerc

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 2256
  • Department of Redundancy Department.
    • View Profile
Re: Media Storage Idea
« Reply #12 on: January 05, 2008, 08:04:57 pm »
This subject is offtopic to this thread. So I have posted it in a new topic, called "Ubuntu Home Server", copying this post. Please continue discussion of it in that other topic.


This project is coming along nicely:
http://www.ubuntuhomeserver.org/

I hope that both of these projects do not try to do to much (focus on getting the things they can perfect) but in the future could work together to make one hell of a system.  I like the security of the UHS, the GPL, and its data storage capabilites.  I like LMCE to play media and automate the house.  I worry about security in LMCE.

That UHS project is designed to make Ubuntu 0804 include more features for the home, including simpler GUIs for administering services for a networked multimedia home. Since LMCE is supposed to be a further packaging on top of Ubuntu, there's no reason they can't work together, if designed properly modularly. They each need to be able to omit their specific chosen components in favor of workalike substitutes that meet the same API. And their installers need to be complementary.

So UHS needs to be able to use the LMCE component where the two projects present alternatives. And LMCE needs to be able to install as packages onto UHS the way it currently does on Kubuntu, but also select (and substitute) between packages where they conflict between the two. The Debian/Ubuntu APT installation system accommodates that requirement well, so if it's used properly we can do it. But the main upgrade to LMCE is just being able to upgrade an LMCE's underlying Ubuntu against the Ubuntu repository systems, including a new Ubuntu version, without trashing the rest of LMCE. LMCE 0710 is supposed to allow that. If it does, integrating the two complementary systems will be fairly straightforward.

And beneficial. Mainly because there would be a larger, unified community of developers, who could work together on the unified set of tasks. And because the "conflicts" in each project's selections for functions also offer more choices, therefore appeal to a larger group of users who might prefer one or another choice. And also because the requirements for the integration would improve the overall robustness of each project, more integrable not only with each other, but also with other possible systems, either current or to come in the future.

I recommend people participate in this discussion in each project's forums. The UHS project started in reference to LMCE back in May, so we're catching up with them in cooperation. Even if the UHS project dies, it's worth looking at for more features and robustness as a home solution. Thanks for bringing it to our attention.

So first you say this is offtopic and should be discussed in another thread, and then you go on to discuss it here...  ::)
"Change is inevitable. Progress is optional."
-- Anonymous


Matthew

  • Douchebag
  • Addicted
  • *
  • Posts: 567
    • View Profile
Re: Media Storage Idea
« Reply #13 on: January 05, 2008, 08:10:38 pm »

So first you say this is offtopic and should be discussed in another thread, and then you go on to discuss it here...  ::)

Yeah, that's how to continue a conversation when forking it. It's like a pager program overlapping successive pages by their first/last lines, so readers can preserve the context.

Now you are complaining about my offtopic post, which is metaofftopic (offmetatopic? offtopmetic?).

Zaerc

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 2256
  • Department of Redundancy Department.
    • View Profile
Re: Media Storage Idea
« Reply #14 on: January 05, 2008, 09:00:19 pm »

So first you say this is offtopic and should be discussed in another thread, and then you go on to discuss it here...  ::)

Yeah, that's how to continue a conversation when forking it. It's like a pager program overlapping successive pages by their first/last lines, so readers can preserve the context.

Now you are complaining about my offtopic post, which is metaofftopic (offmetatopic? offtopmetic?).
Seems more to me, that's how you annoy people with crossposting.  And I'm not complaining, just pointing out the stupidity of it all.  Maybe I should have just started another useless thread about that too, as it seems like the fashionable thing to do these days... :P

"Change is inevitable. Progress is optional."
-- Anonymous