Author Topic: Media Storage Idea  (Read 3741 times)

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5501
  • DOES work for LinuxMCE.
    • View Profile
Re: Media Storage Idea
« Reply #15 on: January 05, 2008, 10:48:52 pm »
dsk: It is a known problem, and we are trying to rectify it... We would really like some help in this area.

-Thom

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5501
  • DOES work for LinuxMCE.
    • View Profile
Re: Media Storage Idea
« Reply #16 on: January 06, 2008, 12:24:47 am »
every bit of help would be fantastic. :)

-Thom

Sparkie

  • Guest
Re: Media Storage Idea
« Reply #17 on: January 06, 2008, 03:55:27 am »
Thanks for making the other post. Not sure how this went off topic. But it's okay! Let's continue helping me build my media storage!
Refer to the first post.
Help me out fellas!

dragon_788

  • Regular Poster
  • **
  • Posts: 15
    • View Profile
Re: Media Storage Idea
« Reply #18 on: January 07, 2008, 07:03:53 am »
Sparkie, you might look into unRAID, its a VERY promising solution (that delivers from what I've read) that allows you to have one disk crash without losing any data, and even if two disks die, you ONLY lose the data on those two disks, the rest of the disks are still readable (this is NOT possible with any other kind of consumer affordable RAID). Please check the forked thread for a link to my post in the Ubuntu Home Server forums about my dream of the ultimate trifecta of servers/software. BTW, unRAID is sold by Lime Technology, but they have a free version that supports 3 drives (2 data 1 parity) so you could get at least 2tb of storage just by buying 3 1tb drives (pricey, but hey, that's a LOTTA space).

delta_9

  • Regular Poster
  • **
  • Posts: 25
    • View Profile
Re: Media Storage Idea
« Reply #19 on: January 07, 2008, 11:57:07 am »
don't have much to say other than get the drives as big as you can 1TB as the more disks you have the more power you use to spin the little suckers up the drives cost more upfront but will save in the long run man if you run a core 24/7 make you cash back in no time good luck getting it "smooth" man.

dgirard

  • Newbie
  • *
  • Posts: 14
    • View Profile
Re: Media Storage Idea
« Reply #20 on: January 07, 2008, 06:55:17 pm »
Here's another related question...

What's the best platform for the storage?...NFS?...Jbod (local disk)?...or Samba/Cifs?

On the data safety standpoint...the idea of unRaid sounds appealing to me and it seems I need to do a little reading on that one...from a backup perspective...I'm using Mosy...4.95 per month for unlimited on-line system backup....it's going to take me a year to get everything backed up a 700K/sec, but then it'll be safe in the event of any catastropy...I wonder if they'll still like me as a customer once I get 4Tb backed up there...  ;D

I dont' knwo if Mosy has a unix client...I do mine from my windows server.

posde

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 3014
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Media Storage Idea
« Reply #21 on: January 07, 2008, 08:07:53 pm »
What's the best platform for the storage?...NFS?...Jbod (local disk)?...or Samba/Cifs?

For a comparison, look here http://www.javvin.com/NFSdoc/nfsperformance.html, and to broaden your choice even more, look at iSCSI, which might be an even better choice, if you want to have network connected storage http://www.technomagesinc.com/papers/ip_paper.html

Apart from that, local storage is ALWAYS fastest.

rgds
Oliver

Sparkie

  • Guest
Re: Media Storage Idea
« Reply #22 on: January 08, 2008, 04:37:49 am »
Since I am building the core from scratch. Would it make sense to buy a mobo with 6 or so sata ports? 1 disk for Core software. and 2 or 4 disks plugged directly into the other ports? Using software RAID?

blackoper

  • Veteran
  • ***
  • Posts: 110
    • View Profile
Re: Media Storage Idea
« Reply #23 on: January 08, 2008, 06:15:52 am »
ooh goody finally a topic I'm an expert in..
for $2000 you can get massive amounts of storage.
I personally think it is wasteful to use an entire computer just as a fileserver. There is not much resource usage on one, as you are pretty much limited by the drives. I have a combined system using software raid. Currently there are 12 Sata drives in it, and one ide 2.5" laptop system drive. Total storage ~5.5TB
Also with the newest 2.6.24 kernel that will be out in the next month, sata port multipliers will be supported by default. Buy two sata sil3132 pci-e 2 channel sata 2 cards and you have 20 drive software raid (i'm already doing it but it takes a custome compiled kernel)

The sweet spot right now is the 750GB drives. For $143 a piece I bought some brand new western digital 750gb drives with 3 year warranty.
so cost breakdown of a kick arse file server or combo box

Case - Coolmaster Stacker - up to 20 drives in the front using 5in3 sata backplanes $90 @- http://www.directron.com/stct01ww1.html

5in3 sata backplanes - go for about $100 - here is a comprehensive listing http://wiki.lime-technology.com/wiki/index.php?title=SATA_Hard_Disk_Stackers

Power supply - something over 600 watts with active PFC to have high effiency. I'm using a 750 Toughpower

Mobo and cpu- amd or intel.. anything with low power usage and high reliability. Also the more pci and pci-e slots the better. Try getting one with built in firewire as well. I really like the intel E2180 chip as it uses very little power (12-18 watts in idle) it also overclocks to 3.5ghz easily on air if you need it to.

Ram - 2gb of cheap ram is fine. There is always a deal going on with HP ram for $7 per GB after rebate pc 800 chips

So pricelist so far
5x750gb drives in a software raid 5 - less than $750 also there are 4 internal ide spts in the included 4in3 in the stacker case, so you can buy a few more and include those in the raid5. If you use more than 5 disks in a raid5, I'd recommend having one set up as a hotbackup in case one fails.
stacker case $100
5in3 sata backplane - $100
power supply - $100
motherboard - we'll say  $100
processor - again we'll say $100
ram - $20 after rebate
high quality power supply (do not skimp!) - $100
for 5 sata drives you can probably use on board without extra pci-e add on cards  so free there
System drive - $50

total cost = less than $1420

As for a comparison of Samba vs NFS vs iSCSI
Samba transfer speeds over gigabit ethernet - Raid 0 - 7200 rpm sata 2 32MB cache drives - write - 85MB/s  - read - 98MB/s - gigabit maximum is 125MB/s - I have dual bonded gigabit - adding a 2nd computer using the servers bonded ethernet - total write - 110 MB sustained writes, Reads- 138MB/s
iSCSI - write - 50MB/s with a much higher top processor/memory usage - Read - 70MB/S - iSCSI isn't a good standard. It's a resource hog - couldn't push it anyhigher using bonded gigabit
NFS - about the same as samba, but I get a lot of IObound errors when using it with myth. Gave up on nfs as samba is more universal.
« Last Edit: January 08, 2008, 09:04:06 am by blackoper »

Matthew

  • Douchebag
  • Addicted
  • *
  • Posts: 567
    • View Profile
Re: Media Storage Idea
« Reply #24 on: January 08, 2008, 06:56:26 am »
I'll be receiving this week a $20 11-bay tower case with 350W powersupply. Into it will go the first 4 750GB drives I got for $140ea/shipped. I'm going to try using some $8 SATA/USB adapters, each with its own SATA PS. The case's own PS is for future equipment, and at $20 why turn it down? A $30 7-port USB hub will let it all plug into my Core's single USB port. Since I've got a 100Mbps LAN, the 4 drives will bottleneck at the LAN, not over USB's <480Mbps.

For those not doing the math at home, that's $642 for 3TB, expandable to 7.5GB for another $918. I could probably go for an extra $75 450W PS and power all the drives without their dedicated PS bricks, probably a power savings.

If it doesn't work out as USB, I'll probably add a cheap mobo with 100Mb ethernet, with either a lot of onboard SATA connectors, or some multiport SATA PCI cards, or both, whichever is cheaper.

LMCE doesn't use anywhere near the performance available on these high-capacity components. In fact, I wish I didn't have to pay for such fast transfer rates, seek times and caches, if I could save an extra 10% for that much more storage.

Anyone know of a <$1000 jukebox for 500-800GB tapes, that will work with LMCE like the 200-disc changer supported internally by mtx?