Author Topic: Newbie with Installation from Scratch  (Read 16100 times)

colinjones

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 3003
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #30 on: January 28, 2009, 05:29:05 am »
I'm sorry, Indulis, I simply do not agree. For the last 8 years I worked for a large multinational - as the Infrastructure Operations manager. I was intimately involved in the design, specification, purchasing and project implementations of numerous SAN, DAS, NAS, iSCSI, archival platforms and ultimately made the decisions on what approach to take. I made these decisions in light of years of experience with storage systems and in conjunction with Professional Services project advice from vendors such as EMC and IBM. On several occasions they even modelled performance of various suggested configurations.

In point of fact, on high end systems like SANs, the actual containers are so far abstracted from the RAID subsystems through meta- and hyper-LUNs, as almost to make no difference. Nevertheless, in the last 5 years, across hundreds of LUNs, many hundreds of servers, and several data centres, I recall only once implementing a RAID5 array, and that was a compromise at the time (which of course have a tendancy to stick and come back to haunt you!)

Notwithstanding that, asking such a vendor to implement a RAID 5 array is invariably met with strange looks, and ardent advice to the contrary. In my interface with peers in other medium and large enterprises with which this organisation operated, none ever considered RAID 5 to be an "enterprise" solution. In fact I have to go back 12 years, to my days as a small solutions provider to offices and retail establishments of 10 people or less and a single "server" plus dialup modem, before I can recall regularly using RAID 5.

Saying that RAID controller caches mask write penalties is a profound misunderstanding. It is simply wrong. Caches always run at 100%, therefore it merely shifts the problem... and that is circular... you don't get something for nothing.

Bringing this to the point of LMCE. Performance between different RAID technologies varies dramatically depending on what you are doing, as you pointed out. For instance, random writes on RAID10 are quiet poor compared with other technologies (such as RAID5). However, in pretty much every other test, RAID10 is far superior to RAID5. And in particular, sequential reads are vastly faster - note the vast majority of what LMCE does is sequential reads. Note that with RAID10 (and equivalents), adding spindles progressively and nearly linearly improves performance for most operations. In RAID5, write operations in particular get slower and slower as the subsystem has to read more stripes from more disks to calculate parity before writing it. This in turn causes blocking I/Os within the disk subsystem. Advanced systems can offload some of this to an extent, but never completely.

Transactional read performance is almost irrelevant - this is where "caching" does come in. In every database technology you can name, the db engine sets up a read-through buffer in core, and typcially has very sophisticated replacement algorithms. eg in MS SQL and particularly MS Exchange, the bulk of reads need to come from this buffer for the system to perform acceptable - this is particularly true of Exchange, which commonly achieves 70-90% buffer hits. This demonstrates where a disk cache becomes pointless - the buffer is usually so much larger than the disk cache and dedicated, that if it didn't hit the buffer, then it certainly isn't going to hit the disk cache! This is usually even true of high end Clariion and Symetrix SANs which have disk caches of at least 8-16GB (not MB!)
« Last Edit: January 28, 2009, 05:35:04 am by colinjones »

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5549
  • DOES work for LinuxMCE.
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #31 on: January 28, 2009, 07:07:46 am »
Thank you for nailing that on the button. If _I_ had said it, he would probably be barking back at me.

-Thom

colinjones

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 3003
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #32 on: January 28, 2009, 07:47:20 am »
Taking a look at http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5 is instructive, particularly the section on RAID5 performance. It doesn't completely enumerate the issue with read-modify-write, although I have read detailed technical breakdowns elsewhere, but it does give a feel for it. Essentially, say you wanted to increment a counter in a database, the steps you would need to take are:

Read that entry from disk
Increment it in memory
Write the entry back to disk (atomic transaction)
   Sequential write to transaction log
      Read all data blocks plus parity block in that stripe
      Calculate new parity block value once transaction entry is written
      Write transaction log entry
      Write new parity block
   Random write to database
      Read all data blocks plus parity block in that stripe
      Calculate new parity block value once db entry is updated
      Write db entry
      Write new parity block
End process

If this was on an 6 disk RAID5, it would correlate to 17 soft I/O operations to the disk surface. A similar RAID10 would correlate to 5 such operations, moreover, the number increases which each disk added to a RAID5, it remains the same when increasing a RAID10.

Also, note the article compares with RAID0 - which for read purposes is the same or slower than RAID10 (because you can read multiple sequential or random pieces of data from multiple channels and disks simultaneously) - and it indicates that RAID5 is "slightly slower". For write performance in clearly indicates that it is dramatically slower (sometimes by as much as "an order of magnitude"). And mentions a group I had never heard of! BAARF - Battle Against Any RAID Five. An organisation I would have been a card carrying member of had they existed and I had been aware of them back in my DB days!! :)

nite_man

  • NEEDS to work for LinuxMCE
  • ***
  • Posts: 1019
  • Want to work with LinuxMCE
    • View Profile
    • Smart Home Blog
Re: Newbie with Installation from Scratch
« Reply #33 on: January 28, 2009, 09:04:28 am »
I have Asus M3N78-EM on my core/hybrid. So far I found following issues with it:
  • Integrated NIC works with modern kernels starting from 2.6.26. So, to use it with 0710 kernel should be recompiled
  • Sound over HDMI doesn't work under 0710 even with the latest kernel and ALSA. Maybe I did something wrong when recompile the kernel. It definitely should work under 0810.
  • Some strange problem with connection over HDMI. TV brightness is changed periodically. Especially when menu items are changed (browsing of video file options, for example). Sometimes it happens during playback DVD rip. I changed the HDMI cable but result is still the same. I have to find some other device with HDMI output (the rest are connected via SCART and work fine) and see if it solves the problem.

I didn't test RAID because I use NAS.
Michael Stepanov,
My setup: http://wiki.linuxmce.org/index.php/User:Nite_man#New_setup
Russian LinuxMCE community: http://linuxmce.ru

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5549
  • DOES work for LinuxMCE.
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #34 on: January 28, 2009, 05:17:00 pm »
The storage radar will always automatically pick the NAS with the most storage space in the storage selector by default.

-Thom

colinjones

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 3003
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #35 on: January 28, 2009, 10:00:56 pm »
Hi,
Thanks for the very elaborate explanation. Looks like I will be revising my storage setup and ditch the RAID5.
If I do end up with several NAS units, is there any way to specify that music should go on NAS1, movies on NAS2, etc? Or expendable vs. valuable?

And mentions a group I had never heard of! BAARF - Battle Against Any RAID Five. An organisation I would have been a card carrying member of had they existed and I had been aware of them back in my DB days!! :)
;D

Not sure I follow - music doesn't "go" anywhere as such. The only thing that "go"es somewhere in the sense you are talking about is when you record live TV as this the only time that LMCE writes something new. And Thom has answered that for you. Oh yeah, ripping DVDs/CDs, silly me - same thing, it will present you the options which you can then override if you wish...

colinjones

  • Alumni
  • LinuxMCE God
  • *
  • Posts: 3003
    • View Profile
Re: Newbie with Installation from Scratch
« Reply #36 on: February 03, 2009, 09:19:06 pm »
I'm assuming that the logic is just how Thom suggests, it will show the partition with the most space. I don't think you can set a default, certainly not by media type anyway. Really, its just going to present you with 2 or 3 location options in a menu with the largest at the top, so its a single click. You have to navigate to the subfolder you want the rip to be in anyway, so when ripping, allong with choosing which tracks, public/private, and giving it a name, saving on a single extra click will do nothing for ease or useability!

totallymaxed

  • LinuxMCE God
  • ****
  • Posts: 4660
  • Smart Home Consulting
    • View Profile
    • Dianemo - at home with technology
Re: Newbie with Installation from Scratch
« Reply #37 on: February 03, 2009, 09:44:57 pm »
- The 8300 chipset is MOBO integrated, so selecting a different one because 8300 would be wasteful means selecting a different MOBO. I have no issue selecting a different one but would then appreciate knowing which NVIDIA would be the best to have without going overboard. The page in hardware here (wiki.linuxmce.org/index.php/Hardware) mentions "It is recommended to use one from a GeForce 6200 to a GeForce 8500". That's why I went with the 8300. Don't know if it matters, but obviously I prefer to be able to send 1080p whenever possible. If it's that important to go lower I could save about $20.00 right there, so that's good. So please advice what number should I go to.
- I'll drop to 1Gb RAM (saves me some money to buy the NIC)  ;)
- I'll drop the internal HD to say...160GB? (saves some more money there too)

   Now for the big question: How could I play BluRay? Should I (can I) just buy a separate stand-alone player and somehow connect it to LinuxMCE? And, still be able to keep one control over everything, so LinuxMCE controls everything? The video mentions connectivity to VHS, so I assume you could also connect a BluRay player. It'd be kinda strange to have two different drive bays there, but hey! you do what you need to do, right...
What say the gods?

Thank you,
Ray


Hi Ray,

Colin is right in saying that currently there is not too much to be gained from using any nVidia card above say a 7300. But if you want to be able to exploit the forthcoming improvements in the nVidia driver and or use onboard chipset based nVidia graphics then really you should look at nVidia 8000 series cards or chipsets and above.

Re the external BlueRay player... yes this is possibly the best route to go currently. Something like the Denon 2500 or 3800 RS232 enabled BlueRay player routed through a Denon Surround Amp such as the 2309 would work well. Currently there are no device templates for the 2500/3800 players (we will be working on one or you could create one yourself). Even without the device template and RS232 control this would provide a very good BlueRay source (they are not cheap though!).

All the best

Andrew
Andy Herron,
CHT Ltd

For Dianemo/LinuxMCE consulting advice;
@herron on Twitter, totallymaxed+inquiries@gmail.com via email or PM me here.

Get Dianemo-Rpi2 ARM Licenses http://forum.linuxmce.org/index.php?topic=14026.0

Get RaspSqueeze-CEC or Raspbmc-CEC for Dianemo/LinuxMCE: http://wp.me/P4KgIc-5P

Facebook: https://www.facebook.com/pages/Dianemo-Home-Automation/226019387454465

http://www.dianemo.co.uk

Ray_N

  • Veteran
  • ***
  • Posts: 57
    • View Profile
    • Your Plan B!
Re: Newbie with Installation from Scratch
« Reply #38 on: February 03, 2009, 11:59:24 pm »
Thanks for the insight Andrew, it does help. However, I just received all the parts shipment yesterday and started building the system already. On the BluRay, I decided to go the "rip somewhere else" route...'cause you're right on... those players aren't cheap.

Thank you...looking forward to contributing my successes/frustrations. :-)
R