Rebuilding my network with LinuxMCE this weekend, looking for advice

[caddywhompus]

I already had LinuxMCE running a few months ago, but shelved it temporarily because I had some issues.  I'm not in the process of putting it back in, and am looking for some advice or guidance on the hardware and layout I'm working with.

First, I've already built and installed the Core.  I'm not opposed to re-doing it at this time if that should be required.  I mostly built it as a test-bed for my hardware to assure it was all going to work and be stable, which it is.  My Core is a PIII 1.4t with 512MB and a 40Gb drive.  I have (2) 3com PCI 10/100 server NIC cards in it, and a USB 2.0/firewire 400 pci card for plug-n-play storage.  This box is rock-solid stable and has been running for several days already.  I do not intend to put my tuner card in this box, nor do any transcoding.  This box is strictly intended to be the Core, and serve only those functions, so I think the modest hardware is doable.  Correct me if I'm wrong.

Now, the first MD has some more power.  It is a dual-cpu PIII 1.4 with 2GB and a 500GB IDE drive.  It also has the PVR-500 for TV recording.  (This was my MythTV backend server and it worked very well)  I would like this machine to perform all the heavy-lifting for MythTV and I would also like the internal drive to be available storage space for recordings.  This part I'm not sure will work, so I need help here.  It appears LMCE will not use the internal drive for storage if I net-boot it.  Any way around this?  What if I move the drive to the Core as a 2nd drive?  I'm open to ideas here, obviously the 40GB drive the core runs on is not going to cut it!

And last, I am concerned about LMCE spreading my stuff out all over the network.  It detects network shares (very nice) and asks if it can use them.  I'm hesitant to allow it for a couple reasons.  First, will LMCE use up too much available space and fill up the hard drive of a Windows box?  And second, if my stuff is spread out all over the network, what happens if one drive fails?  Or maybe even more common what if that Windows box is turned off?  Do I lose data or corrupt files?  How can I safely backup my media if it is spread out across 4 or 5 network shares?

Thanks in advance for your assitance.

BTW all my TV recording is Standard Def, and will remain so for the foreseeable future.

[freymann]

First, I've already built and installed the Core.  I'm not opposed to re-doing it at this time if that should be required.  I mostly built it as a test-bed for my hardware to assure it was all going to work and be stable, which it is.  My Core is a PIII 1.4t with 512MB and a 40Gb drive.  I have (2) 3com PCI 10/100 server NIC cards in it, and a USB 2.0/firewire 400 pci card for plug-n-play storage.

You seem to like the P3's ;-)

I think you'd be better off with at least 1 Gig of RAM. 512MB is cutting it pretty thin.

MythTV records to the boot drive in the core, so your 40GB should be swapped out for that 500GB IDE drive you have in your MD.

You don't need a drive in your MD but if you want, drop the 40GB in there and let the core detect it, then use the LMCE User Structure on it for storage.

And last, I am concerned about LMCE spreading my stuff out all over the network.  It detects network shares (very nice) and asks if it can use them. 

LMCE doesn't put stuff on those network shares, YOU do! :-) It's up to you if you fill them up.

If you have media on a network drive, and you power off the unit, or the drive fails, then LMCE will know those files are no longer available and they won't display on the grid.

One thing that doesn't work well in this regard, is if you have music on those shares that aren't available? If you go to Music and say Play All it builds a playlist with all your music allright, including what is offline, so when it reaches one of those offline tunes? the music just stops playing and you're left wondering wtf? That's a bug, not a feature.

How you backup your media is up to you. In my case:

-our pictures are contained on our personal computers, and we copy them to the core. So we have multiple places where our pictures are stored (including portable devices that have copies too).

-my movies exist either in original DVD format (I just rip them as needed) or anything I download and want to keep, I burn to a data DVD. So if the system goes down, I can always copy over whatever I need from DVD again.

-my music, which is 23GB's, exists on original CD's (I have hundreds) and what I've downloaded in MP3 format. I recently spent a lot of time adding cover art just for LMCE (all my attributes I spent many hours on previously before my LMCE days). What I do for my music is mirror the audio directory to an external USB drive in my office (that isn't shared to LMCE at all). This way I have a duplicate copy of all my music and all the pretty little *.id3 files that spruce up the collection. Various chunks of our music exist on our notebooks and various hand held devices too.

Good luck with your system, and have fun!

[caddywhompus]

Hey, thanks for the fast response, I really appreciate it.

You seem to like the P3's ;-)

Guilty as charged.   Actually, I just have a bunch of them and trying to put them to good use.  The 1.4t was the "king of the P3" line and actually benchmarks faster than some P4 chips.  I know they are old, but I'm trying to use what I got.

I think you'd be better off with at least 1 Gig of RAM. 512MB is cutting it pretty thin.

For what it's worth, I agree.  However, the system can only handle 512mb.  It's limited by the chipset.

MythTV records to the boot drive in the core, so your 40GB should be swapped out for that 500GB IDE drive you have in your MD.

So MythTV records over the network?  That seems like a bandwidth problem even on a 100mbit switch?  What if I had 4 or 5 tuners?  Curious.  I also wonder then about the speed of post-processing like comm-flagging and transcoding which would then also be streaming across the network, unless they were occuring on the Core.  Hmmm, may have to rethink some of this.  How does the Core decide where to post-process MythTV stuff?

LMCE doesn't put stuff on those network shares, YOU do! :-) It's up to you if you fill them up.

Maybe I misunderstood then.  I thought LMCE was taking all the remote shares and grouping them into one "mount" on the Core.  Kinda like creating a Raid array over the network?  Am I wrong here?

and let the core detect it...

How exactly does one do that?  It appears that when I PXE boot a MD, the internal drive is completely ignored.  Is it because they are formatted NTFS?

my music, which is 23GB's...

It must suck to have such a limited music collection.  My collection of mp3 files is 208GB at last check.  I also have about 1TB of DVDs and downloaded movies/TV on an external firewire drive that I haven't even plugged into LMCE yet.

[freymann]

For what it's worth, I agree.  However, the system can only handle 512mb.  It's limited by the chipset.

I can't say for sure if 512mb of RAM is going to be a problem so perhaps somebody else can offer a pointer on that item.

So MythTV records over the network?  That seems like a bandwidth problem even on a 100mbit switch?  What if I had 4 or 5 tuners?

 I have one tuner in the core, and one on a MD, no issues with recording and playback with 4 machines. I think if you scan over the user setups (look here:  http://wiki.linuxmce.org/index.php/Category:User_Setups ) you'll see this isn't a problem for anybody.

 I think my commercial flagging happens right after a recording has finished. To be honest, I've never paid attention to which machine is actually doing the flagging, I assumed it has been the core.

 Under MythTV alone, you can configure if commercial flagging should be run on the master backend only, or on the backend that did the recording. I don't know how LinuxMCE has configured things, but again, I'd "guess" the core does the work.

Maybe I misunderstood then.  I thought LMCE was taking all the remote shares and grouping them into one "mount" on the Core.  Kinda like creating a Raid array over the network?  Am I wrong here?

Yes, that is not the case. This page may help:  http://wiki.linuxmce.org/index.php/Howto:_Using_Network_Shares

How exactly does one do that?  It appears that when I PXE boot a MD, the internal drive is completely ignored.  Is it because they are formatted NTFS?

If you have your dual network cards set up properly, then you'd boot your MD off the internet network. Once that MD is given it's IP address from the LMCE DHCP server, I believe there's a background script that will see if you have a drive in that box, and then it will prompt you if you want to use it, on screen. What I'm not sure of, is if you need there to be a public share or if it will just go ahead and create one. If you say Yes, you want to use that drive as a fileserver, then it should ask you if you want to use the LMCE User Directory structure. You are better off to say YES. It doesn't erase anything, it just creates folders in the manner explained above in the Using Network Shares link.

There is lots of info relating to network shares, network nas, et al. The forums and wiki should be pretty easy to search for more info on that. Sorry, I don't have a link here bookmarked to give you on that one.

It must suck to have such a limited music collection.  My collection of mp3 files is 208GB at last check.  I also have about 1TB of DVDs and downloaded movies/TV on an external firewire drive that I haven't even plugged into LMCE yet.

LOL. I thought it was a pretty good collection! :-)

When you do plug that firewire 1TB drive in, again, you should be asked about it, as mentioned above. Once LMCE creates the user shares, you'll need to move your music into the /public/data/audio/ folder, your videos into the /public/data/videos/, your pictures into /public/data/pictures/. And then,  give LMCE a few hours (or more) to fully catalog your content. Your music and pics will likely show up fine, but your videos will require some manual work to get them to display on the grid (when not sorting by Filename). Again, this is something you should look up eventually, the Files & Media system in the web admin, is where you add these 'attributes' you will hear about.... lots of forum talk about that one, what to do, what NOT to do, etc.

Just take things one step at a time. When you've accomplished one item, move on to the next, and if you're having trouble, post a message about that one issue, get that working, move on to the next, etc.

Maybe you can help me out a bit too by clicking on the applaud link by my name. For some reason my karma level has dipped below 0 in the last couple days!

[caddywhompus]

Maybe you can help me out a bit too by clicking on the applaud link by my name. For some reason my karma level has dipped below 0 in the last couple days!

I would certainly do so, if I could find what you are talking about?

I think my commercial flagging happens right after a recording has finished. To be honest, I've never paid attention to which machine is actually doing the flagging, I assumed it has been the core.

 Under MythTV alone, you can configure if commercial flagging should be run on the master backend only, or on the backend that did the recording. I don't know how LinuxMCE has configured things, but again, I'd "guess" the core does the work.

This I would like to get clarification on, maybe if someone else knows they could chime in.  I am very familiar with MythTV having run MythDora for 2 years now.  As such I definitely took advantage of the ability to process my jobs on the right hardware.  Hopefully LMCE has a way to determine which of the MDs can function as a Myth Backend and process jobs, and which can only be frontends and playback only.

[freymann]

Maybe you can help me out a bit too by clicking on the applaud link by my name. For some reason my karma level has dipped below 0 in the last couple days!

I would certainly do so, if I could find what you are talking about?

Scroll up a bit, while just reading the forums (not while typing your reply)...  in the left column you should see who wrote the message you are reading, and by the username is an "applaud" and "smite" link, which I guess translates to: click on applaud - gives that person a pat on the back; click on smite - gives that user a lump of coal.

Under MythTV alone, you can configure if commercial flagging should be run on the master backend only, or on the backend that did the recording.

Yes, but under LinuxMCE, it likes to keep things configured the way it likes. What you may run into is making some changes to the mythtv config only to lose those next reboot or reload router. Sometimes it's easier to do it the LinuxMCE way instead of trying to fight the system.

[caddywhompus]

Quote
Maybe you can help me out a bit too by clicking on the applaud link by my name. For some reason my karma level has dipped below 0 in the last couple days!
I would certainly do so, if I could find what you are talking about?

Scroll up a bit, while just reading the forums (not while typing your reply)...  in the left column you should see who wrote the message you are reading, and by the username is an "applaud" and "smite" link, which I guess translates to: click on applaud - gives that person a pat on the back; click on smite - gives that user a lump of coal.

Sorry, I got nothing.  Maybe because I'm using Opera.  It renders some screens a little different.

Sometimes it's easier to do it the LinuxMCE way instead of trying to fight the system.

Oh I absolutely agree!  What I meant was I was hoping there was a setting, maybe thru the admin page, to tell a MD to be a backend or not.  This just makes sense to me because of the varying hardware that a MD could be comprised of.  I definitely agree that following the appliance approach is the best way to get LMCE working stable.  When I read the forum (and I do) most of the problems I see people having a generally a result of trying to fight the current, so to speak.  I do not want to swim against the current, just looking for the information that I need to get on the raft!