Author Topic: Reorganizing development for scalability  (Read 34477 times)

darrenmason

  • Addicted
  • *
  • Posts: 529
    • View Profile
Re: Reorganizing development for scalability
« Reply #30 on: October 17, 2007, 01:22:24 am »
Daniel,

Have you managed to build the MythTV devices? I imagine this is where the focus of your fixes would be.

I have not tried to build a complete system as most of the changes have been specific to existing devices or new devices so they can be easily built in isolation and testing with any working system. I am guessing you are in this position as well. So you should be able to try your changes without waiting for the source respository/build process. Getting those changes out to everyone else may be more of an issue.

I am interested what the changes you have identified are, or actually more interested in what the actual problems are.
I personally think that there is a lot of issues in how tightly coupled the orbiter has become to some of the media devices and think that this is the cause of a lot of issues, as it makes the code brittle and too hard to do simple changes.
I also think that all teh Media player (XXX_player) devices should be contained by the media director itself and not the onscreen_orbiter - but that is another issue.

Glad to see you are showing an interest and look forward to your fixes.

Regards
Darren

darrenmason

  • Addicted
  • *
  • Posts: 529
    • View Profile
Re: Reorganizing development for scalability
« Reply #31 on: October 17, 2007, 01:30:08 am »
My own pet peeve is the inability to configure LinuxMCE to use an existing MythTV frontend - I have a frontend that has been recording since September 2004. It's rock solid and my family relies on it heavily. I am not willing to replace that with LinuxMCE, at least not in the foreseeable future.

I did manage to get a media station to use my existing backend but I think it required me to do something from the command line; can't remember the details.

Eloy,
Is it really an existing frontend that you want to use - or an existing backend?
I have had no problems using non linuxMCE frontends with the linuxMCE mythTV backend (on the CORE). My desktop runs Ubuntu 7.04 and that has a MythTV frontend that runs perfectly - and has been very stable.

From what I can see, there would also not be too many major issues with using an existing backend from linuxMCE as well (as long as versions matched etc) as the only really tie in that I can see between linuxMCE and MythTV is done on the MediaDirector and the shared file system.

Regards
Darren

eloy

  • Regular Poster
  • **
  • Posts: 19
    • View Profile
Re: Reorganizing development for scalability
« Reply #32 on: October 17, 2007, 03:27:00 am »
Is it really an existing frontend that you want to use - or an existing backend?

Whoops, sorry - I made a typo in my message two messages above. What I meant is that I'd love to be able to configure LinuxMCE to use my existing *backend*, which has been stable for years. The assumption today is that you'll use the mythbackend on the core.

Quote
From what I can see, there would also not be too many major issues with using an existing backend from linuxMCE as well (as long as versions matched etc) as the only really tie in that I can see between linuxMCE and MythTV is done on the MediaDirector and the shared file system.

Right. I got a media director to use my existing backend. I don't remember exactly what I did but it may have required CLI access, which is not a big deal. I just wish LinuxMCE played a bit better with things you may have running already, like MythTV and Asterisk.

Personally, I think that relying on any specific distribution is not the best way. For example, I run MythTV on Debian. It's nice that the MythTV software doesn't force me to run Fedora, or Ubuntu, or anything. I look forward to the day when I can do "apt-get install linuxmce" on my Debian box and get a working LinuxMCE installation without having to install another distribution, and without having to use versions of Asterisk or MythTV that have been tied to a specific LinuxMCE version. I hope this day comes one day :-)

Cheers,

Eloy.-
« Last Edit: October 17, 2007, 03:29:19 am by eloy »

danielk

  • Guru
  • ****
  • Posts: 153
    • View Profile
Re: Reorganizing development for scalability
« Reply #33 on: October 17, 2007, 05:22:02 pm »
Have you managed to build the MythTV devices? I imagine this is where the focus of your fixes would be.

I haven't actually tried this yet. I've build the Orbiter and some other modules. For MythTV the first thing I want to do is actually just configuration stuff. When running full screen it shouldn't be using a window frame, when recording NTSC it should record the captions, when recording PAL it should record teletext, it should be compiled with threading support, it should use a decent V-Sync (RTC or OpenGL V-Sync), the video directory should use the XFS filesystem and a daily defrag should be run, autodelete should leave more of the disk available for non-mythtv stuff by default to avoid both fragmentation and because we should assume that the partition contains more than MythTV recordings, we should set up the Linux kernel for A/V (enabling preempt smaller time slices), MythTV should be coordinating the time of the Schedules Direct download, MythTV should be themed to match the LinuxMCE theme, the orbiter OSD and MythTV OSD's are not coordinated, etc. etc.

MythTV is also just not as stable as it should be in the LinuxMCE environment. This will probably take a bit more work to get to the bottom of and fix. It looks like we're pulling some qt 3.3.8 packages, which is known not to work well in multi-threaded environments, but the problem is probably more complicated than that.

My own pet peeve is the inability to configure LinuxMCE to use an existing MythTV frontend - I have a frontend that has been recording since September 2004. It's rock solid and my family relies on it heavily. I am not willing to replace that with LinuxMCE, at least not in the foreseeable future.

I think you must mean an existing backend. This can be done, but is a bit problematic at the moment because of how the listings are downloaded. You basically need to make the LinuxMCE core machine the master backend, and turn your existing master backend into a slave backend. Or you can just keep both master backend running independently and point your frontend to the one you want to watch recordings from. This can be done by making a copy of your ~/.mythtv directory and editing the mysql.txt in that new directory with the DB info for the other master backend; then just run mythfrontend from that directory when you want to connect to the other master backend.

I personally think that there is a lot of issues in how tightly coupled the orbiter has become to some of the media devices and think that this is the cause of a lot of issues, as it makes the code brittle and too hard to do simple changes.
I also think that all teh Media player (XXX_player) devices should be contained by the media director itself and not the onscreen_orbiter - but that is another issue.


Can you give an example?

wsuetholz

  • Regular Poster
  • **
  • Posts: 44
    • View Profile
Re: Reorganizing development for scalability
« Reply #34 on: October 20, 2007, 01:19:46 am »
Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

On another note..  I would like to see the mythtv frontend be fully supported in the orbiter with the orbiter and rest of the system being more of an addon/plugin to mythtv rather then mythtv being an afterthought to the home automation that Pluto@Home provides.  In other words, I would like the MythTV Backend to basically take over the messaging/device handling portion of the system and basically become the HA/MM core that would allow for whatever as a frontend.  Be it an onscreen orbiter or a remote orbiter.  I wouldn't even mind seeing something like FreeVO or Elisa as a frontend to myth.


danielk

  • Guru
  • ****
  • Posts: 153
    • View Profile
Re: Reorganizing development for scalability
« Reply #35 on: October 20, 2007, 02:08:54 am »
Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

Those were definitively not ignored. We've been talking off-line. He's using a server that is also being used for other things so he can't let other people on it, except in a very limited way, nor is he comfortable using standard ports for various services on that machine so it's not ideal.

I have an SVN/trac/mailman server set up that I'm testing right now. Just give me a few more hours...

danielk

  • Guru
  • ****
  • Posts: 153
    • View Profile
Re: Reorganizing development for scalability
« Reply #36 on: October 20, 2007, 06:03:21 am »

SVN+TRAC: http://svn.charonmedia.org
DEV MAILING LIST: http://www.charonmedia.org

We may want to move this to linuxmce.org later, but this should work for now. I haven't finished setting up trac, but it has some spam prevention enabled, as does the mailing list (first post is moderated.) There are directions for the getting the svn trunk using a tbz download followed by an svn update. Other checkouts are left as an exercise to the reader, the tbz is less than400MB so it saves a lot of time on the checkout of the 1.4GB trunk branch.

There is no build script we can use yet. I've been working on this locally for a few days, but it will take a few more before I get something basic going.

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5549
  • DOES work for LinuxMCE.
    • View Profile
Re: Reorganizing development for scalability
« Reply #37 on: October 20, 2007, 07:14:58 am »
wsuetholz: MythTV's backend doesn't begin to approach the functionality offered by Pluto's DCERouter or its connected systems, I think you should take a serious look at the code. As for danielk and the rest of the developers being a bit quiet right now, I can understand this, there is a lot of scrambling going on to set things up. Be patient, or help out.

-Thom

chriss

  • Veteran
  • ***
  • Posts: 140
    • View Profile
Re: Reorganizing development for scalability
« Reply #38 on: October 20, 2007, 10:30:53 am »
Daniel,

awesome that you have the server up and running now!

@all,
as Daniel said before - my server is running some other stuff and I had to limit services (e.g. svn over http). Despite of that it's only a virtual server beeing not powerful enough when more people start developing. That's why we decided to wait until Daniel's server is up.

chriss

  • Veteran
  • ***
  • Posts: 140
    • View Profile
Re: Reorganizing development for scalability
« Reply #39 on: October 20, 2007, 11:00:05 am »
Daniel,

great job you did with the repository. I just noticed two things, first you tagged a "pluto-lmce-18249" revision. Is this meant to keep in sync with the pluto guys? Are you still working on getting Paul's two updates into the trunk? The second thing I was wondering about are all those commits having a log message starting with a hoizontal line ("---------------...)". Is this also pluto stuff, because it would be easier to track commits in the "Revision Log" if they have a good summary in the first line of the log.

Well, nothing to really worry about. Thanks for your work :)

/Chriss

danielk

  • Guru
  • ****
  • Posts: 153
    • View Profile
Re: Reorganizing development for scalability
« Reply #40 on: October 20, 2007, 05:13:26 pm »
The tag is there to help with any future syncs from pluto's repo, there is also a vendor branch where most of the action happens. (See the subversion documentation on vendor branches).

I haven't heard back from Paul since early last week, the plan was to let him commit his changes to the new tree.

All the commits starting with a horizontal line are pulls from the Pluto repo. I used a couple of scripts to do this. This was before I had trac installed so, I didn't realize that the extra lines would cause this problem, otherwise I would have filtered them out. The comments are still there and available via "svn log". Each commit also contains the pluto revision number it corresponds to. These were applied in order so you can see that there are some missing revisions. This is mostly due to commits to other branches in the same repo, unrelated to LMCE.

chewi

  • Veteran
  • ***
  • Posts: 69
    • View Profile
Re: Reorganizing development for scalability
« Reply #41 on: October 20, 2007, 05:58:26 pm »
Greate Work !!!

As to the question on the trac-wiki: I don't think, an automatic import to trac would be a good idea...

avajon

  • Veteran
  • ***
  • Posts: 120
    • View Profile
Re: Reorganizing development for scalability
« Reply #42 on: October 21, 2007, 09:33:14 pm »

SVN+TRAC: http://svn.charonmedia.org
DEV MAILING LIST: http://www.charonmedia.org

We may want to move this to linuxmce.org later, but this should work for now. I haven't finished setting up trac, but it has some spam prevention enabled, as does the mailing list (first post is moderated.) There are directions for the getting the svn trunk using a tbz download followed by an svn update. Other checkouts are left as an exercise to the reader, the tbz is less than400MB so it saves a lot of time on the checkout of the 1.4GB trunk branch.

There is no build script we can use yet. I've been working on this locally for a few days, but it will take a few more before I get something basic going.

really cool  ;)
thanks daniel!!!

wsuetholz

  • Regular Poster
  • **
  • Posts: 44
    • View Profile
Re: Reorganizing development for scalability
« Reply #43 on: October 22, 2007, 04:28:41 am »

Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

Those were definitively not ignored. We've been talking off-line. He's using a server that is also being used for other things so he can't let other people on it, except in a very limited way, nor is he comfortable using standard ports for various services on that machine so it's not ideal.

I have an SVN/trac/mailman server set up that I'm testing right now. Just give me a few more hours...

Thanks for clearing that up.  It seemed like somebody was offering to help, and had already set up most of it, but was being ignored..  Nothing more needs to be said...

wsuetholz

  • Regular Poster
  • **
  • Posts: 44
    • View Profile
Re: Reorganizing development for scalability
« Reply #44 on: October 22, 2007, 04:46:33 am »
wsuetholz: MythTV's backend doesn't begin to approach the functionality offered by Pluto's DCERouter or its connected systems, I think you should take a serious look at the code. As for danielk and the rest of the developers being a bit quiet right now, I can understand this, there is a lot of scrambling going on to set things up. Be patient, or help out.

-Thom


I understand that MythTV's backend is doesn't contain the functionallity of DCERouter at this time.  My suggestion was that maybe it should.

My real issue has always been the disconnect between a MythTV Backend setup and the way that the Pluto Media access works.  If you wish to have a MythTV Frontend accessing the media, you need to define all the information for the media once for MythTV and once for Pluto.  OTH I like the additional options allowed by having different storage trees for different users.  Also I find the MythTV addon to the orbiter to be rather clunky... I also realize that this project is still taking baby steps trying to transition to a multi-developer project.  Kudos for the job done so far, it really is quite amazing for one person to have accomplished so much in so little time.  I have been playing with this system off and on for a couple of years, and most of the issues I have with it are inherited from Pluto, and I'm sure will get resolved now that there is some momentum for the LinuxMCE project.

Helping..   :D  Everytime I commit to help with a project like this I get hammered at work...  I will be attempting to get a AMD 64bit version compiled sometime in the near future I did try once already, but that drive is gone.., For right now I've abandoned my LinuxMCE setup in favor of a 64bit MythTV install which was  working for me better then the LinuxMCE install was until the TV's HDMI port died  S-Video stinks.

I explained my reasons for the message in my response to DanielK, it wasn't impatience at all.