LinuxMCE Forums

General => Developers => Topic started by: danielk on October 10, 2007, 03:51:56 pm

Title: Reorganizing development for scalability
Post by: danielk on October 10, 2007, 03:51:56 pm
Hi,

I've been lurking here for a while and have decided to get involved. I just started running LinuxMCE at home recently; I want to set up an Asterisk system for my phones instead of relying on packet8, and want this server to run MythTV as well. The MythTV integration in LinuxMCE could use some work, so that's my coding interest in LinuxMCE. But I've talked with Paul about setting up a public SVN repository and otherwise making it easier for developers to get involved.

The public SVN doesn't need to be sold as an idea, some of you have already set up a repository, but I would also like to make the build scripts simpler so that you can just do a "./configure; make ; make install" on the modules, and maybe a "make cdrom" to make an install disk. I also think a mailing list rather than a forum would make sense for developers. And finally, I would like to move to trac for bug tracking, mostly because it can be integrated with svn and show exactly which issues have been fixed and in which version they were fixed. This wouldn't impact the 6 month LinuxMCE release cycle, but it would make it easier for developers like myself to get involved.

Opinions?

--
Daniel Kristjansson
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 10, 2007, 06:40:24 pm
honestly, why don't we use a distributed version control system like git or darcs?

given the immense size of the code-base, git may very well be the better option.

-Thom
Title: Re: Reorganizing development for scalability
Post by: danielk on October 10, 2007, 07:31:58 pm
darcs is not ready for prime time.

As for git, there is no trac equivalent that I'm aware of, and it would also be harder to sync with other code bases which use svn such as MythTV, Pluto, Asterisk, etc.
Title: Re: Reorganizing development for scalability
Post by: chriss on October 10, 2007, 09:26:55 pm
Daniel,

thanks for that post ;)

Quote
But I've talked with Paul about setting up a public SVN repository and otherwise making it easier for developers to get involved.

What are the results of that talk? What is Paul's opinion about a public SVN? I already offered a public SVN over at this post (http://forum.linuxmce.org/index.php?topic=2277.45 (http://forum.linuxmce.org/index.php?topic=2277.45)):
Quote
some posts above I offered to setup a SVN repository for the sources. I'm surprised nobody commented on that. Did I miss something, like some repository somewhere else? Anyway, I did setup the repository by now; unfortunately it is nothing more than the 0704 sources from bittorrent with the following policy:
  • anonymous read access
  • authentication is required for modifications/commits (anybody who wants to become a developer?)
  • Traffic limited to 50GB/month @ 2MBit/s (25 kBit/s when exceeded)

To be honest, I'm surprised that nobody responded to this post (or the one beeing about two weeks older). I remember a lot of people asking where to get sources, to submit patches and so on. However, none of them gave any comments :(
Now, this repository is ready to go with the 0704 sources, there's Trac up and running (though not fully configured yet).

Cheers,
Chriss
Title: Re: Reorganizing development for scalability
Post by: hari on October 10, 2007, 09:47:07 pm
Quote
To be honest, I'm surprised that nobody responded to this post (or the one beeing about two weeks older). I remember a lot of people asking where to get sources, to submit patches and so on. However, none of them gave any comments :(
Now, this repository is ready to go with the 0704 sources, there's Trac up and running (though not fully configured yet).
can you make the url public?

best regards,
Hari
Title: Re: Reorganizing development for scalability
Post by: danielk on October 10, 2007, 10:09:20 pm
Paul likes the idea of a public SVN. But he doesn't have the time to administer it. Last I heard from him he was going to set me up with root so I could do it, this was a couple days ago. But he also told me he had pressing issues to attend to outside of LinuxMCE right now.

I would like to get as much revision history as possible into this tree so starting from 0704 would not be ideal. I also contacted Pluto about getting the trees synced up. If possible, I may actually start with a clone of their tree and just add any LinuxMCE changes on top. I just talked to their LinuxMCE liaison, Razvan, today and he tells me that his tree is pretty much in sync, but the differences between LinuxMCE and Pluto's product are significant these days. Unfortunately, the e-mail exchange was cut short due to the time difference; he's somewhere in Europe and I'm in NYC.

Hari, did you make the URLs public? Maybe we can at least use the trac there until I get svn set up with a revision history. We don't want contributions to fall through the cracks.
Title: Re: Reorganizing development for scalability
Post by: chriss on October 10, 2007, 10:54:55 pm
Quote
I would like to get as much revision history as possible into this tree so starting from 0704 would not be ideal

I agree with that and I already asked for an SVN dump in my first post. If anybody would provide that revision history I will put it into the repository. However, I think this repository will only be a temporary solution so it should be possible to apply all accepted diffs to the complete revision history later on. 8)
My point was to get the development going.

Quote
Hari, ...
Chriss, not Hari.  ;)
Quote
...did you make the URLs public
But no, not yet, because I was waiting for a discussion like this one.

Quote
Maybe we can at least use the trac there
Uh, does it really work to host Trac and the repository on different servers? Didn't know that...
Title: Re: Reorganizing development for scalability
Post by: chriss on October 10, 2007, 10:57:18 pm
Hari,

Quote
an you make the url public?

sorry, I didn't make the URL public yet, because I want to avoid everybody grapping the code and we are ending up with several SVNs not beeing in sync  ;D

/Chriss
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 10, 2007, 11:45:37 pm
would you care to elaborate on why "darcs is not ready for prime time?"

or are you just talking out of your arse?

-Thom
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 10, 2007, 11:52:39 pm
also, you might want to do a little bit more searching....

http://download.fedora.redhat.com/pub/epel/5/i386/repoview/trac-git-plugin.html

you can take your foot out of your mouth, now.

-Thom
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 10, 2007, 11:55:48 pm
Also, where do you get off saying it would be harder to integrate with other SVN trees with these methods? why don't you do a little research? People like you seriously piss me off because you just say things without actually either (a) looking them up, or (b) having the experience of extensively using systems that others suggest.

I happen to have both on my side.

-Thom
Title: Re: Reorganizing development for scalability
Post by: teedge77 on October 11, 2007, 12:16:59 am
 :D

everyone gets so crazy on this forum. i thought the people on the general hospital forums my wife talks about were bad.
Title: Re: Reorganizing development for scalability
Post by: PeteK on October 11, 2007, 12:22:46 am
When we're all done with our little nerd fight, can someone please set something, anything up?  ;D I've got an alpha/beta of the Insteon plug-in, and I'd like to start getting it out and getting some early testing done.  I've tried getting a hold of Paul over the last few days, but it sounds like he's got other commitments.
Title: Re: Reorganizing development for scalability
Post by: billytwowilly on October 11, 2007, 12:23:33 am
as mentioned earlier, we should just use launchpad. We're integrating with kubuntu anyway, and it's what they use to track the ubuntu project, and it's free and it has great bug tracking via bazaar. Did I mention it's free and we can have it setup in an hour if there is an svn repository to pull from? Oh yah, and anyone can make a branch for themselves to play in?
Title: Re: Reorganizing development for scalability
Post by: danielk on October 11, 2007, 12:52:24 am
I tested a bunch of revision control systems last year and darcs lost data. As did arch, the one with my favorite UI of the distributed revision control systems at the time. Darcs was also difficult to work with, a problem that arch and git don't suffer from.

I didn't know someone had written trac scripts for git, I stand corrected. But I still don't want to use a 0.0.1 script to link the revision control system and bug tracking systems. The whole reason I want to get involved is to make it easier for developers to jump right in and start coding without having to learn a whole bunch of new systems. Perforce, svn, cvs, rcs and ClearCase are probably the only revision control systems that fit that criteria. Of those, svn and cvs are only ones it makes sense to use for a new repository for open source development, and most projects that were using cvs have switched to svn for faster diffs and better branching.

If this is a monumental success with hundreds of active developers we can always switch to git or bazaar or whatever is best at the time later. The important thing now is to get a repository setup with all the latest changes and on a server that can handle the load..
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 11, 2007, 01:39:08 am
okay...so...you...tested and glanced at a smattering of version control systems....

I can understand the familiarity with svn, with cvs..

however, given the sheer size of the code, the number of individual sub-systems within this code, and the potential for so many people to be working on these subsystems independently, it is much easier to test individual ideas within git, or darcs, or bzr, than it is to deal with svn.

I know i'm pressing hard on this issue, but i want you all to seriously consider and weigh the pros and cons. SVN and CVS are badly suited for this project in the long run.

-Thom
Title: Re: Reorganizing development for scalability
Post by: darrenmason on October 11, 2007, 01:56:00 am
Lets not turn this into a "this tool is better than that tool" debate.

If someone sets something up and it meets the requirements, then use it.

The system that Pluto had was effective (at least from a user/developer end).
Although the code changes are important, the metadata is just as much so and often coupled together anyway.
For instance, if I add a new Device Template and maybe a couple of new parameters to go with it, then I write C++ code to implement that - then in order to contribute that back to the community I need to be able to submit the code as well as the database delta. Pluto had this working with their sqlCVS product  (that is part of the code they developed)
Changes would still need to be accepted by admins I believe.

At a minimum we need to setup the same system that Pluto has, they may still be open to us using their servers for it and granted some form of access, since most changes will directly benefit them.

I am not sure that free server space or a limited hosting arrangement will meet the requirements, but I don't have experience setting this up which is why I have kept out of it. I have only posted this as a clarification of what I think our(mine?) requirements are.

regards
Darren
Title: Re: Reorganizing development for scalability
Post by: eloy on October 11, 2007, 03:52:17 am
Hey Daniel,

I agree with everything you said in your first post in this thread. Your reputation precedes you (thanks for your leadership and contributions to MythTV) so I hope you become involved in this project as well since I am sure that'll make things improve a lot.

Cheers,

Eloy Paris.-
Title: Re: Reorganizing development for scalability
Post by: chriss on October 11, 2007, 09:25:26 am
Hey Folks,

personally, I don't see why SVN shouldn't be suited to handle that code size (I'm using way bigger repositories at work), but I think this debate is kind of obsolete at this particular moment. This project is suffering from a missing repository and the goal should be to set up one ASAP! Since Pluto used SVN and (as far as I remember) several forum members, including Paul, stated that they are using SVN, I see no reasong why we shouldn't start with SVN - it's just the easiest way to get this rolling ;)

I agree, that we should discuss the matter of the right repository - espacially if it turns out that SVN does not work as expected. But please, don't discuss everything until it's dead :/

just my 2 cents,
Chriss
Title: Re: Reorganizing development for scalability
Post by: hari on October 11, 2007, 10:17:49 am
I agree, that we should discuss the matter of the right repository - espacially if it turns out that SVN does not work as expected. But please, don't discuss everything until it's dead :/

so true..
Title: Re: Reorganizing development for scalability
Post by: danielk on October 12, 2007, 08:03:55 pm

Status report...

I've had some problems contacting Paul and Razvan, but I'm moving ahead with setting up a server with svn + trac. I'll just start the repository using what sources we have. The server I'm setting up for the SVN is an old P3, but it has a gig of RAM and a 1000 GB per month of download bandwidth, this should be enough for SVN. If we want to replicate the kind of nightly build process Pluto uses we'll need a more powerful server for that. I think the server Paul has is sufficient for that purpose and it's probably best to keep SVN on a separate machine anyway. Plus, I don't think the nightly build scripts Pluto uses will work for us, the scripts seem to be pretty hard coded with IP addresses and the like right in the scripts, so we'll need to edit these for our purposes before we can use them.
Title: Re: Reorganizing development for scalability
Post by: chriss on October 15, 2007, 10:53:02 am
Another status report ;)

We have a repository along with trac up and running. Right now, it only cotains the 0704 sources (as shared via BitTorrent). Daniel is working on restoring the LinuxMCE history after the 0704 release, i.e. apply all patches that are needed for the two updates.
In the meantime I'm setting up a mailing list to be used as a development list later.

The repository will go public as soon as we are in sync with the current updates :)
Title: Re: Reorganizing development for scalability
Post by: chewi on October 15, 2007, 11:24:49 am
Awesome !!! Great to hear that...
I'm really looking forward to getting things to work...

So now the question arrises; who will be maintainer of the repository and who will receive commit-rights ?
Will you, chriss, be the liaison in matters of maintaining the source within the svn ?
Will you also move the bugs from the mantis-bug-tracking to your trac or will the bug- and wiki-stuff be deactivated or will we start over with ne bug-tracking in trac or is there someone who will judge the bugs from mantis and dublicate the "real" bugs to the trac-system ?

best regards, Chewi
Title: Re: Reorganizing development for scalability
Post by: DTS on October 15, 2007, 11:40:49 am
Another status report ;)

In the meantime I'm setting up a mailing list to be used as a development list later.


Why are you setting up a mailing list? What are the benefits compared to a forum like this?
Title: Re: Reorganizing development for scalability
Post by: danielk on October 15, 2007, 03:12:47 pm
DTS, the main benefit of the mailing list is more instantaneous communication.

Chewi, for commit rights we'll give these to those that contribute patches to trac. Initially it will be me, Chris and Paul, when we've committed a few patches from someone that person will be asked if they want commit rights and the responsibility that comes with it. Chewi do you want to move tickets over from Mantis? Someone needs to do it...
Title: Re: Reorganizing development for scalability
Post by: chewi on October 15, 2007, 04:08:59 pm
If I said, I'd love to, I guess it would be a lie... ;)
But besides that, I'm working on my thesis and must not get enganged with any further work... Hoping that the svn will not screw the thesis up for me... ;)

So will the bugtracking be trac only or is trac going to be the tracker for "real" bugs ?
Title: Re: Reorganizing development for scalability
Post by: chriss on October 15, 2007, 06:07:13 pm
Chewi, we have not decided where to track bugs yet. I have to admit that I actually don't know Mantis. IMHO it is more useful to have all development related stuff in one place (which obviously would be Trac...). This would also include the developer documentation from the current wiki.
The nice thing about having all this stuff together is to use Trac's real features, like automatically linking between commits, tickets, wiki and so on.

Edit:
Quote
So will the bugtracking be trac only or is trac going to be the tracker for "real" bugs?
Short answer: IMO it should be only Trac. And it should only be real bugs that we are tracking. Feature requests and all other stuff can stay in the forums  ::)
Title: Re: Reorganizing development for scalability
Post by: countzer0 on October 15, 2007, 10:27:28 pm
danielk,

are you the same as the danielk mythtv dev?
Title: Re: Reorganizing development for scalability
Post by: danielk on October 16, 2007, 12:07:43 am
are you the same as the danielk mythtv dev?

Yup. And one of the reasons I'm getting involved is because the MythTV integration needs a little loving.

I also have a drawer full of X10 stuff I haven't unpacked since I moved recently and I think Pluto is a much better option than MisterHouse or my own scary scripts for setting something reasonable up. And since I moved, I've been planning to set up an Asterisk server as well. LinuxMCE is the only thing that promises to get all my hardware to play nicely together. I think it's pretty close, some of the MythTV issues I see will take me 10 or 15 minutes to fix once we have source repository to work with and a build process that will let me test the changes easily.

BTW I've yet to get LinuxMCE to compile from the ground up, but I've gotten some parts of it to compile. One of the things we need to do early on is get a proper build process in place. Right now I'm trying to reconcile the LinuxMCE 0704 code and Pluto's svn. Then I'll set up the public repository and mailing list with Chriss' help, then we'll see where things go...
Title: Re: Reorganizing development for scalability
Post by: eloy on October 16, 2007, 05:53:21 pm
Yup. And one of the reasons I'm getting involved is because the MythTV integration needs a little loving.

Glad you're here Daniel! :-)

Quote
I think it's pretty close, some of the MythTV issues I see will take me 10 or 15 minutes to fix once we have source repository to work with and a build process that will let me test the changes easily.

Could you elaborate on what some of the MythTV integration issues are, as you see them? Just curious...

My own pet peeve is the inability to configure LinuxMCE to use an existing MythTV frontend - I have a frontend that has been recording since September 2004. It's rock solid and my family relies on it heavily. I am not willing to replace that with LinuxMCE, at least not in the foreseeable future.

I did manage to get a media station to use my existing backend but I think it required me to do something from the command line; can't remember the details.

Cheers,

Eloy.-
Title: Re: Reorganizing development for scalability
Post by: darrenmason on October 17, 2007, 01:22:24 am
Daniel,

Have you managed to build the MythTV devices? I imagine this is where the focus of your fixes would be.

I have not tried to build a complete system as most of the changes have been specific to existing devices or new devices so they can be easily built in isolation and testing with any working system. I am guessing you are in this position as well. So you should be able to try your changes without waiting for the source respository/build process. Getting those changes out to everyone else may be more of an issue.

I am interested what the changes you have identified are, or actually more interested in what the actual problems are.
I personally think that there is a lot of issues in how tightly coupled the orbiter has become to some of the media devices and think that this is the cause of a lot of issues, as it makes the code brittle and too hard to do simple changes.
I also think that all teh Media player (XXX_player) devices should be contained by the media director itself and not the onscreen_orbiter - but that is another issue.

Glad to see you are showing an interest and look forward to your fixes.

Regards
Darren
Title: Re: Reorganizing development for scalability
Post by: darrenmason on October 17, 2007, 01:30:08 am
My own pet peeve is the inability to configure LinuxMCE to use an existing MythTV frontend - I have a frontend that has been recording since September 2004. It's rock solid and my family relies on it heavily. I am not willing to replace that with LinuxMCE, at least not in the foreseeable future.

I did manage to get a media station to use my existing backend but I think it required me to do something from the command line; can't remember the details.

Eloy,
Is it really an existing frontend that you want to use - or an existing backend?
I have had no problems using non linuxMCE frontends with the linuxMCE mythTV backend (on the CORE). My desktop runs Ubuntu 7.04 and that has a MythTV frontend that runs perfectly - and has been very stable.

From what I can see, there would also not be too many major issues with using an existing backend from linuxMCE as well (as long as versions matched etc) as the only really tie in that I can see between linuxMCE and MythTV is done on the MediaDirector and the shared file system.

Regards
Darren
Title: Re: Reorganizing development for scalability
Post by: eloy on October 17, 2007, 03:27:00 am
Is it really an existing frontend that you want to use - or an existing backend?

Whoops, sorry - I made a typo in my message two messages above. What I meant is that I'd love to be able to configure LinuxMCE to use my existing *backend*, which has been stable for years. The assumption today is that you'll use the mythbackend on the core.

Quote
From what I can see, there would also not be too many major issues with using an existing backend from linuxMCE as well (as long as versions matched etc) as the only really tie in that I can see between linuxMCE and MythTV is done on the MediaDirector and the shared file system.

Right. I got a media director to use my existing backend. I don't remember exactly what I did but it may have required CLI access, which is not a big deal. I just wish LinuxMCE played a bit better with things you may have running already, like MythTV and Asterisk.

Personally, I think that relying on any specific distribution is not the best way. For example, I run MythTV on Debian. It's nice that the MythTV software doesn't force me to run Fedora, or Ubuntu, or anything. I look forward to the day when I can do "apt-get install linuxmce" on my Debian box and get a working LinuxMCE installation without having to install another distribution, and without having to use versions of Asterisk or MythTV that have been tied to a specific LinuxMCE version. I hope this day comes one day :-)

Cheers,

Eloy.-
Title: Re: Reorganizing development for scalability
Post by: danielk on October 17, 2007, 05:22:02 pm
Have you managed to build the MythTV devices? I imagine this is where the focus of your fixes would be.

I haven't actually tried this yet. I've build the Orbiter and some other modules. For MythTV the first thing I want to do is actually just configuration stuff. When running full screen it shouldn't be using a window frame, when recording NTSC it should record the captions, when recording PAL it should record teletext, it should be compiled with threading support, it should use a decent V-Sync (RTC or OpenGL V-Sync), the video directory should use the XFS filesystem and a daily defrag should be run, autodelete should leave more of the disk available for non-mythtv stuff by default to avoid both fragmentation and because we should assume that the partition contains more than MythTV recordings, we should set up the Linux kernel for A/V (enabling preempt smaller time slices), MythTV should be coordinating the time of the Schedules Direct download, MythTV should be themed to match the LinuxMCE theme, the orbiter OSD and MythTV OSD's are not coordinated, etc. etc.

MythTV is also just not as stable as it should be in the LinuxMCE environment. This will probably take a bit more work to get to the bottom of and fix. It looks like we're pulling some qt 3.3.8 packages, which is known not to work well in multi-threaded environments, but the problem is probably more complicated than that.

My own pet peeve is the inability to configure LinuxMCE to use an existing MythTV frontend - I have a frontend that has been recording since September 2004. It's rock solid and my family relies on it heavily. I am not willing to replace that with LinuxMCE, at least not in the foreseeable future.

I think you must mean an existing backend. This can be done, but is a bit problematic at the moment because of how the listings are downloaded. You basically need to make the LinuxMCE core machine the master backend, and turn your existing master backend into a slave backend. Or you can just keep both master backend running independently and point your frontend to the one you want to watch recordings from. This can be done by making a copy of your ~/.mythtv directory and editing the mysql.txt in that new directory with the DB info for the other master backend; then just run mythfrontend from that directory when you want to connect to the other master backend.

I personally think that there is a lot of issues in how tightly coupled the orbiter has become to some of the media devices and think that this is the cause of a lot of issues, as it makes the code brittle and too hard to do simple changes.
I also think that all teh Media player (XXX_player) devices should be contained by the media director itself and not the onscreen_orbiter - but that is another issue.


Can you give an example?
Title: Re: Reorganizing development for scalability
Post by: wsuetholz on October 20, 2007, 01:19:46 am
Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

On another note..  I would like to see the mythtv frontend be fully supported in the orbiter with the orbiter and rest of the system being more of an addon/plugin to mythtv rather then mythtv being an afterthought to the home automation that Pluto@Home provides.  In other words, I would like the MythTV Backend to basically take over the messaging/device handling portion of the system and basically become the HA/MM core that would allow for whatever as a frontend.  Be it an onscreen orbiter or a remote orbiter.  I wouldn't even mind seeing something like FreeVO or Elisa as a frontend to myth.

Title: Re: Reorganizing development for scalability
Post by: danielk on October 20, 2007, 02:08:54 am
Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

Those were definitively not ignored. We've been talking off-line. He's using a server that is also being used for other things so he can't let other people on it, except in a very limited way, nor is he comfortable using standard ports for various services on that machine so it's not ideal.

I have an SVN/trac/mailman server set up that I'm testing right now. Just give me a few more hours...
Title: Re: Reorganizing development for scalability
Post by: danielk on October 20, 2007, 06:03:21 am

SVN+TRAC: http://svn.charonmedia.org
DEV MAILING LIST: http://www.charonmedia.org

We may want to move this to linuxmce.org later, but this should work for now. I haven't finished setting up trac, but it has some spam prevention enabled, as does the mailing list (first post is moderated.) There are directions for the getting the svn trunk using a tbz download followed by an svn update. Other checkouts are left as an exercise to the reader, the tbz is less than400MB so it saves a lot of time on the checkout of the 1.4GB trunk branch.

There is no build script we can use yet. I've been working on this locally for a few days, but it will take a few more before I get something basic going.
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 20, 2007, 07:14:58 am
wsuetholz: MythTV's backend doesn't begin to approach the functionality offered by Pluto's DCERouter or its connected systems, I think you should take a serious look at the code. As for danielk and the rest of the developers being a bit quiet right now, I can understand this, there is a lot of scrambling going on to set things up. Be patient, or help out.

-Thom
Title: Re: Reorganizing development for scalability
Post by: chriss on October 20, 2007, 10:30:53 am
Daniel,

awesome that you have the server up and running now!

@all,
as Daniel said before - my server is running some other stuff and I had to limit services (e.g. svn over http). Despite of that it's only a virtual server beeing not powerful enough when more people start developing. That's why we decided to wait until Daniel's server is up.
Title: Re: Reorganizing development for scalability
Post by: chriss on October 20, 2007, 11:00:05 am
Daniel,

great job you did with the repository. I just noticed two things, first you tagged a "pluto-lmce-18249" revision. Is this meant to keep in sync with the pluto guys? Are you still working on getting Paul's two updates into the trunk? The second thing I was wondering about are all those commits having a log message starting with a hoizontal line ("---------------...)". Is this also pluto stuff, because it would be easier to track commits in the "Revision Log" if they have a good summary in the first line of the log.

Well, nothing to really worry about. Thanks for your work :)

/Chriss
Title: Re: Reorganizing development for scalability
Post by: danielk on October 20, 2007, 05:13:26 pm
The tag is there to help with any future syncs from pluto's repo, there is also a vendor branch where most of the action happens. (See the subversion documentation on vendor branches).

I haven't heard back from Paul since early last week, the plan was to let him commit his changes to the new tree.

All the commits starting with a horizontal line are pulls from the Pluto repo. I used a couple of scripts to do this. This was before I had trac installed so, I didn't realize that the extra lines would cause this problem, otherwise I would have filtered them out. The comments are still there and available via "svn log". Each commit also contains the pluto revision number it corresponds to. These were applied in order so you can see that there are some missing revisions. This is mostly due to commits to other branches in the same repo, unrelated to LMCE.
Title: Re: Reorganizing development for scalability
Post by: chewi on October 20, 2007, 05:58:26 pm
Greate Work !!!

As to the question on the trac-wiki: I don't think, an automatic import to trac would be a good idea...
Title: Re: Reorganizing development for scalability
Post by: avajon on October 21, 2007, 09:33:14 pm

SVN+TRAC: http://svn.charonmedia.org
DEV MAILING LIST: http://www.charonmedia.org

We may want to move this to linuxmce.org later, but this should work for now. I haven't finished setting up trac, but it has some spam prevention enabled, as does the mailing list (first post is moderated.) There are directions for the getting the svn trunk using a tbz download followed by an svn update. Other checkouts are left as an exercise to the reader, the tbz is less than400MB so it saves a lot of time on the checkout of the 1.4GB trunk branch.

There is no build script we can use yet. I've been working on this locally for a few days, but it will take a few more before I get something basic going.

really cool  ;)
thanks daniel!!!
Title: Re: Reorganizing development for scalability
Post by: wsuetholz on October 22, 2007, 04:28:41 am

Danielk I have seen chriss say several times that he already has SVN and trac setup, why are those messages being ignored by you?

Those were definitively not ignored. We've been talking off-line. He's using a server that is also being used for other things so he can't let other people on it, except in a very limited way, nor is he comfortable using standard ports for various services on that machine so it's not ideal.

I have an SVN/trac/mailman server set up that I'm testing right now. Just give me a few more hours...

Thanks for clearing that up.  It seemed like somebody was offering to help, and had already set up most of it, but was being ignored..  Nothing more needs to be said...
Title: Re: Reorganizing development for scalability
Post by: wsuetholz on October 22, 2007, 04:46:33 am
wsuetholz: MythTV's backend doesn't begin to approach the functionality offered by Pluto's DCERouter or its connected systems, I think you should take a serious look at the code. As for danielk and the rest of the developers being a bit quiet right now, I can understand this, there is a lot of scrambling going on to set things up. Be patient, or help out.

-Thom


I understand that MythTV's backend is doesn't contain the functionallity of DCERouter at this time.  My suggestion was that maybe it should.

My real issue has always been the disconnect between a MythTV Backend setup and the way that the Pluto Media access works.  If you wish to have a MythTV Frontend accessing the media, you need to define all the information for the media once for MythTV and once for Pluto.  OTH I like the additional options allowed by having different storage trees for different users.  Also I find the MythTV addon to the orbiter to be rather clunky... I also realize that this project is still taking baby steps trying to transition to a multi-developer project.  Kudos for the job done so far, it really is quite amazing for one person to have accomplished so much in so little time.  I have been playing with this system off and on for a couple of years, and most of the issues I have with it are inherited from Pluto, and I'm sure will get resolved now that there is some momentum for the LinuxMCE project.

Helping..   :D  Everytime I commit to help with a project like this I get hammered at work...  I will be attempting to get a AMD 64bit version compiled sometime in the near future I did try once already, but that drive is gone.., For right now I've abandoned my LinuxMCE setup in favor of a 64bit MythTV install which was  working for me better then the LinuxMCE install was until the TV's HDMI port died  S-Video stinks.

I explained my reasons for the message in my response to DanielK, it wasn't impatience at all.

Title: Re: Reorganizing development for scalability
Post by: danielk on October 22, 2007, 07:31:57 am
I understand that MythTV's backend is doesn't contain the functionallity of DCERouter at this time.  My suggestion was that maybe it should.

The concept behind MythTV and Pluto are one and the same, to create the "Mythical convergence device" for the home. Consequently there is a lot of overlap in functionality. MythTV is bolted on to Pluto for the PVR functionality, but it actually contains many of the components for which Pluto already has home grown solutions. In many cases these are more mature solutions. Pluto is a lot like Freevo, and in it's architecture is more like a bazaar than the cathedral like MythTV. In their development the roles are reversed which helps explain the strengths and deficiencies of both.

I think what you really want to do is have a choice in what DVD player to use or what movie player to use and get the full functionality of whatever player you choose without needing to reconfigure everything. The good news is that LMCE is open, it is what you do with it. If you write code to translate between the Pluto and MythTV movie databases then they will play nice for you and everyone else.

LMCE can bring the strengths of both these approaches to the same problem together into a unified whole. Paul has done a lot porting the Pluto components to Ubuntu, but there are two problems we need to address right now, stability and integration. And we have to do this without forgetting to apply patches for functionality like Peter's Insteon driver. Part of the stability and integration problem will probably be solved by both the MythTV and Pluto groups, but parts are unique to the LMCE software or unique to the esoteric hardware used out there by LMCE users installing the software on their own PC's.

Step 1 for me is to get LMCE to bootstrap itself. This is the best way to make sure that all the components can be build independently and that all the dependencies can be satisfied on a running LMCE system. (I've run into missing dependencies trying to compile various individual components, I would rather resolve them all and share with everyone else than just resolve my own issues and move on.)
Title: Re: Reorganizing development for scalability
Post by: PeteK on October 22, 2007, 07:40:45 am
Daniel--

I want to add my two bits in and thank you again for helping out with this project.  This is my first crack at helping develop OSS.  Thanks to guys like you, Paul, and that Zaerc dude who must monitor these forums 24/7 , I'm excited about this and I'm guessing more developers are as well.
Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 22, 2007, 07:47:10 am
I am very excited as well. I'm trying hard to get my head around this code-base so that I can build a games plugin (at first mame, but there will be others.)

-Thom
Title: Re: Reorganizing development for scalability
Post by: darrenmason on October 22, 2007, 08:24:56 am
Daniel,

Is there space/capability on the servers that you setup to host the sqlcvs master(server)?

New devices and device templates as well as changes to existing ones are just as crucial to move forward as being able to compile the code base.

Regards
Darren
Title: Re: Reorganizing development for scalability
Post by: danielk on October 22, 2007, 06:58:17 pm
Is there space/capability on the servers that you setup to host the sqlcvs master(server)?
Yes.

New devices and device templates as well as changes to existing ones are just as crucial to move forward as being able to compile the code base.

I don't think it is just as crucial, we can update the DB in the compile and so add devices and device templates. When we get things compiling, I'll look at what it will take to get a sqlcvs working for us.
Title: Re: Reorganizing development for scalability
Post by: wsuetholz on October 22, 2007, 07:00:24 pm
I am very excited as well. I'm trying hard to get my head around this code-base so that I can build a games plugin (at first mame, but there will be others.)

-Thom


Yes, that's one of the reasons I switched back to MythTV..  I have a bunch of old NES games that my kids play using FCEU.  Why write a new plugin when the functionality is there in a MythTV plugin already?
I did try at one point to work with Pluto on how to plug in the MythTV Game plug in into the orbiter but didn't really get a good response.  I wanted the same kind of search screen that they had for videos.

Title: Re: Reorganizing development for scalability
Post by: tschak909 on October 22, 2007, 07:07:33 pm
That is precisely what I am wanting to do, and I'm gonna keep digging through until it makes sense, which is ironic, esp considering that the code is very good... it's flexible as hell... it's just...not...documented worth a fuck... code forensics takes time. :-P

but really, I do think making a player for each type of game you wish to play, with a plugin to manage passing to the appropriate player (using the External Media Identifiers like everything else), will produce something REALLY neat and scalable.. hell, you can even put it on the floorplan! :-)

-Thom
Title: Re: Reorganizing development for scalability
Post by: wsuetholz on October 22, 2007, 07:28:49 pm
I understand that MythTV's backend is doesn't contain the functionallity of DCERouter at this time.  My suggestion was that maybe it should.

The concept behind MythTV and Pluto are one and the same, to create the "Mythical convergence device" for the home. Consequently there is a lot of overlap in functionality. MythTV is bolted on to Pluto for the PVR functionality, but it actually contains many of the components for which Pluto already has home grown solutions. In many cases these are more mature solutions. Pluto is a lot like Freevo, and in it's architecture is more like a bazaar than the cathedral like MythTV. In their development the roles are reversed which helps explain the strengths and deficiencies of both.

I saw a post once that really pointed out the different mindset between MythTV and Pluto@Home(LinuxMCE)..  Lets see if I can paraphrase...

Everything to Pluto@Home is a device of some kind, the Device is King everything else is subordinate to the device.  Whereas MythTV is more concerned with the storage, capture and presentation of media.

I think I got the gist of the comment.  I agree with you that both wish to be the convergence device, they just come at it from two different directions.  LinuxMCE is a Home Automation device that with the addition of MythTV attempts to also be your Media Center device as well.  It's unfortunate that they couldn't have made the orbiter be a better(integrated) MythTV frontend.  In most respects, I really like the orbiter concept (it is of course a device).

My contributions to this conversation, are getting somewhat off topic, so thanks for your effort helping out the project.

Title: Re: Reorganizing development for scalability
Post by: chriss on October 22, 2007, 08:00:50 pm
Thom,

you wrote:
Quote
the code is very good... it's flexible as hell... it's just...not...documented worth a fuck... code forensics takes time. :-P

from scrolling through some parts of the code I totally agree with you. If you're really digging into the code and doing code forensics it would be awesome if you could just add (doxygen) comments with the information you're gathering. This would help the others understanding the code, too   8)

Cheers,
/Chriss
Title: Re: Reorganizing development for scalability
Post by: Zaerc on October 22, 2007, 08:58:19 pm
... and that Zaerc dude who must monitor these forums 24/7 ...
I strongly resemble that comment! ;D
Title: Re: Reorganizing development for scalability
Post by: darrenmason on October 24, 2007, 01:32:06 am
I don't think it is just as crucial, we can update the DB in the compile and so add devices and device templates. When we get things compiling, I'll look at what it will take to get a sqlcvs working for us.
I don't know if I agree with you here. The device configuration effectively equates to code in this system and updating the database only through the DB build scripts does not give an environment conducive to change. And the change is likely to be from the whole user base rather than just developers.

This was an effective feature of PlutoHome. Someone could do the work to add and configure a new device template (say for a new TV or receiver) and other users could create devices of this template within a couple of days, rather than waiting for a release cycle.
On reading through the forums this morning there was at least two explicit comments that a user has created a template that they wish to share but had no mechanism to do that.

Anyway, hopefully it is not too much effort to get running (and then maintain) on the servers that you have setup.

regards
Darren
Title: Re: Reorganizing development for scalability
Post by: PeteK on October 24, 2007, 01:48:59 am
Daniel--

I agree with Darren.  The fact that LinuxMCE is so dependent on Sql requires some sort of CVS control with multiple people working on it.  For another example, I've created several new commands as well as a new template to support Insteon integration.  I'm not sure how to distribute those changes for beta testing without some sort of CVS control of the database.
Title: Re: Reorganizing development for scalability
Post by: danielk on October 24, 2007, 03:12:07 am
I don't see how we can point everyone to the new sqlCVS unless we get the sucker to build...

I don't think sqlCVS is unimportant.

BTW If someone could get it working on their own and then tell me how to get it working on the server this would speed up the process of getting it working...
Title: Re: Reorganizing development for scalability
Post by: PeteK on October 24, 2007, 03:28:24 am
What?  You generously offer to help and we just want more?  I'm shocked, shocked! ;)

I'll try to take a look at it this weekend and see if I can make any headway.

-Pete
Title: Re: Reorganizing development for scalability
Post by: bulek on October 24, 2007, 09:10:24 am
Hi,

I have another proposition. I'm almost certain that Pluto guys will help on that, maybe even according to past cooperation, Paul can make some kind of agreement so they can maintain sqlcvs for a start. I think this is in common interest. Sqlcvs is afterall a Pluto's baby (from Aaron better said) ...

If that fails, at least we could contact Pluto sqlcvs guy to help out to get things started... They will also be using majority of contributions made this way...

HTH,

regards,

Bulek.


Title: Re: Reorganizing development for scalability
Post by: chriss on October 24, 2007, 10:41:53 am
Bulek,

I disagree on that. AFAIK, Pluto is already hosting a public sqlCVS, but they are reviewing every commit before they put it into the public/productive area. I think, we should have our own sqlCVS server to have a lower latency in reacting to new templates and stuff. This would help to increase the dynamic of development.
Of course we should give Pluto the opportunity to use our database or even mirror it to theirs...

just my 2 cents,
Chriss
Title: Re: Reorganizing development for scalability
Post by: danielk on October 24, 2007, 07:14:30 pm
Let's move this discussion to the mailing list...