Letter to the Community: LinuxMCE 0810 - The Cold Honest Reality

[tschak909]

I can't sleep. So i've decided to write down some thoughts for the community to ponder.

LinuxMCE is becoming a suck on my soul, and the fun that once defined it is going away.

There has been a huge shift in the populace of the free software community. This shift has created a vast gulf between developers and users, and the trend is becoming exponentially larger with each passing day.

This is especially a problem with a project such as LinuxMCE, because, there are over 3 million lines of our own code.

Sit back, and let that sink in.

LinuxMCE began as the dream of a man, who had the wish of becoming the ultimate platform on which to build smart home systems. This is a lofty goal, and one that requires the fulfillment of many objectives which must ALL be accomplished in parallel.

It is also worth noting that LinuxMCE's code base was the work of many developers, including the original designer, who were paid very little money, to work for roughly 5 years developing this code base. These people, save one, are no longer involved with the project. They got this code base to where it is today.

 LinuxMCE has many differentiating deliverables:

A multi-targeted synchronized and distributed user interface, in the form of Orbiter, the only one of its kind.
Interfaces to many home automation interfaces and busses
Interfaces to Infra-Red and RS232 and Ethernet Controllability
Media Players
Asterisk and associated wrapper device, which...
talks to a Telecom Plugin to provide the telecom functionality
data storage and acquisition for tabular data in the form of the Datagrid
Security system intefaces
Climate control interfaces
We  build various pieces of the software on Windows, Linux, and Symbian, and soon, MAC OS X.
and so much more...

All of these things must be worked on by a team of people who must work together to produce a single completed system.

We have....roughly 5 people. If that.

Very few of them work on it full time. I do because I haven't had a job in over 6 months.

We have a huge number of people who have installed the system, over 112,000 according to our installation counter, and yet, we can't seem to find enough people who are willing to put in time to help produce a complete system.

You people constantly ask us, “when is 0810 going to be ready?” Probably never, at this rate.

Probably never, unless those of you who have ANY skill set at all, step up and help us.

You probably are saying that you do not have the skill set to help. I'm going to be equally honest with you right now in return. If you do not have the skill to develop the software, you do not have the skill to use the software in its current state.

Yes, I said it.

I said it because it's true. We are still at a point where we need considerable amount of development JUST TO FINISH THE REMAINING 10% OF THE FEATURE SET AND TO DEBUG IT. I am putting that in caps so that those of you reading the bold type method may actually get something out of this thread.

If you guys want an 0810, you'll have to earn it, all of you. The few of us here can't carry the burden of doing all the work on our own.

-Thom

[massabuntu]

Hi Thom, i understand your frustration, and let me really really thanks you guys for all the things you had done, you are doing and hope you will.

Maybe the problem could be in the organization of the development, i think now a regular user thinks at the development like:

" I need something that doesn't work or is not yet implemented, I search in the forum/wiki and ask in the if anyone have solved, and if is not, i think if i can develop this "

Because a "normal user",maybe with some development skill,of any kind, doesn't have a "general picture" of the problems wich can have a Gigantic Platform like MCE.

So maybe you want to reorganize the development structure and create some Teams, that can focus on some problems.

I hope i explain myself in a decent way, because my english...

[NikAmi]

I hope i explain myself in a decent way, because my englih...

I don't mean to to be mean or derisive because I know this was a simple typo, but in the context of the sentence it was hilarious

[qball4]

I think massabuntu hit the nail on the head with having teams. With teams come team leads, which is what I think is really needed to turn interest in development into *actual* development. For example, I'm interested in UI work. If there were a definite person in charge of UI, instead of just hacking it out on my own and asking for help when I need it, there would be someone I could go to and say, "here's my skillset; what would you like me to work on first?" or "what would help this part of LinuxMCE most right now and where should I start?". Now this may seem like hand holding, but I bet there's a whole bunch of potential developers out there that could be motivated by such clearly defined organization.


Here's how I see something like that working: (Thom, I'm using you because A. you're the OP and B. you're the resident Designer expert)
Me: Thom, I hear you're the UI Team Lead.
Thom: Yes, I am. Would you like to join the team? What skills do you have that we could use?
Me: Well, I'm decent with graphic design and I know my way around HADesigner and the pluto_main database.
Me: I really like spider-man so I'm going to make a UI that looks like a spider web. How can I do that?
Thom: Don't be stupid. I need you to make new icons for the Myth PVR screens.
Me: Oh... okay. I'm going to make them blue and red so they remind me of spidey.
Thom: That's nice. First, make them to match the new Basic theme. Maybe the spider-man theme should wait until we're done with this one.
Me: Wow - there sure are a lot of icons to do!
Thom: Why don't you start with screen ### and check in with me when you're done?
Me: This isn't so hard now that I know what direction to go. I might not be doing the whole skin by myself but I feel like I'm helping!


Does this add responsibility to the already overworked devs? Yes, but unfortunately, I just don't see many highly-motivated self-starters chomping at the bit to pitch in. Not only that, but those of us that aren't skilled enough to do what the core devs do but have usable skills nonetheless can start contributing and stop feeling like leeches.

Or maybe that's just me.

:Matt

[massabuntu]

Me: I really like spider-man so I'm going to make a UI that looks like a spider web. How can I do that?
Thom: Don't be stupid. I need you to make new icons for the Myth PVR screens.

ahahahahahahahahahahhahahaha

Does this add responsibility to the already overworked devs?

Yes but add responsability to the junior devs too.

Anyway you center perfectly what i meant!

[wierdbeard65]

Thom,

I was going to post this in response to a couple of other posts (the one about the GSOC and the one about the website re-design) but you have started a very much needed and welcome discussion here.

I don't want to get sycophantic and spend ages expressing my grattitude. You should know by now (all of you!) that those of us starting out are grateful for the work done by the Devs.

I think massabuntu has made some valid points, but I also think (and I've said this before) that the core devs need to take some time off from MCE and concentrate on the website. Let me say why.

In various threads about the website, people have made suggestions about the site's design. These have either been flatly rejected (Hari's response to my suggestion of a wiki plugin) or met by comments from you like "stop TALKING about it and just DO it". I agree - HOW? Who has the neccessary login credentials to the web server to update the main page? I don't. They aren't (and nor should they be) published anywhere.

You like most (all) the developers are not paid for this. I don't know your background or how you aquired your skill-set. but it probably involved hard work and research. Like most people, I'm lazy. I'd love to help, but don't want to put in a huge amount of effort up-front (I'm exaggerating!). My background is that I have played with Linux systems (amongst others) and hacked together solutions to real-world problems. I have a smattering of programming languages I am familiiar with (C, C++, Java, JavaScript, SQL, HTML, Basic......) but would not consider myself an expert in any. Perhaps the best description would be "Advanced Beginner". How do I move forward? If someone took the time (and YES I COULD probably do this, but I WON'T, partially due to laziness and partially due to it needing to be someone WITH knowledge, not looking for it!) to add some structure to the wiki and some additional developer resources, then it would pay HUGE dividends. Not only would the foolish questions be reduced, but you may also get additional developers on-board.

What am I looking for. Ok, a list of tutorials (these don't have to be on the site, links to other sites are fine if appropriate) that will show me:-

How to program in C++ (and I don't just mean "Hello World") to the kind of level used by MCE.
How to manually manage Ubuntu (adding drivers, reading logs etc)
How to submit patches / modifications etc. to the system (Yes, IF I take the time to develop this, how will it be made available to others?)
How to put together a development environment and then (and this is the important part) walk-through examples of some development. Could be anything (maybe several anythings). How about creating new device templates? What about mucking around with UIs? What about adding support for new hardware that is laready supported by Linux (is this the same as device templates?)

Trying to get involved, as a new user, is like trying to swim in molasses. PLEASE Thom (and everyone else!) help others who want to become involved and help you!

I'm sure others will pitch in and dispute my list, or add to it. Some of the stuff may already be there, but is difficult to find. I think we need two new pages adding that are available right from the front page.

1) You like the system, how do you get / install it? (New USERS, we have discussed this before)
2) You want to get involved in development, how to set up and use a development rig. (New DEVELOPERS)

Like I said, MOST of this info is already there, so it would be a simple matter of linking to it so people like me can both find it easily and have a coherent overview of the process.

Please don't take anything I have said here as criticism, it isn't meant to be. If the project lost folks like Thom, it would probably never recover.

I'm going to end by echoing Thom's sentiments. It is unreasonable for people to expect MCE completly free. We may not PAY for it, but we should CONTRIBUTE to it. That's what (to me) FOSS is all about.

[gadget]

Hi Thom,

For a project of this size to be managed and developed by so few is a magnificent achievement and deserves applause.

The re-design of the website and training/documentation is definitely a must in moving forward. I like what qball4 and massabuntu  are saying about people telling you what they are capable of, and a web page listing jobs/priorities would help in this regard.

The fact the the system is so huge and does so many things is part of the problem. I believe that a core set of features needs to be decided upon to make the 810 release more maintainable. These core set of features then need to be well documented and present well for new users, which will then bring in more templates. This is my biggest issue that every piece of equipment I have I have had to create my own template for.

I am still in process of building my dev machine ,but will attempt to take on the task of developing the windows orbiters for 810 assuming they need a windows platform to be built on and that I have learn't enough about the backend now.

gadget

[colinjones]

Just a few points....

First, whilst recognising Thom's points and frustrations, I would like to point out that there is definite (albeit slow) traction in gaining new programmers to help out. Over the last few months we have seen a few new names popping up, getting interested and skilling up (merkur2k and fido spring to mind). By their own admission they are well off, but kudos to them, and they have contributed small pieces... the start, perhaps, of good things to come!

Some things that may make contributing more accessible to more people....

We have Trac... but we really need a more accessible list of simple, outstanding tasks/bugs/features, on a wiki page, Trac is more focused on the overall "project" and coming releases, milestones and so on. If there was a simple bullet point list or table in the wiki, people could just pick off small tasks and give them a go. I've looked at Trac so many times trying to find something, but get overwhelmed and unable to find something really small to bite off. A little script editing, conf files, researching, etc. Most seems to require a full dev environment, understanding of the builder, detailed DCE knowledge, C++, etc... This point is about accessibility...

I agree, we need to have teams - SMEs as team leaders picking up broad areas, and agreeing to be "touchstones" and mentors to give advice to those who want to give something a go. At the moment it can be very difficult to attract someone's attention for advice... or even knowing who to go to that might have that advice.... broad areas only, say Thom-UI/Orbiters, Hari-HA, CHT-VDR, etc (just examples!)

But, I believe the biggest impediment to getting one's teeth into helping out is simply knowing which area to look it. Getting an understanding of the architecture, each device, their responsibilities, how they hang together, and basically something as simple as - if I play some media, or tell a light to do something, what actually happens, is it a DCE command? Which sends that command, what command is it, where does it get sent to, what events are likely to be triggered as a result and what happens next....

In short, that horrible word "Documentation". This I have no realistic suggestions on how we approach it.... but I fear without it, I see no realistic prospect of how potential helpers can climb the otherwise very/too steep learning curve!!! Discuss?

[MarcoZan]

Just a brief note, I'm at work currently

what may help to move forward the project and to get more developers onboard could be to breakdown the remaining task in a smaller size that can be more manageable also by not-so-highly-skilled devs.
I'm sure there are many around, but probably they are scared about the HUGE amount of code they are supposed to study in order to get autonomous and be fully helpful, or simply can't afford to do it in a reasonable amount of time.

With a set of small and defined task, with a defined target and some advice on how to get started probably more guys would jump in.

To those that may already be preparing their flames, this is NOT a way to say "if you do not have devs, that's your fault", but is a way to say "probably there are not much skilled guys around this project, so probably it's better to go with plan B and nurse some novices".

Just my 0,02
Marco

[Dale_K]

In short, that horrible word "Documentation". This I have no realistic suggestions on how we approach it.... but I fear without it, I see no realistic prospect of how potential helpers can climb the otherwise very/too steep learning curve!!! Discuss?

Amen brother.

I have toyed with some of the dev stuff (setup my own dev environment, even made a couple of wiki articles) and I'd love to get deeper into the guts of LMCE.  The problem is, I have extremely limited Linux knowledge and almost no programming knowledge at all.

Now, having said that, I love to learn and I'm blessed to be one of those lucky people that learn something the first time around.  To that end I've been learning C++ specifically to enable me to work on LMCE.  But, as others have pointed out, I simply don't understand the guts of LMCE.

There are spatterings of information here and there in the WIKI, but the system is so foreign to me that I don't possess the the ability to be able to assemble it all in my mind.  I'm left somewhat overwhelmed with the "Where the hell do I start?" affliction.

I realize that it seems counterintuitive to suggest that in order to relieve the work load of the devs we need the devs to do something more, but as another poster so aptly put it, I think it will pay dividends.

[tschak909]

I thank you for all your comments, and...yes..you guys are right....

The problem is, i'm stretched thin as it is.

It is also worth noting, that as of right now, i'm the one person besides the original creator of the software (aaron.b) who knows and touches every single part of the system, and even he hasn't touched significant parts of it since 2006.

I need to be able to continue doing that, because I know exactly how this system needs to go together

-Thom

[wierdbeard65]

The problem is, i'm stretched thin as it is.

We want to help, we honestly do! Please show us how.

It is also worth noting, that as of right now, i'm the one person besides the original creator of the software (aaron.b) who knows and touches every single part of the system, and even he hasn't touched significant parts of it since 2006.

I need to be able to continue doing that, because I know exactly how this system needs to go together

Another reason why it would be a disaster for the project if you left. Also, and seriously, what would happen if something happened to you? You really do need a brain-dump onto the Wiki to document all this stuff.

I couldn't help spotting this posting from a few years back that has just been added to. http://forum.linuxmce.org/index.php?topic=3188.0;topicseen I'm not making fun, but try to remember how you felt when you posted that originally. The frustration with the lack of documentation. Add in a lower general level of developer knowledge (from me, at any rate) and you will understand why so many of us have responded the way we have.

If you could simply take a few days off development to describe, beginning to end, how to start with a pile of left-over junk and an internet connection and turn it into a development environment with a couple of example plugins / templates / whatever created and uploaded, then I'll join the effort the day after it appears (and I'd bet a lot of others would too). I can't promise great things or to spend the kind of time on it that you do, but I WILL become involved. (BTW, I have little to no money and almost as much time to throw at the problem, but I DO have enthusiasm! )

[skeptic]

Serious question - is LinuxMCE too complicated or does it simply suffer from lack of documentation?  Take for example the HADesigner.  It's the only thing I've looked at from a pseudo-development point of view.  Until life-stuff pulled me away I was going to make a serious effort at learning it and the UI setup.  What I saw was a huge complicated program with tons of "stuff" that needs to be set just right.  Ignoring the Designer program itself which, as I understand it, was a quick hack to get something useful and not intended to be the ultimate design tool, it gave me a brief idea at how complicated the underlying system must be.

Just about every large complicated system goes through the occasional or rare re-write.  I know I'm speaking from complete ignorance here, but would it make sense to do this with LinuxMCE?  Manpower would be a huge issue I know, but I really think if LinuxMCE was more modular with some kind of plug-in system it would be easier to develop for and allow easier addition of functionality.  Instead of one huge install, perhaps you would install a base system.  If you want zwave support, you then install the zwave or a more generic home automation package.  Same for media, asterisk, MAME, etc. 


edit:  I almost forgot - Big thanks to you Thom for the fantastic work you've done.

[tschak909]

No. There will not be a rewrite. The architecture is where it needs to be. If you press me, I really will give the long answer.

-Thom

[skeptic]

No. There will not be a rewrite. The architecture is where it needs to be. If you press me, I really will give the long answer.

-Thom

I think a number of us would like the long answer.  To say LMCE will never get a rewrite, never support some kind of plug-in or modular system seems like a mistake.  Perhaps rewrite is a bit too general of a term, my suggestion is to rewrite/rework/extend/whatever parts of the code to allow flexibility.  Call it a rewrite, rework, extension, upgrade, whatever.  In the end you are unhappy because not enough devs are helping and potential devs are unhappy because the system as it is today is too large, complex, and insufficiently documented for people otherwise willing to lend a hand.