LinuxMCE Forums

General => Developers => Topic started by: Matthew on January 15, 2008, 06:49:29 am

Title: UI3 Discussion
Post by: Matthew on January 15, 2008, 06:49:29 am
Aaron B (Pluto CEO) posted a long examination of his thoughts about the next version of the Pluto/LMCE GUI, that he calls "UI3" (not the UI2/alpha-blended that people sometimes call UI3), in the wiki article "UI3" (http://wiki.linuxmce.org/index.php/UI3). We can discuss it in this forum topic. Here's the current version of the article:
Quote from: aaron.b
UI3 (http://wiki.linuxmce.org/index.php?title=UI3&action=history)
Background

Macromedia Flash has become the defacto standard for creating user interface's for the web, with a rich development environment to create an appealing UI, and an efficient player to do the rendering with lot's of eye candy and visual effects. Media centric applications with 10' user interfaces ("MCE apps") have the same needs, but there really does not exist any software that can do this for desktop applications, like Flash does for web applications.

A few MCE apps try to use Flash for their user interfaces. But this does not work very well since this isn't Flash's intended purpose. For example, the UI in a MCE app cannot be stand alone and isolated, rather it must be a UI that floats on top of whatever media the MCE app is delivering. The Flash player is not able to blend it's rendered output on top of the media application, like Xine or MythTV. So although you could create a really attractive EPG program guide in Flash, you cannot blend that guide on top of the live TV in MythTV.

Therefore, nearly all the MCE applications, like Xine, Myth, Windows MCE, etc., typically code their own UI's. This is inefficient; it is like re-inventing the wheel each time. And, most of the time, the developers are more focused on their core application, not the UI, and cannot create a rich UI engine. For example, Xine, MythTV, Mplayer, Freevo, etc., all have their own UI's, but none are rich with eye candy and visual effects. In general, only Microsoft has been able to develop a really eye-candy rich UI with lots of visual effects, and most other media center applications have fairly boring, static UI's, even if the core media application itself is rich. For example, while MythTV developers began implementing a rich OpenGL UI engine over two years ago, this is still only used for some of the menu system and had not been extended to the video overlay or plug-ins, and because there is no UI editor the capabilities of it's graphics engine go largely unused.

The goal, therefore, is to create (1) a UI player, like the Flash player, which is efficient and can render high-res, 3d, user interfaces with lots of eye candy on a variety of platforms (x86 Linux/Windows, PDA's, cell phones, etc.), and (2) a comprehensive development environment, like the one Macromedia developed for Flash, that is well documented and not overly technical so creative designers can use it to build innovate UI's.

The intention is that this can be used to create UI's for a variety of MCE apps, like MythTV, MPlayer, etc., and that unlike Flash which is designed to run as an isolated application, this UI will be designed from the ground up to bolt on top of the core MCE app. In other words, you can create a beautiful EPG, like you could with Flash, but it can be blended on top of MythTV's live tv feed and used to control MythTV. The UI tool should be modular and flexible so graphics designers can create interesting UI's without needing the help of the coders of the core MCE app. For example, a designer could create, say, an innovate 3d TV program guide on a rotating sphere and add it to MythTV without needing the MythTV developers to write new code. Having a unified UI will also allow us to address some of the vexing issues, like text entry using remote controls of various types, with new UI widgets which only need to be learned once for the entire class of 10' applications.

It is understood that this is a big, complicated task which hasn't really been done before. However, we feel it is an important gap that needs to be filled. The first thing most end users notice is the eye candy in the UI and how pretty it is, and only Apple and Windows Vista have had the resources to create such UI's. This will allow other MCE apps to have UI's that are just as catchy and the developers can focus on the core functionality.

Target user / financial model

Naturally this tool would be an asset to the FOSS MCE apps, like MythTV. However, there are many commercial applications as well. For example, only TiVo and Moxi have created rich UI's for their PVR's. There are tens of millions of PVR's out there from Motorola and Scientific Atlantic distributed by the cable operators, but they cannot justify the huge expense of creating a rich UI, so in general they have ugly, plain, static UI's. If there were an inexpensive, easy way to add a beautiful UI to these PVR's it would be worth it. Also, most TV's have a user interface for on screen menu's that is static and boring even though the TV's themselves have graphics processing chips in them that have the power to render a rich UI. There are numerous DMA's and set top boxes as well that could benefit from a rich UI as well. Mobile phones and PDA's also have media players and mini MCE apps.

In general these UI's are powered by a few single chip solutions, such as Sigma Designs, Broadcom and Marvell, and specialty chips for TV's, like AMD's Xillion. Macromedia Flash's player has been ported to many of these platforms so you can, for example, navigate a flash UI on a mobile phone. But, again, since Flash doesn't work well with MCE apps, in general these MCE apps all have crude, basic UI's, or they look to specialized middle-ware providers like Syabase, Mediabolic, Digion, etc., to create a UI for them.

The UI player will first be developed for X86, probably Linux and Windows in parallel. The software would be developed in such a way that the UI itself was subject to the GPL license, so there is an immediate potential to license the UI player for some commercial MCE apps. There's concern, for example, that SageTV will have a hard time remaining viable since it just doesn't have the pretty UI that Windows MCE does. They would be a potential customer to license the UI player outside the GPL since they would not want to release their UI under the terms of the GPL.

Over time the UI player could be ported to other platforms, like the Sigma, Broadcom, etc. Then the makers of set top boxes could use the UI generator to build rich UI's which could be played back on those set top boxes. By being cross-platform, we would have a strong advantage because the UI developer could target any platform for which the player existed, like Flash developers do now. Also, by being open source, the barrier of entry would be quite low. Anybody could start using the tool set, build it upon it if needed to add new visual effects, and only need to license it when they wanted to deploy a commercial application. And there would not be a concern of being locked into a closed, proprietary solution like the existing middle ware providers.

This relates to Pluto's core business of licensing an interoperability platform. Once a company adopted the use of the UI, Pluto would be able to offer a variety of modules, like lighting control, telephony, etc., that could easily 'bolt on' to the UI delivering a lot of extra functionality without writing any specialized code. If SageTV, for example, used the UI as their front end, Pluto could then offer lighting control modules, some to view security cameras, make phone calls, etc. which could be added to the SageTV package without them writing any new code. Pluto will be willing to sponsor the development of this new UI in exchange for receiving a license allowing Pluto to commercially license the UI software outside the GPL.

If QT is providing the underlying graphics library and hardware abstraction, then this benefits Trolltech as well because every commercial user who didn't want to release the UI under the GPL would also need to obtain a license to QT. The hope is that QT would also have an interest in furthering the development of this software.

How this relates to LinuxMCE

The UI in LinuxMCE, Orbiter, was developed to accomplish this same task. It has succeeded in some ways. For example, LinuxMCE's UI can be played on both Linux and Windows pc's, web pads, pda's, mobile phones, desktop phones, and web browsers. All those platforms are rendering the same UI and same set of screens created with a single design tool, called Designer. The problem is that Orbiter and Designer were created nearly 5 years ago, and were not very well designed. They are still quite crude. Designer is nearly impossible to use and does not appeal to graphic designers. The limited eye candy and visual effects in LinuxMCE, like the 3d cube media browser, are hardcoded into the player, Orbiter, rather than being part of the design as they should be. So, Orbiter and Designer are not general purpose enough to be useful to other MCE apps.

Skill sets needed

Here are 4 groups we need:

  • First we need people who really understand how to design a great UI. These could be Flash designers who have created rich UI's before. They need to outline what the new UI software must do to appeal to the designers that will be building the UI's. That is the most essential thing since, if designers are creating, say beautiful, rich TV program guides with this software, it will be easy to approach the commercial PVR makers and show them why they need to use this UI in their product. So this team is more composed of graphic designers, not C++ coders.
  • Next we need to know how to develop the UI design tool. There will be a lot of overlap with #1 since the people in #1 will be the users of the code written by this group. The UI design tool will not need to be cross platform. Windows/Linux x86 is acceptable. But it will have to be logical and similar to other design tools, like Macromedia's flash development tools, so designers can start using it without a steep learning curve.
  • Next we need people who create MCE apps and know what the player must do to integrate well with those apps. For example, what are the low level C-language hooks the player must have to handle alpha blending the UI on top of MythTV. What is the socket-based control needed to allow the player and MythTV to seamlessly communicate as though the player were developed as part of MythTV, so the user is not aware that the UI is in fact a separate piece of software.
  • The player needs to be highly efficient using a lot of the tricks that the video game industry uses to create high res 3d effects with smooth animation. For this we need some people who understand the Windows Direct X architecture, as well as the OpenGL fragment programs used by Linux and OS X, and some of the single chip solutions, so the player can be designed in such a way that as much code as possible is shared across all platforms, yet the player takes advantage of the unique hardware acceleration tricks each platform offers to be lean and efficient.

Proposed next steps

Some of the people needed will be in the Bay Area anyway for the KDE/Google event on Jan 18. We propose having a mini developers conference on Jan 19th. Pluto has agreed to sponsor this and provide airfare and accomodations to some key people, as well as pay change fees for some already at the KDE/Google event who want to stay an extra day.

The goal is to get representative people from the four skill sets to brainstorm some of the general concepts and come up with an organizational structure and delegate responsibilities. Also, key developers could be chosen to be sponsored full time.

We would also like to finalize an agreement with Trolltech so that if a committment is made to use QT as the underlying library for this new UI, it can be determined how Trolltech and Pluto will work together on the commercial licensing, and what resources Trolltech is willing to commit to the project.
Title: Re: UI3 Discussion
Post by: blackoper on January 15, 2008, 07:02:05 am
wow that's a pretty thought out strategy. Has then been posted to places like the mythtv mailing lists?
Title: Re: UI3 Discussion
Post by: Matthew on January 15, 2008, 07:10:05 am
I'm very excited about the prospect of a new & improved UI3. Especially the prospect of a single GUI across all devices and modes (eg. X app and browser). And a themeable (whether stylesheets and/or skins) GUI paradigm that is also tiered properly into layers encapsulating their proper independent/interdependent functions.

My first question about this proposal is just why is Adobe Flash so unsuited to be the platform for UI3? If its proprietary nature is a problem, there's OpenLaszlo (http://en.wikipedia.org/wiki/OpenLaszlo). Which also now has the benefit of generating not only both SWF-compatible code and DHTML from the same source project, but is also being aggressively distributed on set-top boxes, PVRs and mobile phones (including iPhone) in a joint venture with Sun, making OpenLaszlo part of JME (so no additional SWF player is needed). JME is also the basis for the BD-J (http://en.wikipedia.org/wiki/BD-J) that is part of Blu-Ray (BD menu systems are all in BD-J) and also DVB.

OpenLaszlo can run on all of those devices, and present effectively the same GUI in either standalone apps or a web browser, and is inherently networked for distributed execution. It's already got not only the execution environment (including Adobe's Flash player) but also design/development environments, existing code, and - maybe best of all - an existing developer community.

If OpenLaszlo doesn't quite serve (eg. 3D and maybe alpha-rendering are lacking), its source is open, so it can be revised. If UI3 is going to undertake some serious development, why reinvent the wheel, rather than just extend something like OpenLaszlo that's already mostly there?

Alternately, if OpenLaszlo, like Flash, is just too big and complex, what about SVG? SVG with ECMAScript can make lightweight but featureful GUIs that run in browsers and can be encapsulated in runtimes (Java or native x86/etc binaries). Again, why reinvent the wheel, rather than just build a better car or road for it?

(Note: I have no vested interest in OpenLaszlo or related companies, I just think it's the best fit for these requirements.)
Title: Re: UI3 Discussion
Post by: tschak909 on January 15, 2008, 07:16:11 am
my only concern is to make sure that the traditional orbiter devices are supported. This includes:

* Cisco 7970
* Nokia 770/N800
* Windows Webpads
* Symbian/Windows Smartphones

because, I do _NOT_ want to lose retargetability to other devices, in the process of moving across platforms. I will scream bloody murder, if that happens.

-Thom
Title: Re: UI3 Discussion
Post by: Matthew on January 15, 2008, 07:25:15 am
my only concern is to make sure that the traditional orbiter devices are supported. This includes:

* Cisco 7970
* Nokia 770/N800
* Windows Webpads
* Symbian/Windows Smartphones

because, I do _NOT_ want to lose retargetability to other devices, in the process of moving across platforms. I will scream bloody murder, if that happens.

I agree. I was attracted to LMCE partly because the Orbiter already runs on such a variety of devices. Including the Cisco 7970 - seeing that in the demo video is probably what clinched it for me (along with seeing the XL-1B jukebox supported). OpenLaszlo/SWF doesn't currently run on the 7970 (how about those other devices you mentioned?) But OpenLaszlo is XML, and the 7970 does support XML apps, so maybe porting an OpenLaszlo GUI to 7970 XML is an option. Perhaps even a faster/better option than rewriting a whole new UI3 Orbiter for it from scratch.
Title: Re: UI3 Discussion
Post by: totallymaxed on January 15, 2008, 12:09:44 pm
I'm very excited about the prospect of a new & improved UI3. Especially the prospect of a single GUI across all devices and modes (eg. X app and browser). And a themeable (whether stylesheets and/or skins) GUI paradigm that is also tiered properly into layers encapsulating their proper independent/interdependent functions.

My first question about this proposal is just why is Adobe Flash so unsuited to be the platform for UI3? If its proprietary nature is a problem, there's OpenLaszlo (http://en.wikipedia.org/wiki/OpenLaszlo). Which also now has the benefit of generating not only both SWF-compatible code and DHTML from the same source project, but is also being aggressively distributed on set-top boxes, PVRs and mobile phones (including iPhone) in a joint venture with Sun, making OpenLaszlo part of JME (so no additional SWF player is needed). JME is also the basis for the BD-J (http://en.wikipedia.org/wiki/BD-J) that is part of Blu-Ray (BD menu systems are all in BD-J) and also DVB.

OpenLaszlo can run on all of those devices, and present effectively the same GUI in either standalone apps or a web browser, and is inherently networked for distributed execution. It's already got not only the execution environment (including Adobe's Flash player) but also design/development environments, existing code, and - maybe best of all - an existing developer community.

If OpenLaszlo doesn't quite serve (eg. 3D and maybe alpha-rendering are lacking), its source is open, so it can be revised. If UI3 is going to undertake some serious development, why reinvent the wheel, rather than just extend something like OpenLaszlo that's already mostly there?

Alternately, if OpenLaszlo, like Flash, is just too big and complex, what about SVG? SVG with ECMAScript can make lightweight but featureful GUIs that run in browsers and can be encapsulated in runtimes (Java or native x86/etc binaries). Again, why reinvent the wheel, rather than just build a better car or road for it?

(Note: I have no vested interest in OpenLaszlo or related companies, I just think it's the best fit for these requirements.)

I tend to agree with you on the issue of not re-inventing the wheel. But the issue I think in technical terms is that if you want to have effects like transparency in the UI so that UI objects can blend correctly with the live picture from TV or some other video source then Flash (or any other stand alone player) will not know how to do this. The problem therefore is at the Player or runtime engine end of the solution... the runtime engine would need to know how to composite itself with transparency and other effects in the way described... Flash player does not do that and is closed source so we cant make it do that either. As OpenLaszlo relies on the Flash player at the target machine for its runtime engine this causes the problem that Aaron describes.

Title: Re: UI3 Discussion
Post by: craasch on January 15, 2008, 05:17:22 pm
my only concern is to make sure that the traditional orbiter devices are supported. This includes:

* Cisco 7970
* Nokia 770/N800
* Windows Webpads
* Symbian/Windows Smartphones

because, I do _NOT_ want to lose retargetability to other devices, in the process of moving across platforms. I will scream bloody murder, if that happens.

-Thom


A balance would be good though, to support the existing devices (or a reasonable subset of existing devices) but eventually new features/ desires/ capabilities will simply eclipse the capabilities of what we have now.  I would hate to see future development restricted to support devices that have are out of production.  Don't get me wrong, I have no desire to purchase new equipment, In fact I'm trying to see if it is possible to create a DCE template for my hauppauge MVP running mvpmc software.

I wonder if the discussion could start with looking at the layout description language (probably xml but not absolute) and then choosing the appropriate tools to use for the different devices.  I realize that this could end up fragmenting development, but it doesn't need to.

I like the idea of open laszlo (or similar), and I would also suggest maybe the use of xforms or similar for layout as there are already reference implementations across many platforms and languages.  I think the use of an international standard like this could entice more corporate development / sponsorship as well.  I think the CORE will stay a restricted development environment, but bringing in something like webservices and xml based layouts will allow for some exciting interfaces.  I could easily see adding something like Red5 with laszlo and having game consoles like the PS2/3, Wii, nintendo DS as orbiters, or a number of flashlite 3 enabled devices like the Chumby

-Chris
Title: Re: UI3 Discussion
Post by: Matthew on January 15, 2008, 05:32:50 pm
I tend to agree with you on the issue of not re-inventing the wheel. But the issue I think in technical terms is that if you want to have effects like transparency in the UI so that UI objects can blend correctly with the live picture from TV or some other video source then Flash (or any other stand alone player) will not know how to do this. The problem therefore is at the Player or runtime engine end of the solution... the runtime engine would need to know how to composite itself with transparency and other effects in the way described... Flash player does not do that and is closed source so we cant make it do that either. As OpenLaszlo relies on the Flash player at the target machine for its runtime engine this causes the problem that Aaron describes.

Flash (SWF) content does not depend exclusively on the Adobe Flash player. There is also GNASH (http://en.wikipedia.org/wiki/Gnash), which is GPL and in active development - and it's not the only alternative (http://en.wikipedia.org/wiki/Gnash#See_also). It would most probably be less effort to extend GNASH to use compositing etc than to rewrite from scratch. Aside from the lesser costs (especially time to market), the benefits of using SWF would include the huge library of existing reusable content, including lots of recent GUI components, consistent integration with video like YouTube (and lots of other Net video) players, and probably most important the existing developer community and tools for SWF. Plus, SWF is already an open format (despite Adobe's dominance), not a new format that we'd have to introduce as yet another GUI format standard.
Title: Re: UI3 Discussion
Post by: tkmedia on January 15, 2008, 05:42:07 pm
http://www.rebol.org/cgi-bin/cgiwrap/rebol/art-display-article.r?article=jgp446wq#section-1.2 (http://www.rebol.org/cgi-bin/cgiwrap/rebol/art-display-article.r?article=jgp446wq#section-1.2)

Excerpts from above.

Quote
In fact, there's absolutely no simpler solution for cross-platform GUI creation, anywhere (you can create complete, functional graphic applications with 1 line of code).

Quote
Rebol's multimedia functions are easy to use, and it's built-in graphic tools are powerful enough to handle 3D animation without any external requirements (no OpenGL, DirectX, etc. needed)

Don't know if I am way off base,  but maybe add this to the brain storm.
Title: Re: UI3 Discussion
Post by: totallymaxed on January 15, 2008, 06:20:54 pm
I tend to agree with you on the issue of not re-inventing the wheel. But the issue I think in technical terms is that if you want to have effects like transparency in the UI so that UI objects can blend correctly with the live picture from TV or some other video source then Flash (or any other stand alone player) will not know how to do this. The problem therefore is at the Player or runtime engine end of the solution... the runtime engine would need to know how to composite itself with transparency and other effects in the way described... Flash player does not do that and is closed source so we cant make it do that either. As OpenLaszlo relies on the Flash player at the target machine for its runtime engine this causes the problem that Aaron describes.

Flash (SWF) content does not depend exclusively on the Adobe Flash player. There is also GNASH (http://en.wikipedia.org/wiki/Gnash), which is GPL and in active development - and it's not the only alternative (http://en.wikipedia.org/wiki/Gnash#See_also). It would most probably be less effort to extend GNASH to use compositing etc than to rewrite from scratch. Aside from the lesser costs (especially time to market), the benefits of using SWF would include the huge library of existing reusable content, including lots of recent GUI components, consistent integration with video like YouTube (and lots of other Net video) players, and probably most important the existing developer community and tools for SWF. Plus, SWF is already an open format (despite Adobe's dominance), not a new format that we'd have to introduce as yet another GUI format standard.

Hmmm... well at some point in the future I am sure that Gnash will be a viable runtime for UI3 but as you can easily see from the Gnash site http://www.gnashdev.org/?q=node/25#release this time has not come yet. They have no stable version yet... they are still in alpha mode and they still have significant gaps in the repertoire of content and formats supported let alone the fact that Gnash is not currently supported on  many other OS's. We cant build a strategy based on technology that may or may not meet our requirements at some point in the future.

Sorry the above sounds all negative... but I am just trying to look at this with deliverable working code in a reasonable period of time as my criteria. I have no axe to grind here at all... but we need a stable runtime engine for this to be viable in Flash otherwise we still hit all the issues that Aaron alludes to in his Wiki article.
Title: Re: UI3 Discussion
Post by: colinjones on January 15, 2008, 06:42:59 pm
I understand the concerns about not loosing the "retargetability" to other (existing working) devices - but isn't that the whole point of the different UIs being selectable by Orbiter? Adding UI3, means that UI2 and UI1 are still available for other device Orbiters - I don't think that anybody was really expecting to have a new UI3 that works with all the cool alpha trans, 3d and animation effects on a mobile phone, or web orbiter, were they? Goodness, UI2 only works with AB on a specific set of nVidia chipsets reliably!

Surely that UI1 and 2 (AB and non-AB) will continue to be available for those orbiters, whilst we don't hamstring the capabilities of UI3, and really go for the "eye-candy"?

I understand the concern about re-inventing - what aaron.b described seems like the ideal world, but sounds like a hell of a lot of coding to get there from the ground up. At the same time, we probably don't want to limit or hack-in something less ideal for the sake of expediency. Does anybody have a feel for how long the plan as described by Aaron would take to implement? Does Aaron still read these forums? Haven't seen much from him recently - you out there aaron.b? :)
Title: Re: UI3 Discussion
Post by: tschak909 on January 15, 2008, 06:54:11 pm
well the difference also lies in the proposed TOOLS to make such UI's.. none of you have had to deal with the hell that is HADesigner... I have... (excluding Totallymaxed)...and it's the only thing that can do UI1 and UI2 at the moment.

-Thom
Title: Re: UI3 Discussion
Post by: colinjones on January 15, 2008, 06:58:40 pm
Very true - I assume if it goes the way aaron.b outlines, then one of the design deliverable bullet-points would have to be " * develop descent design tools"!
Title: Re: UI3 Discussion
Post by: Matthew on January 15, 2008, 07:48:32 pm
Hmmm... well at some point in the future I am sure that Gnash will be a viable runtime for UI3 but as you can easily see from the Gnash site http://www.gnashdev.org/?q=node/25#release this time has not come yet. They have no stable version yet... they are still in alpha mode and they still have significant gaps in the repertoire of content and formats supported let alone the fact that Gnash is not currently supported on  many other OS's. We cant build a strategy based on technology that may or may not meet our requirements at some point in the future.

Sorry the above sounds all negative... but I am just trying to look at this with deliverable working code in a reasonable period of time as my criteria. I have no axe to grind here at all... but we need a stable runtime engine for this to be viable in Flash otherwise we still hit all the issues that Aaron alludes to in his Wiki article.

Compare the effort necessary to take a alpha GNASH (or some other GPL SWF player) to where it needs to be for UI3 (only, not necessarily generic SWF support at first), against the effort necessary to write UI3 either from scratch, or from some other codebase even more rudimentary than Gnash's current incarnation. I'm not saying we shouldn't undertake basic development to get UI3, but just that we should start from the closest point to where we can get the most benefit. Even if TrollTech is going to help, I think we'd all prefer to spend the time just adding their features to something that already does the basics, rather than spending that time just getting to the basics and then working more from there.

Also, looking at the Gnash project, it doesn't seem to be very unstable (http://wiki.gnashdev.org/wiki/index.php/Testcases#Critical_bugs_in_cvs_head), another release is slated for the end of this month, and its roadmap (http://wiki.gnashdev.org/wiki/index.php/Release_0_8_2) looks very straightforward to release. And it's still committed to running on embedded devices well, better than is Adobe's Flash, and uses OpenGL where available (or Cairo where not), which might mean it's "composite ready". Again, it looks like it's exactly in the same vein as the UI3 requirements, but already largely complete, and with a developer community (and large library) attached.
Title: Re: UI3 Discussion
Post by: danielk on January 15, 2008, 09:30:58 pm
On the developer tools, I've talked with Aaron B (Pluto) and Aaron Seigo (TrollTech KDE) and the plan is very much to have a high end design tool to make theming easier.

As for the underlying toolkit, this would probably be based on libplasma. Plasma is a KDE library initially written to support the next generation desktop for KDE, but which has since been recognized as a portable library with a good design for writing sexy widgets. Depending on the software design chosen it may even be possible to place KDE desktop widgets in the new UI and have those widgets be aware of the 10' UI layout and font size restrictions. Theming would include not just be adjusting the look but also allow control over how the UI elements behave and even allow adding or subtracting applications within that layout without any need for a programmer to be involved.
Title: Re: UI3 Discussion
Post by: tschak909 on January 15, 2008, 09:54:34 pm
Yes.. this has been discussed a bit on the list, but I paraphrase here as well... we all know and understand that this is for UI3..and some affordance needs to be investigated for the old orbiter UI, either in the form of a replacement Orbiter that runs on the small devices, or a separation of dev tools so that traditional orbiter designs can be manipulated, alongside the Plasma layouts in UI3.

-Thom
Title: Re: UI3 Discussion
Post by: Matthew on January 15, 2008, 10:08:07 pm
On the developer tools, I've talked with Aaron B (Pluto) and Aaron Seigo (TrollTech KDE) and the plan is very much to have a high end design tool to make theming easier.

As for the underlying toolkit, this would probably be based on libplasma. Plasma is a KDE library initially written to support the next generation desktop for KDE, but which has since been recognized as a portable library with a good design for writing sexy widgets. Depending on the software design chosen it may even be possible to place KDE desktop widgets in the new UI and have those widgets be aware of the 10' UI layout and font size restrictions. Theming would include not just be adjusting the look but also allow control over how the UI elements behave and even allow adding or subtracting applications within that layout without any need for a programmer to be involved.

That strategy would seem to make LMCE dependent on KDE in a way that is almost negligible now. Does it really make the most sense to make MDs dependent on an entire desktop when it's supposed to be a vastly simpler paradigm than a desktop, and supposed to run on little devices, including mobile phones? And again, why reinvent the wheel with new high end design tools when there are already so many, and so many people using them, to create SWF?
Title: Re: UI3 Discussion
Post by: tschak909 on January 15, 2008, 10:15:26 pm
Plasma is desktop environment independent.

-Thom
Title: Re: UI3 Discussion
Post by: Ender on January 16, 2008, 03:43:04 pm
Hi everybody,

What about using Gnash? http://www.gnu.org/software/gnash/ (http://www.gnu.org/software/gnash/) Seems to have almost everything we need. ;)
Title: Re: UI3 Discussion
Post by: totallymaxed on January 16, 2008, 04:00:35 pm
Hi everybody,

What about using Gnash? http://www.gnu.org/software/gnash/ (http://www.gnu.org/software/gnash/) Seems to have almost everything we need. ;)

GNASH has already been suggested earlier in the thread ;-)

It looks good... but my take was it might be a little early in its dev cycle... but on the other hand maybe not
Title: Re: UI3 Discussion
Post by: Matthew on January 16, 2008, 06:11:37 pm
Plasma is desktop environment independent.

OK, so compare Plasma to, say, GNASH (which is not the only alternative, just the first one that I found which seems right).

GNASH is already just about to release, includes evidently all the features UI3 requires, comes with its own developer community, and uses SWF format. SWF of course has a huge content developer community as well as mature tools they use, open source alternative tools, is a default format for practically every desktop and browser in the world, and lots of existing content. SWF/GNASH also already plays video in FLV, which is what YouTube and most Internet video sites like it use. Adobe's pushing FLV (as is Google with YouTube), so it will be a primary "Internet TV" format for years, which will mean LMCE will have to support it somehow anyway. And GNASH uses ffmpeg for that video, which is already part of LMCE (though LMCE would probably need to wrap ffmpeg in a new UI module to integrate it fully), showing that LMCE already has consistency with GNASH, and common code reduces LMCE's (considerable) bloat. GNASH also runs on embedded/small devices, like really lightweight MDs will be, and mobile phones. The same source code generates either SWF or DHTML.

Plasma has some (new) libraries. It's unsuited to anything but a standalone app on the local machine (ie. no Web browser). There's no design tools, content management, developers, existing content, established techniques. It will take a lot more work to get a UI3 in it than it would applying that work to GNASH.

Maybe this is a false choice. Maybe the way to get the best UI3 the fastest with the best extra perks (community, tools, content) would be to spend the UI3 development time adding Plasma features to GNASH, where GNASH needs it. That would be the best for everyone, including the broader GNASH community (which would help them become contributors, directly or indirectly, to LMCE).

FWIW, substitute any equivalent to GNASH for GNASH in this discussion. As I said, GNASH is just the first alternative that I found that looked better than the Plasma route. There are likely others, though we have to find only one that works.
Title: Re: UI3 Discussion
Post by: aaron.b on January 16, 2008, 09:12:38 pm
I'll divide my comments in 3 parts: 1) the design tool, 2) the player, 3) financial issues.  Both the design tool and the player are *almost* what we need, but, lacking in a couple areas.

--Design tool--

I really would like to find a way to somehow base this on Flash because it will be impossible to reinvent Flash's design suite.  I have never used it, but just from looking at the demo video on Adobe.com, it is clearly a very rich, robust, polished design suite that allows the designer to create gorgeous UI's that do almost anything with sophisticated illustration and 3d tools.  The thought of us saying "Let's rebuild something that's as good as Adobe's flash designer" is daunting; that software was 10 years in the making with hundreds of full time developers.  We just can't make something as elegant as it.  And we'll never be able to reproduce having tens of thousands of developers who know it intimately so they can create cool eye candy effects in our UI.

The one area that Adobe's design tools may be lacking in (I don't know for sure) is the cross-platform stuff.  Pluto's HADesigner is crude as can be, but, it was designed to target lots of platforms.  For example, the 'play' button in UI1 on windows ce pda, and the 'play' button in UI2 on the TV, as well as on the Cisco phone and mobile phone, are actually all the same object.  You just assign different attributes for each of the variations (like object oriented inheritance), so you can make it be animated and round on a more powerful box, and a small static square on a mobile phone.  You can also cluster the 'play/pause/stop' buttons into a 'media playback group', and say that the group itself is also different on a tv vs. a mobile phone (ie object containers).  Then, when you're creating a media remote control, you just drop the 'media playback group' on the screen and whether you're using a mobile phone or a tv it still works.  I'm not sure if Adobe's flash designer is object oriented like that, supporting inheritance and overrides, so you can create, say, a button that looks and behaves differently on various platforms.  I've never seen Flash used like this; the general goal of flash is the opposite: to have a player that renders the flash file exactly the same on all platforms, rather than having one UI that behaves differently on the different platforms.

While this may be a weakness, it's not necessarily a deal breaker since, after all, if Flash designer is not object oriented with inheritence, the person designing the UI could work around this by just creating separate flash files for the mobile phone, the pda, the cisco phone, the TV, etc.  Most designers would probably prefer to create 5 UI's using Adobe's flash designer vs. 1 UI with Pluto's HADesigner because our HADesigner is that bad and it's unlikely we'd have the manpower to create something that rivaled Adobe's flash designer.

The other thing I'm not sure of is how well flash scales.  In general all the flash files I've seen are created for a target resolution (say 800x600).  With Pluto's HADesigner, OrbiterGen scales the UI to any resolution, rotates it (ie some devices are vertical not horizontal), and stretches it (16:9 vs. 3:4).  This would be a big deal breaker if Flash couldn't do this because, although we could tell designers they need to create 5 different UI's for all the target platforms, telling them to create each of those 50 times (5 UI's at 10 different resolutions) would be ridiculous.  If they wanted to change the command the 'play' button sent, rather than doing it in 1 place like you do with HADesigner, you'd have to make the same change to 50 UI's if Flash didn't support scaling/rotating/stretching and didn't support object oriented inheritance with overrides.

--Player--

Here what's basically lacking is the ability to blend 2 different applications together.  Flash is designed to work stand alone, not bolt on top of and blend into something else.  We're looking for a UI that can bolt on top of MythTV, or Xine, or MPlayer, or Firefox (for a 10' web browser), etc.  If, however, we commit to using Adobe's flash designer as a back end, then that means we've committed to the flash file format and we're better off to put resources into adding the alpha blending capabilty to something that already plays back flash files, like Gnash, than to start from scratch reinventing a flash player.

--Financial Model--

Here is where it gets tricky.  Pluto has investors, and when we put in the budget expenses for development, we have to show that there's some way that we can at least make enough revenue to cover the expenses, and ideally make a profit, otherwise it's considered throwing money away.

Normally the way companies do this with FOSS is to offer software under the GPL that, in order to be used commercially, needs to link with non-GPL code, and the company then licenses it outside the GPL for this commercial use.  For example, with MySQL, the client libraries were originally LGPL, but that meant commercial companies could use them in non-GPL apps without paying a license fee.  So, in order to generate revenue, MySQL changed the client libraries to GPL.  Trolltech did the same thing with QT.  Except for cases where the software is offered b2b as part of support contracts (think SuSe, Asterisk, etc.), which doesn't apply to us, licensing the software outside the GPL for commercial use seems to be the standard way that a company can offer software that is open source for the FOSS community, but that they can still make money for with commercial use.

That was our intention with UI3.  We want all the FOSS projects to be able to use it for free under the GPL, since they will release their UI's under the GPL too.  But, since we will be hiring a lot of developers for this, and probably sponsoring several more, our justification to our investors for spending that money is to be able to license the same software outside the GPL, like MySql and Trolltech do.

So if the back end design tool is Adobe Flash Designer, and the front end player is either Adobe or Gnash, we have no way to explain why we should invest a lot of money on development.  If the approach of using Flash Designer + Gnash was close enough to being a workable solution, or if there were enough volunteers to do the work, then it's not an issue; we don't need a revenue model because we won't need to invest a lot of money in it anyway.  However, my gut feeling is that this likely will require a substantial investment because, as an example, MythTV's devs would obviously like to have a UI as pretty as TiVo or Apple or Vista, and if it really was that easy to create a beautiful UI in Flash Designer and bolt it on top of Myth, Xine, MPlayer, etc., then they all would have been doing it already.  Since all the media center apps need pretty eye candy UI's, and it seems nobody has figured out how to do it with Flash or some other off the shelf toolset, it would seem to indicate that it's really not so simple and therefore a significant investment will be required to make it happen.  If that's the case, then perhaps the only way we could get approval on the investment is to develop everything from scratch so we could hold the licensing rights.  But as a developer myself, the last thing I want to do is rewrite code or reinvent something that's already done and working, and I'd much rather start with something that exists.

What would be great is if we could get some people involved in the discussion on the 19th who know Flash, Gnash and the other existing software really well so they can let us know what exists already and how much work would be required to make it work for UI3.  If these devs are free on the 19th and, ideally, not too far from California, Pluto will cover the cost of bringing them over.  If not, we plan to setup a web cam conference anyway.
Title: Re: UI3 Discussion
Post by: tschak909 on January 16, 2008, 09:27:02 pm
I would like to be present on the discussions, however my health (I have paroxysmal atrial tachycardia) will not permit me to go to SF (even when you and I had talked on the phone Aaron, I had just had an attack an hour earlier..)... so I would like an alternative means to participate.

-Thom
Title: Re: UI3 Discussion
Post by: Matthew on January 16, 2008, 11:06:21 pm
I really would like to find a way to somehow base this on Flash because it will be impossible to reinvent Flash's design suite.

I'm glad we agree on that general premise. And I'm even more encouraged that we're thinking this all out loud in the community, and that it's not just some "done deal" between Pluto and TrollTech, with the rest of us just watching as we get to decide whether to just "go along for the ride" or not. And in case it's not obvious, I'm also very grateful to Pluto (and everyone else involved) for all the excellent work done so far. Not just the final supercool product, but all the design in it that makes it so expandable while so complexly featureful and integrated. And of course the "open mind" to use whatever works when it's available.


The one area that Adobe's design tools may be lacking in (I don't know for sure) is the cross-platform stuff.  Pluto's HADesigner is crude as can be, but, it was designed to target lots of platforms.  For example, the 'play' button in UI1 on windows ce pda, and the 'play' button in UI2 on the TV, as well as on the Cisco phone and mobile phone, are actually all the same object.
(...)
I'm not sure if Adobe's flash designer is object oriented like that, supporting inheritance and overrides, so you can create, say, a button that looks and behaves differently on various platforms.  I've never seen Flash used like this; the general goal of flash is the opposite: to have a player that renders the flash file exactly the same on all platforms, rather than having one UI that behaves differently on the different platforms.

GNASH is designed to be better than Adobe's Flash player in this regard, the support of many very different platforms with different presentation and computation resources (also a more efficient VM especially to benefit embedded devices). I'm not sure how GNASH supports "robustness failover", though I believe you are correct in seeing the Adobe Flash paradigm as the same as their Acrobat paradigm: the point is to present the same bitmaps on every platform. But keep in mind that GNASH supports SWF and ActionScript, not just Adobe's "Flash", so it has other language features as well. I don't know whether inheritance and overrides are supported techniques. But I do know that OpenLaszlo (again, an SWF/ActionScript tool, not necessarily Flash) generates each of SWF and DHTML by compiling the same source code with different output format requests. So there's at least one technique for targeting differing GUI toolkits on different classes of device, but retaining all the other benefits.


While this may be a weakness, it's not necessarily a deal breaker since, after all, if Flash designer is not object oriented with inheritence, the person designing the UI could work around this by just creating separate flash files for the mobile phone, the pda, the cisco phone, the TV, etc.  Most designers would probably prefer to create 5 UI's using Adobe's flash designer vs. 1 UI with Pluto's HADesigner because our HADesigner is that bad and it's unlikely we'd have the manpower to create something that rivaled Adobe's flash designer.

In fact I saw an interview (http://blogs.zdnet.com/Stewart/index.php?p=177) with GNASH's originator, Rob Savoye in which he implies the approach for handling "degraded" GUIs on lesser devices is typically to produce different SWF content targeting what's supported there. But I expect there's a way to balance the two techniques, especially if there's going to be coding the player involved.


The other thing I'm not sure of is how well flash scales.  In general all the flash files I've seen are created for a target resolution (say 800x600).  With Pluto's HADesigner, OrbiterGen scales the UI to any resolution, rotates it (ie some devices are vertical not horizontal), and stretches it (16:9 vs. 3:4).  This would be a big deal breaker if Flash couldn't do this because, although we could tell designers they need to create 5 different UI's for all the target platforms, telling them to create each of those 50 times (5 UI's at 10 different resolutions) would be ridiculous.  If they wanted to change the command the 'play' button sent, rather than doing it in 1 place like you do with HADesigner, you'd have to make the same change to 50 UI's if Flash didn't support scaling/rotating/stretching and didn't support object oriented inheritance with overrides.

Well, one of SWF's central features is that it's primarily vector graphics, which are inherently scalable. And I've certainly seen plenty of little Flash widgets for as small as 32x32 animated icons. And Sun's incorporation of OpenLaszlo in its JME for little devices indicates that's a design requirement for its use in UIs.


Here what's basically lacking is the ability to blend 2 different applications together.  Flash is designed to work stand alone, not bolt on top of and blend into something else.  We're looking for a UI that can bolt on top of MythTV, or Xine, or MPlayer, or Firefox (for a 10' web browser), etc.  If, however, we commit to using Adobe's flash designer as a back end, then that means we've committed to the flash file format and we're better off to put resources into adding the alpha blending capabilty to something that already plays back flash files, like Gnash, than to start from scratch reinventing a flash player.

I think making GNASH do alpha blending is exactly the way to go. Since SWF (even with its recent 3D techniques) is designed to render into a plane, and Adobe's many other products that are designed in terms of bundles of layers, this could be relatively straightforward within the overall architecture. Also there is likely room to add some 3D support beyond blended layers to GNASH, where UI3 requirements exceed GNASH's current capabilities. I think adding those features to GNASH would also make a lot of friends with users and developers who would be assets to the LinuxMCE community. Of course it would be faster to market to add those features to GNASH than to invent a new player that competes with it and with Flash. Leaving more time to actually create the GUI "content" itself that executes in the GUI platform.


Here is where it gets tricky.  Pluto has investors, and when we put in the budget expenses for development, we have to show that there's some way that we can at least make enough revenue to cover the expenses, and ideally make a profit, otherwise it's considered throwing money away.

Normally the way companies do this with FOSS is to offer software under the GPL that, in order to be used commercially, needs to link with non-GPL code, and the company then licenses it outside the GPL for this commercial use.  For example, with MySQL, the client libraries were originally LGPL, but that meant commercial companies could use them in non-GPL apps without paying a license fee.  So, in order to generate revenue, MySQL changed the client libraries to GPL.  Trolltech did the same thing with QT.  Except for cases where the software is offered b2b as part of support contracts (think SuSe, Asterisk, etc.), which doesn't apply to us, licensing the software outside the GPL for commercial use seems to be the standard way that a company can offer software that is open source for the FOSS community, but that they can still make money for with commercial use.

That was our intention with UI3.  We want all the FOSS projects to be able to use it for free under the GPL, since they will release their UI's under the GPL too.  But, since we will be hiring a lot of developers for this, and probably sponsoring several more, our justification to our investors for spending that money is to be able to license the same software outside the GPL, like MySql and Trolltech do.

So if the back end design tool is Adobe Flash Designer, and the front end player is either Adobe or Gnash, we have no way to explain why we should invest a lot of money on development.  If the approach of using Flash Designer + Gnash was close enough to being a workable solution, or if there were enough volunteers to do the work, then it's not an issue; we don't need a revenue model because we won't need to invest a lot of money in it anyway.  However, my gut feeling is that this likely will require a substantial investment because, as an example, MythTV's devs would obviously like to have a UI as pretty as TiVo or Apple or Vista, and if it really was that easy to create a beautiful UI in Flash Designer and bolt it on top of Myth, Xine, MPlayer, etc., then they all would have been doing it already.  Since all the media center apps need pretty eye candy UI's, and it seems nobody has figured out how to do it with Flash or some other off the shelf toolset, it would seem to indicate that it's really not so simple and therefore a significant investment will be required to make it happen.  If that's the case, then perhaps the only way we could get approval on the investment is to develop everything from scratch so we could hold the licensing rights.  But as a developer myself, the last thing I want to do is rewrite code or reinvent something that's already done and working, and I'd much rather start with something that exists.

Well, if the platform is SWF+ (including some extensions that let UI3 do a few things SWF doesn't already do), then it won't cost the investors as much to produce. The main financial benefit to the upgrade investment is that the better product will sell more. To the point, more competitive with Windows MCE, no longer held back by comparison to the familiar (and therefore "easy", at least sold as such) Windows GUI. The SWF architecture also creates the opportunity for Pluto to create separate GUI content that doesn't have to be GPL (or Creative Commons, or anything else other than simple private Pluto copyright). Pluto is free to create GUIs better than anything else. To create branded GUIs in private deals with equipment manufacturers or retailers/installers, that the community can't make, because it's not a simple corporation. Pluto's expertise in the way the new GUI works puts it in the lead to make the best GUIs. The community can make its own GUIs, perhaps with Pluto's help once Pluto recoups its investment on the "private label" GUI. All of which will show the power of the platform, which will grow the whole ecosystem, with Pluto still at its center.

In short, Pluto and the community can collaborate to cut the costs of the extra development on top of reusing the FOSS already incorporated. And position Pluto to make money from the new GUI and overall sales in ways that are more profitable than before.


What would be great is if we could get some people involved in the discussion on the 19th who know Flash, Gnash and the other existing software really well so they can let us know what exists already and how much work would be required to make it work for UI3.  If these devs are free on the 19th and, ideally, not too far from California, Pluto will cover the cost of bringing them over.  If not, we plan to setup a web cam conference anyway.

I'm in NYC, and I'm not exactly an expert in GNASH. But I've used the public SWF tools before (and I used to develop developer tools for Apple), as well as IDEs for targeting multiplatform (and crossplatform) content and executables. And I think I have a handle on the issues here, at least from the perspective of an outside developer who wants to help LMCE get as far as possible as fast as possible. If you want to fly me out, I'm game, but I'm OK with a webcam and speakerphone. And I'm really enthusiastic about the way that Pluto is going about exploring all the dimensions of this new chapter in LMCE, which is consistent with it's history on the project.
Title: Re: UI3 Discussion
Post by: aaron.b on January 17, 2008, 02:49:03 am
Matthew,  That's great.  I sent a private email to you about joining the meeting.  My email is aaron.b  [at] plutohome (dot) com

I also contacted Rob Savoy, lead dev of Gnash, and have invited him.  We already exchanged emails and he sounds interested.  We might be able to get him to work with us on this.
Title: Re: UI3 Discussion
Post by: aaron.b on January 17, 2008, 07:45:49 am
I had a nice chat with Matthew.  One thing that's clear is perhaps I didn't correctly state what I meant by needing a UI with object oriented properties, like inheritance.  Here's a real world example...

1.   Someone is creating a new screen in the design tool.  In the tool, there exists style sheets with all the font styles.  The style "Mid-size button" is defined as Times Roman 10 point. I create a button called 'play'.  It has a blue background that pulsates with an animated effect 3 times per second, and has the text "PLAY" on top of it in that "Mid-size button" style.  I cluster this together with a "pause" and "stop" button in a container called "Media Controls".

2.   A different designer doesn't like the blue background, and instead wants something with wood grain.  He doesn't want to recreate the screen, or change the layout or have to duplicate the commands associated with the buttons.  Just change the background.  So, he creates a new skin called 'wood' in the design tool, and creates a wood grain bitmap, and in the design tool associates this new bitmap with the skin.  Now, when the user selects the 'wood' skin he sees the same button with 'play' in Times Roman 10 point, but the background is wood grain.  The only thing is the pulsating animated effect doesn't look good on top of the wood, so he chooses another effect called "scroll" which makes the background wood grain image scroll a bit.  There may be 100 buttons the original designer created with the blue background and the pulsating effect.  This designer shouldn't have to open the properties for each button one at a time, he should be able to make a global change that says for this new skin, use this wood grain bitmap and scroll effect instead.

3.   Now there's a user running the UI on a PDA.  He finds that with the original blue skin it looks ok except that the pulsating effect is using all the cpu, and with the wood skin, the scrolling is just too much for the pda.  So he opens the 'play' button in the design tool, clicks on the 'effects' tab, chooses the 'blue' skin and sees the 'pulsate' effect, he clicks 'criteria' and under "target device" selects "PDA", and then moves to the property field "times per second" and changes it from 3 to 1 so it goes slower on the pda.  Then he chooses the 'wood' skin, again sets the criteria to 'target device=pda', and this time sets 'effect' to 'none'.  This doesn't effect either user #1 or #2 since they don't use a pda.

4.   Now there's another guy using a Cisco VOIP phone.  He sees that on his phone all the effects are too much for the phone and need to be disabled.  And although the 'Times Roman 10 point' text is legible with the 'blue' skin, when he choose the 'wood' skin which has more contrast, the text is unreadable because the Cisco phone's lcd display isn't that sharp.  So, he opens the design tool, selects 'platform = cisco voip phone' and checks off "turn off all effects".  Then he opens the style sheet tool and selects the "mid-size button" style, and clicks on "criteria" and sets "skin" = "wood", and changes the font size from 10 point to 12 point.  Except, he notices, when the language is German.  Then the words are just too big for 12 point font so the user has to live with 10 point, so he edits his criteria and adds to "skin=wood" a "and language<>german".

5.   Another user decides he wants to create a new variation called "sideways" that aligns the clusters vertically rather than horizontally.  He doesn't want to change any of the other stuff.  He just wants to open "Media Controls" cluster, which contains the play/pause/stop buttons in a horizontal group, and he re-arranges them so they are vertical.  They retain all their current properties; the play button still pulsates when it's the blue skin, except on the Cisco phone, and the font size is still 12 point instead of 10 with the wood skin on a cisco phone.  It's just arranged differently.

Note that whatever file format this master stuff is stored in must be something that supports collaborative development, so different users can make changes to the same object, and then those changes are merged back into the 'master'.

This is how Pluto's current HADesigner works (or at least how it was designed to work).  This is what I mean by it's object oriented and objects inherit properties, which you can override based on criteria.  There's a criteria builder (it's not very functional, but it's there), so you can build a criteria "skin=wood, device=cisco phone, resolution=less than 640x480".  And then you can change the properties of a button for that particular criteria.  You can also build a criteria for a text style, so that when 'device=cisco and skin=wood' the font size is 12 point and not 10.  The file format for this 'master' definition is an sql database.  And sqlcvs was created with HADesigner in mind so that the 5 users described above could all be changing the properties of the 'PLAY' button at the same time and have their changes get merged together.  Then, the program OrbiterGen runs which reads the 'master' definition from the sql database, and outputs a .info file which is the native format that Orbiter reads.  Orbiter doesn't read the master definition.  If you are running Orbiter on a cisco phone with the 'wood' skin, when it reads in the 'play' object, it just sees an object that has no pulsate/scroll effects, and is already scaled for the cisco phone, with the wood background and the word PLAY in 12 point text.  OrbiterGen had the responsibility for each object of evaluating the "criteria" to pull out the properties for that object given the target platform, resolution, skin, operating system, ui version, etc.

From a conceptual standpoint, I don't think HADesigner was bad.  The idea that everybody could edit the same objects and, without re-creating those objects, just change certain properties for specific situations made sense.

The challenge is that (a) HADesigner just awful to use, and (b) really limited in it's eye candy.  All the images are bitmap based (not vector based).  You can make them animated by using the MNG format (an animated bitmap, like animated gif), but this hasn't really been used at all.  There's a bunch of other ways that it's totally lacking.  The only thing I like about it is the concept behind the object properties and the collaboration possibilities.

Now let's say we want to let graphics designers use Adobe's Flash Design tools.  It has lots of great widgets for creating pretty objects that are animated, etc.  But, if it didn't support the concept of overriding object properties like I described in those 5 examples (which I don't think it does) it would be nearly impossible to create the kind of UI's that we do that target all those platforms.  Sure, the objects may have properties and the user could attach action script code which changed them.  But having a huge cluster of if statements in action script is not maintainable.  If skin=wood and platform=cisco and <blah blah> then object.text.style.pointsize=12; else if skin=blue and <blah> <blah>..  Just for those 5 steps I described above, you'd have dozens and dozens of nested if statements in a mangle of spaghetti code.  It would be a nightmare for the flash developers to have to attach hundreds of line of 'if' statements to each of the objects in the UI (there are many thousands).  And the developer would go crazy if, for example, every time he wanted to change something for the 'wood' skin he had to edit the action script of 1,000 buttons and add an 'else if skin=wood' to it.  And if collaboration would be impossible since you couldn't merge all that action script code; if 5 developers were each working on 5 skins, and all 5 were changing this huge block of nested else if statements, they'd have conflicts for each of the 1,000 buttons.  So rather than the design tool giving the user action script for each object, or button, the design tool has to support concepts of styles and properties which are inherited across all objects, with global criteria that you can change properties in one place and it propagates across all objects.  And to allow designers to work freely, they shouldn't have to edit code to do this; it should be done wysiwyg in the design tool.  I don't know Adobe's Flash designer, but I think that it doesn't work like this.

Also, I'm even sure the SWF file format is by itself sufficient.  I'm not sure that the swf format supports the ability to have that criteria/object properties.  After all, this isn't what flash and swf were designed for.  And, even if the swf format did support this, you probably can't pass that directly to the target render device (ie the Cisco phone).  As stated in my example with the 5 steps, to determine what the 'play' button should look like on a cisco phone with the 'wood' skin you have to evaluate a bunch of criteria factors; what the skin is, what the platform is, etc.  You don't want the player in the Cisco phone (which has a weak cpu), to have to go through that whole evaluation process to determine what the properties are to display the 'play' button.  It would be way too slow.  Rather there needs to be some other intermediary process that takes the master file that defines how a play button should work on all the various skins/platforms/languages/etc., and pre-processes the play button for the Cisco UI, pre-scaling any bitmaps, pre-rendering any fonts, etc., so that the player in the Cisco phone just gets the finished 'play' button and doesn't need to do a bunch of complex calculation.  This intermediary process is what OrbiterGen currently does.  So there would likely still need to be an intermediary program which reads in the master file, evaluates everything, and outputs a pre-rendered/scaled UI for a specific box.

If we wanted to use the swf format, and if the swf format is, as I suspect it is, really targeted for a certain platform/skin, then likely the 'master' format which defines all the objects and which the design tool uses would need to use another format, and this intermediary program that replaces OrbiterGen is what would need to read this master format, evaluate all the criteria for whatever UI it's targeting, and output a generated swf that could be played on it.  I think this is similar to what Laszlo does.

So this raises a bunch of issues and questions.  What is the design tool?  Adobe's Flash Designer is closed source.  So unless it did everything we need it to do out of the box we couldn't extend it to suit our needs.  Is there another design tool that's really polished and could be extended?  Does swf support all the object inheritence we need so that it can be the master format?  If so, should we feed that 'master' swf to each UI, even though that master swf defines every criteria for each object, or should we do some pre-processing so each UI just gets an swf that's pre-rendered for it?

Does anybody know if there exists a UI design tool that works like HADesigner does with object criteria/properties/inheritance to target multiple platforms and skins like Pluto does?  Is there another way to target all those platforms that doesn't require this approach?  If there's no way to get flash to do what we want with the multiple platform stuff, but if flash will let us create an eye-candy rich UI, maybe we need to consider scrapping the idea of having one unified UI/design tool that targets all those platforms and just use flash for the 10' UI on the TV, but leave HADesigner/OrbiterGen/Orbiter as-is for the other platforms like the Cisco phones...
Title: Re: UI3 Discussion
Post by: totallymaxed on January 17, 2008, 12:09:34 pm
I'll divide my comments in 3 parts: 1) the design tool, 2) the player, 3) financial issues.  Both the design tool and the player are *almost* what we need, but, lacking in a couple areas.

--Design tool--

I really would like to find a way to somehow base this on Flash because it will be impossible to reinvent Flash's design suite.  I have never used it, but just from looking at the demo video on Adobe.com, it is clearly a very rich, robust, polished design suite that allows the designer to create gorgeous UI's that do almost anything with sophisticated illustration and 3d tools.  The thought of us saying "Let's rebuild something that's as good as Adobe's flash designer" is daunting; that software was 10 years in the making with hundreds of full time developers.  We just can't make something as elegant as it.  And we'll never be able to reproduce having tens of thousands of developers who know it intimately so they can create cool eye candy effects in our UI.

The one area that Adobe's design tools may be lacking in (I don't know for sure) is the cross-platform stuff.  Pluto's HADesigner is crude as can be, but, it was designed to target lots of platforms.  For example, the 'play' button in UI1 on windows ce pda, and the 'play' button in UI2 on the TV, as well as on the Cisco phone and mobile phone, are actually all the same object.  You just assign different attributes for each of the variations (like object oriented inheritance), so you can make it be animated and round on a more powerful box, and a small static square on a mobile phone.  You can also cluster the 'play/pause/stop' buttons into a 'media playback group', and say that the group itself is also different on a tv vs. a mobile phone (ie object containers).  Then, when you're creating a media remote control, you just drop the 'media playback group' on the screen and whether you're using a mobile phone or a tv it still works.  I'm not sure if Adobe's flash designer is object oriented like that, supporting inheritance and overrides, so you can create, say, a button that looks and behaves differently on various platforms.  I've never seen Flash used like this; the general goal of flash is the opposite: to have a player that renders the flash file exactly the same on all platforms, rather than having one UI that behaves differently on the different platforms.

While this may be a weakness, it's not necessarily a deal breaker since, after all, if Flash designer is not object oriented with inheritence, the person designing the UI could work around this by just creating separate flash files for the mobile phone, the pda, the cisco phone, the TV, etc.  Most designers would probably prefer to create 5 UI's using Adobe's flash designer vs. 1 UI with Pluto's HADesigner because our HADesigner is that bad and it's unlikely we'd have the manpower to create something that rivaled Adobe's flash designer.

The other thing I'm not sure of is how well flash scales.  In general all the flash files I've seen are created for a target resolution (say 800x600).  With Pluto's HADesigner, OrbiterGen scales the UI to any resolution, rotates it (ie some devices are vertical not horizontal), and stretches it (16:9 vs. 3:4).  This would be a big deal breaker if Flash couldn't do this because, although we could tell designers they need to create 5 different UI's for all the target platforms, telling them to create each of those 50 times (5 UI's at 10 different resolutions) would be ridiculous.  If they wanted to change the command the 'play' button sent, rather than doing it in 1 place like you do with HADesigner, you'd have to make the same change to 50 UI's if Flash didn't support scaling/rotating/stretching and didn't support object oriented inheritance with overrides.

--Player--

Here what's basically lacking is the ability to blend 2 different applications together.  Flash is designed to work stand alone, not bolt on top of and blend into something else.  We're looking for a UI that can bolt on top of MythTV, or Xine, or MPlayer, or Firefox (for a 10' web browser), etc.  If, however, we commit to using Adobe's flash designer as a back end, then that means we've committed to the flash file format and we're better off to put resources into adding the alpha blending capabilty to something that already plays back flash files, like Gnash, than to start from scratch reinventing a flash player.

--Financial Model--

Here is where it gets tricky.  Pluto has investors, and when we put in the budget expenses for development, we have to show that there's some way that we can at least make enough revenue to cover the expenses, and ideally make a profit, otherwise it's considered throwing money away.

Normally the way companies do this with FOSS is to offer software under the GPL that, in order to be used commercially, needs to link with non-GPL code, and the company then licenses it outside the GPL for this commercial use.  For example, with MySQL, the client libraries were originally LGPL, but that meant commercial companies could use them in non-GPL apps without paying a license fee.  So, in order to generate revenue, MySQL changed the client libraries to GPL.  Trolltech did the same thing with QT.  Except for cases where the software is offered b2b as part of support contracts (think SuSe, Asterisk, etc.), which doesn't apply to us, licensing the software outside the GPL for commercial use seems to be the standard way that a company can offer software that is open source for the FOSS community, but that they can still make money for with commercial use.

That was our intention with UI3.  We want all the FOSS projects to be able to use it for free under the GPL, since they will release their UI's under the GPL too.  But, since we will be hiring a lot of developers for this, and probably sponsoring several more, our justification to our investors for spending that money is to be able to license the same software outside the GPL, like MySql and Trolltech do.

So if the back end design tool is Adobe Flash Designer, and the front end player is either Adobe or Gnash, we have no way to explain why we should invest a lot of money on development.  If the approach of using Flash Designer + Gnash was close enough to being a workable solution, or if there were enough volunteers to do the work, then it's not an issue; we don't need a revenue model because we won't need to invest a lot of money in it anyway.  However, my gut feeling is that this likely will require a substantial investment because, as an example, MythTV's devs would obviously like to have a UI as pretty as TiVo or Apple or Vista, and if it really was that easy to create a beautiful UI in Flash Designer and bolt it on top of Myth, Xine, MPlayer, etc., then they all would have been doing it already.  Since all the media center apps need pretty eye candy UI's, and it seems nobody has figured out how to do it with Flash or some other off the shelf toolset, it would seem to indicate that it's really not so simple and therefore a significant investment will be required to make it happen.  If that's the case, then perhaps the only way we could get approval on the investment is to develop everything from scratch so we could hold the licensing rights.  But as a developer myself, the last thing I want to do is rewrite code or reinvent something that's already done and working, and I'd much rather start with something that exists.

What would be great is if we could get some people involved in the discussion on the 19th who know Flash, Gnash and the other existing software really well so they can let us know what exists already and how much work would be required to make it work for UI3.  If these devs are free on the 19th and, ideally, not too far from California, Pluto will cover the cost of bringing them over.  If not, we plan to setup a web cam conference anyway.


Hi Aaron,

As you know from our conversations over the last 18 months or so we also have been looking at building a UI for LinuxMCE in Flash. This work has been underway for about 6-7 months now... but back in October we decided that we had to focus on UI2 so that we could get to market with our customers earlier.

However we do believe that Flash is the way to go because of the great tools for designers that already exist and because the designers out there already understand how to build great UI's in Flash. So I 100% agree with Mathews comments here about Flash/Gnash etc.

So we would be very interested in a UI3 that was Flash based as our UI team is already developing in Flash. Some of our UI code would be redundant under the proposed UI3 architecture but in other areas we would have little or no work to do as much more of the "glue" underneath the skin would be part of the framework as opposed to something we would build on-top.

We would be happy to commit some resources to help support the Open Source effort to allow a full scalable & portable Flash UI3 to be delivered as this would allow us to focus back on our design ideas.
Title: Re: UI3 Discussion
Post by: Matthew on January 17, 2008, 03:38:37 pm
Alternately, if OpenLaszlo, like Flash, is just too big and complex, what about SVG? SVG with ECMAScript can make lightweight but featureful GUIs that run in browsers and can be encapsulated in runtimes (Java or native x86/etc binaries). Again, why reinvent the wheel, rather than just build a better car or road for it?

There are also tools to convert SWF to SVG, including one SWF2SVG (http://www.3d2toy.com/swf2svg/) that's actively developed and just released a version this month. These tools (AFAICT) all render individual frames from SWF to SVG, not animated SVG. But they're at least a starting point for generating SVG from a single codebase that, with OpenLaszlo, could then generate SWF, DHTML and SVG, depending on the device targeted by developers at compile time.
Title: Re: UI3 Discussion
Post by: kir on January 17, 2008, 07:01:45 pm
My thoughts:

- I don't think that Flash is a good choice for the UI3 basis: it is based on different ideas, it is used for different purposes and it is not open. In fact, the biggest common thing with UI3 in Flash is that it is sometimes used for building user interfaces :) In fact very often, it is not: I've seen much more sites based with UI based on pure HTML, HTML+JavaScript, etc. etc. than based purely on Flash. I.e. even for the UI-building task, Flash is not the 'best' tool. Web 2.0 sites for example, are based on AJAX and other similar approaches, not on the Flash itself, although they can embed Flash into them. For the desktop applications (i.e. running outside of the browser), I even cannot remember any _big_ application with Flash-based interface. So - selecting Flash is like selecting an old somebody's else SUV car and trying to use it on races: it was designed for different purpose, it is old and you cannot heavy-tune it.


- The same applies to the Gnash - with addition that Gnash (in the sense of development direction) is a tail of Flash pony: Gnash has to follow the Flash features and evolution. It will never be the separate.


- Regarding the design of UI3: I am not sure that it is a good idea to create it from scratch, because of the following:
0) most of the time it will be not feature-ready and incomplete, so everybody will still use the UI2 and see the "stall in the development"

1) if you want to design it from scratch, you will most probably have to hack all and every piece in whole LinuxMCE system, to propagate new ideas/design approaches - or you will restrict your "open mind" to some subset of things (say, "UI3 has to coexist nicely with applications A,B,C") thus losing half of "from scratch" benefits

2) parts of discussion, especially "big list of all required features and possible uses" reminds me about the well known "second-system" effect ( http://en.wikipedia.org/wiki/Second-system_effect ) - I am afraid this attempt can fail because of too big initial expectations/design features. It should be really lightweight

3) I don't see the list of "problems" with UI2 except that it sometimes lacks developers' attention - wouldn't it be better to start refactoring the existing code, creating "UI evolution roadmap", adding features, switching to new approaches/implementation, etc.


- Regarding the IDE: don't care too much about the big and powerful IDE at the beginning of journey: if the development be successful, the IDE will appear. Of course, certain ideas should be taken into account, such as - it is always easier to modify human-readable file that to issue SQL queries. Like with websites - webdesigners doesn't use IDE 100% of time - they spend a lot of time in notepad-like text editors, tweaking the HTML,CSS,JS,etc. :)


- Personally I think that "evolution" is better than "revolution" and would vote for the creating UI2.1, UI2.2 etc. with extra features/improvements


- For enhancing the usability, I would suggest to ask someone who interested to modify LinuxMCE current UI and see what are the common issues/things-hard-to-do - and compose a list of them and eliminate/optimize them one-by-one.


Now, couple of random thoughts related to the above description of "desired features" and logic of UI design:

- Description of objects properties inheritance and conditional properties evaluation, reminds me about HTML+CSS style of UI implementation - does anybody else see this analogy? Maybe we could utilize it and define the engine layouts in kind of XML-ish format. And use CSS-analogue approach for defining stylesheets, too.


- Targeting the same interface to all possible devices doesn't look as a good approach to me: compare, e.g. big 50" plasma TV and ~160x120px cellphone screen: trying to have the same screen for them, even styled differently doesn't seem feasible. Or the result will be non-optimal. So, why not divide all devices into few categories, based say on screen size, CPU power and network speed:
   screen size: high (TVs and displays), medium (PDAs, Cisco-like phones), low (mobile phones)
   cpu/rendering power: high ( PCs ), medium (PDAs, Cisco-like phones), low (mobile phones)
   network speed: high (100/1000Mbit), medium (WiFi), low (Bluetooth)

The ranges can be more flexible of course :) Of course this doesn't mean that there should be 3*3*3=27 different interfaces: the main could be the screen size (so there should be 3 different interfaces), the CPU power will determine which effects are on/off and if the device is 'active renderer' (like Orbiter) or passive (like plain bitmap display with off-screen renderer sitting e.g. on core). The network speed will allow to decide what media can be streamed, if the device can work as "bitmap display" or should be even falled back to text-only interface (if network is too slow).

So it is kind of "enumeration of capabilities" for different target devices and tuning/selecting proper UI for them.


- Scripting: imho it would be nice to have kind of scripting engine for the GUI. It can be either JavaScript or QTScript depending on what is better. - this will allow to extend UI w/o doing direct coding. Again, many people are familiar with scripting languages so this would be a useful feature. Scripting support was an overkill 5-10 years ago, but now PCs are faster, so this is not an issue. For scaling down to the slow devices, the script code can be executed on core in the way similar to off-screen renderer for bitmap rendering.


- Finally, if creating UI3 from scratch will be selected, I would suggest to utilize 3D-games developers experience - there are many nice tricks/ideas that can be used to create nice and extensible interface. "World organization" (for UI it is about widgets, and binding the actions to them), scripting, skins and even audio/video playback - all this is present in modern games. Including need of scaling to different resolution (within some range).
Title: Re: UI3 Discussion
Post by: Matthew on January 17, 2008, 07:55:08 pm
- I don't think that Flash is a good choice for the UI3 basis: it is based on different ideas, it is used for different purposes and it is not open. In fact, the biggest common thing with UI3 in Flash is that it is sometimes used for building user interfaces :) In fact very often, it is not: I've seen much more sites based with UI based on pure HTML, HTML+JavaScript, etc. etc. than based purely on Flash. I.e. even for the UI-building task, Flash is not the 'best' tool. Web 2.0 sites for example, are based on AJAX and other similar approaches, not on the Flash itself, although they can embed Flash into them. For the desktop applications (i.e. running outside of the browser), I even cannot remember any _big_ application with Flash-based interface. So - selecting Flash is like selecting an old somebody's else SUV car and trying to use it on races: it was designed for different purpose, it is old and you cannot heavy-tune it.


- The same applies to the Gnash - with addition that Gnash (in the sense of development direction) is a tail of Flash pony: Gnash has to follow the Flash features and evolution. It will never be the separate.

Maybe that's true for "Flash", but Adobe's Flash is just their branded instance of the open standard SWF. GNASH is an open SWF player (and FLV server) that doesn't actually have to follow Flash, just SWF. It does, however, follow Flash without LMCE doing anything, so that's not a problem. However, SWF does indeed reflect quite a lot of what UI3, and LMCE in general, need to do, and the ways to do it. As I've posted in this thread, Sun has included OpenLaszlo in its new Java platforms that are already parts of the Blu-Ray, DVB and mobile phone specs. So lots of devices already work according to its design, and even include support for features that OpenLaszlo scripts.

FWIW, just because HTML is used for a lot more GUIs than is Flash doesn't mean Flash isn't good for a GUI, or for specifically UI3. Just because most cars on the road are sedans doesn't mean that pickup trucks are inappropriate for working construction sites. LMCE is much more complex than most software, especially the Web software that's pretty crippled in its UI paradigm and widgets (and which has to get downloaded with every new page).

1) if you want to design it from scratch, you will most probably have to hack all and every piece in whole LinuxMCE system, to propagate new ideas/design approaches - or you will restrict your "open mind" to some subset of things (say, "UI3 has to coexist nicely with applications A,B,C") thus losing half of "from scratch" benefits

Certainly not. LMCE is a very properly tiered architecture. Only the GUI needs to be touched for UI3 to take over. We're talking about the Orbiter, in various platforms (standard Orbiter, Web Orbiter, mobile Orbiter, IP phone Orbiter, etc).


- Regarding the IDE: don't care too much about the big and powerful IDE at the beginning of journey: if the development be successful, the IDE will appear. Of course, certain ideas should be taken into account, such as - it is always easier to modify human-readable file that to issue SQL queries. Like with websites - webdesigners doesn't use IDE 100% of time - they spend a lot of time in notepad-like text editors, tweaking the HTML,CSS,JS,etc. :)

It's irresponsible to replace a GUI with a new system without including a GUI designer (or allowing use of an existing one). Especially when one major limit of LMCE is that its old GUI designer is acknowledged totally unusable by both its users and its authors. This requirement is a driving force in producing a UI3. "It'll come" is no way to plan a project, especially when it'll have to come from us. We have to pick the framework that will most reliably include a GUI design/composition system, like a data format that already has one, or plan to write one ourselves.


- Description of objects properties inheritance and conditional properties evaluation, reminds me about HTML+CSS style of UI implementation - does anybody else see this analogy? Maybe we could utilize it and define the engine layouts in kind of XML-ish format. And use CSS-analogue approach for defining stylesheets, too.

Whatever we decide to do to produce the UI improvements, it will probably include runtime configs that style the GUI. HTML+CSS is one example of that model, but is very limited (owing mostly to the limits of HTML).

- Targeting the same interface to all possible devices doesn't look as a good approach to me: compare, e.g. big 50" plasma TV and ~160x120px cellphone screen: trying to have the same screen for them, even styled differently doesn't seem feasible. Or the result will be non-optimal. So, why not divide all devices into few categories, based say on screen size, CPU power and network speed:

No one is saying that a 50" TV and a cellphone should each show the same GUI. Just that it's valuable to have the most common code/content as far down the development stream towards the user and runtime as possible. Orbiter already does that, but could improve its common base to be shared further up the stream towards the developer. This paradigm is already evident in LMCE in GSD and handling devices that are controlled, not that are used as controllers. Making all of LMCE work that way for users, no matter what the role of the device in the LMCE ecosystem, is another advance that UI3 can bring, if we do it right.


I'm not really sure that you really understand GUI development, or LMCE's architecture. You also seem to be wrong about SWF and how it actually does relate to UI3 (or to Flash) on LMCE. Where are you getting these perspectives from?


- Scripting: imho it would be nice to have kind of scripting engine for the GUI. It can be either JavaScript or QTScript depending on what is better. - this will allow to extend UI w/o doing direct coding. Again, many people are familiar with scripting languages so this would be a useful feature. Scripting support was an overkill 5-10 years ago, but now PCs are faster, so this is not an issue. For scaling down to the slow devices, the script code can be executed on core in the way similar to off-screen renderer for bitmap rendering.

I am working on scripting LMCE (not just the GUI) by integrating Perl. Which is best done by exposing LMCE state to a script engine, and sending DCE messages to the DCErouter. Scripting the GUI itself seems like a redundant, and hard (given the various GUI HW) method that ignores the strength of the DCErouter.
Title: Re: UI3 Discussion
Post by: hari on January 17, 2008, 08:49:13 pm
I'm not really sure that you really understand GUI development, or LMCE's architecture.
but you do understand its architecture fully, Matthew, yes?

kir pretty much hits the nail on that..

best regards,
Hari
Title: Re: UI3 Discussion
Post by: Matthew on January 17, 2008, 08:58:49 pm
I'm not really sure that you really understand GUI development, or LMCE's architecture.
but you do understand its architecture fully, Matthew, yes?

kir pretty much hits the nail on that..

No, I don't understand its architecture fully. But I do think that I'm correct that a new UI3, however it's implemented, will not require touching every part of LMCE, but rather can be confined to the presentation layer. Unless I'm wrong about that, in which case I'd like to learn how so.
Title: Re: UI3 Discussion
Post by: kir on January 17, 2008, 09:30:54 pm
- I don't think that Flash is a good choice for the UI3 basis: it is based on different ideas, it is used for different purposes and it is not open. In fact, the biggest common thing with UI3 in Flash is that it is sometimes used for building user interfaces :) In fact very often, it is not: I've seen much more sites based with UI based on pure HTML, HTML+JavaScript, etc. etc. than based purely on Flash. I.e. even for the UI-building task, Flash is not the 'best' tool. Web 2.0 sites for example, are based on AJAX and other similar approaches, not on the Flash itself, although they can embed Flash into them. For the desktop applications (i.e. running outside of the browser), I even cannot remember any _big_ application with Flash-based interface. So - selecting Flash is like selecting an old somebody's else SUV car and trying to use it on races: it was designed for different purpose, it is old and you cannot heavy-tune it.


- The same applies to the Gnash - with addition that Gnash (in the sense of development direction) is a tail of Flash pony: Gnash has to follow the Flash features and evolution. It will never be the separate.

Maybe that's true for "Flash", but Adobe's Flash is just their branded instance of the open standard SWF. GNASH is an open SWF player (and FLV server) that doesn't actually have to follow Flash, just SWF. It does, however, follow Flash without LMCE doing anything, so that's not a problem. However, SWF does indeed reflect quite a lot of what UI3, and LMCE in general, need to do, and the ways to do it. As I've posted in this thread, Sun has included OpenLaszlo in its new Java platforms that are already parts of the Blu-Ray, DVB and mobile phone specs. So lots of devices already work according to its design, and even include support for features that OpenLaszlo scripts.

OK, the SWF is said to be open (IIRC, with exclusion that GNASH developers have to reverse-engineer SWF files to build SWF player, because they cannot look into SWF standard itself, right?). The Flash itself is very popular (in certain places) and there is even OpenLaszlo utilizing SWF (or DHTML) which is pushed to some devices. But imho, we are discussing different things - I am not attacking the Flash itself, I am in doubt if it has enough flexibility to implement the UI with all the required features. For example, consider video+UI blending, when video is played by Xine_Player and menu overlays video - can Flash/Laslo do it, taking into account that Xine window is a separate window itself? I don't know. Next example: the current version of Linux Flash plugin is 32-bit only, I have not heard about 64-bit version (and even about plans releasing it). Gnash sometimes fails for me even on "not very complex" SWFs - does this mean there will be no 64-bit version of UI3 until some_not_defined_time_point? What about heavy 3D animation / OpenGL-like effects - will SWF be able to handle it and will it give a good frame rate when rendering?


FWIW, just because HTML is used for a lot more GUIs than is Flash doesn't mean Flash isn't good for a GUI, or for specifically UI3. Just because most cars on the road are sedans doesn't mean that pickup trucks are inappropriate for working construction sites. LMCE is much more complex than most software, especially the Web software that's pretty crippled in its UI paradigm and widgets (and which has to get downloaded with every new page).

Seems we interpreted this "cars" analogy differently: when talking about SUVs, I mean the use of "inappropriate tool for reaching some goal", not "win by popularity" logic which is of course stupid. I still not see why the SWF is the best framework for building the UI3.

1) if you want to design it from scratch, you will most probably have to hack all and every piece in whole LinuxMCE system, to propagate new ideas/design approaches - or you will restrict your "open mind" to some subset of things (say, "UI3 has to coexist nicely with applications A,B,C") thus losing half of "from scratch" benefits

Certainly not. LMCE is a very properly tiered architecture. Only the GUI needs to be touched for UI3 to take over. We're talking about the Orbiter, in various platforms (standard Orbiter, Web Orbiter, mobile Orbiter, IP phone Orbiter, etc).

There are definitely some nuances and I am not completely aware about all of them, but for example - the standard Orbiter also does "windows" handling, e.g. when Xine_Player or Pluto_Screen_Server needs to be displayed, the proper window is brought on top, including all required positioning, resizing, etc. When creating the UI3 itself, this functionality will be probably separated into another tool, say magic_windows_manager. This means that either this magic_window_manager should be started only when UI3 is selected, or the same should happen for the UI2 and UI1 Orbiters. This what I mean "design changes propagation".

- Regarding the IDE: don't care too much about the big and powerful IDE at the beginning of journey: if the development be successful, the IDE will appear. Of course, certain ideas should be taken into account, such as - it is always easier to modify human-readable file that to issue SQL queries. Like with websites - webdesigners doesn't use IDE 100% of time - they spend a lot of time in notepad-like text editors, tweaking the HTML,CSS,JS,etc. :)

It's irresponsible to replace a GUI with a new system without including a GUI designer (or allowing use of an existing one). Especially when one major limit of LMCE is that its old GUI designer is acknowledged totally unusable by both its users and its authors. This requirement is a driving force in producing a UI3. "It'll come" is no way to plan a project, especially when it'll have to come from us. We have to pick the framework that will most reliably include a GUI design/composition system, like a data format that already has one, or plan to write one ourselves.

If you design the GUI system well (including the data structures, elements layout formats, etc.) it will be not hard to write a custom IDE for it (when rendering of sample UIs is already present and data format is selected carefully). But I certainly wouldn't recommend to select the UI framework putting the availability of IDE for it. 99% of time people will see the UI3, not the framework for UI3 development, that's the logic.

In addition - yep, Adobe does have an SWF editors, etc. but is all this stuff free? Is it easily extensible/customizable so we can modify it? As in file formats, the XML is better than a binary blob - with editors/creating tools it is easy when you can use free tools to make a modification. If Joe needs to buy an IDE for $300 just to change color of 1 button, what will he say? :)

- Description of objects properties inheritance and conditional properties evaluation, reminds me about HTML+CSS style of UI implementation - does anybody else see this analogy? Maybe we could utilize it and define the engine layouts in kind of XML-ish format. And use CSS-analogue approach for defining stylesheets, too.

Whatever we decide to do to produce the UI improvements, it will probably include runtime configs that style the GUI. HTML+CSS is one example of that model, but is very limited (owing mostly to the limits of HTML).

We are not restricted by the HTML standard, I was talking about "analogy", not direct use of this technology. But it is a bit funny to hear about restriction of HTML+CSS from you at the same time when you speak about the OpenLaszlo which can be also compiled into DHTML.



I'm not really sure that you really understand GUI development, or LMCE's architecture. You also seem to be wrong about SWF and how it actually does relate to UI3 (or to Flash) on LMCE. Where are you getting these perspectives from?


As far as I know, the SWF and Flash are not related to LMCE (at least yet ;-). If you are aware about some freshly developed code - please update me on this. Regarding GUI, LMCE and perspectives - good question. I never developed full-scale GUI framework, but used different kinds of them: VCL, MFC, QT, so I know what it is and what are components of. LMCE - it's a fork of Pluto, right? I am familiar with Pluto system and even submitted some code to them. Of course I don't know _full_ details architecture as it is too large to fit in my head  ;D, but DCE is for Devices,Commands,Events - right? :)


- Scripting: imho it would be nice to have kind of scripting engine for the GUI. It can be either JavaScript or QTScript depending on what is better. - this will allow to extend UI w/o doing direct coding. Again, many people are familiar with scripting languages so this would be a useful feature. Scripting support was an overkill 5-10 years ago, but now PCs are faster, so this is not an issue. For scaling down to the slow devices, the script code can be executed on core in the way similar to off-screen renderer for bitmap rendering.

I am working on scripting LMCE (not just the GUI) by integrating Perl. Which is best done by exposing LMCE state to a script engine, and sending DCE messages to the DCErouter. Scripting the GUI itself seems like a redundant, and hard (given the various GUI HW) method that ignores the strength of the DCErouter.
Looks like we are speaking about different things: when I am speaking about scripting, I mean "small routines, like flashing certain buttons, etc.", on-hover event handlers (change color), etc. Sending of messages to DCERouter for this doesn't happen. I would call this kind of things "local events", to distinct them from DCE Events.
Title: Re: UI3 Discussion
Post by: colinjones on January 17, 2008, 09:50:59 pm
The discussion is certainly interesting, and using a SWF renderer would be exciting if technically possible, from a "outcome" perspective if nothing else. But as far as arguing the detailed specifics of what is possible (both Kir and Matthew have declared they are no experts in SWF/Flash/Gnash) it seems to me that Aaron has already taken the most appropriate next step. In a couple of days time we can get an expert opinion on what the technologies are capable of, from the head of the Gnash development team. Surely he is in an ideal position to take one look at the LMCE interface, and basic design deliverables we are looking for and telling us with some authority as to whether they are capable of doing the stuff we are looking for, and with how much development to integrate it all?
Title: Re: UI3 Discussion
Post by: Matthew on January 17, 2008, 11:00:42 pm
The discussion is certainly interesting, and using a SWF renderer would be exciting if technically possible, from a "outcome" perspective if nothing else. But as far as arguing the detailed specifics of what is possible (both Kir and Matthew have declared they are no experts in SWF/Flash/Gnash) it seems to me that Aaron has already taken the most appropriate next step. In a couple of days time we can get an expert opinion on what the technologies are capable of, from the head of the Gnash development team. Surely he is in an ideal position to take one look at the LMCE interface, and basic design deliverables we are looking for and telling us with some authority as to whether they are capable of doing the stuff we are looking for, and with how much development to integrate it all?

I agree that getting the GNASH expert will settle much of this on the practical points. Though it's important to have goals and requirements ready. So I reply to some relevant points:

- I don't think that Flash is a good choice for the UI3 basis: it is based on different ideas, it is used for different purposes and it is not open. In fact, the biggest common thing with UI3 in Flash is that it is sometimes used for building user interfaces :) In fact very often, it is not: I've seen much more sites based with UI based on pure HTML, HTML+JavaScript, etc. etc. than based purely on Flash. I.e. even for the UI-building task, Flash is not the 'best' tool. Web 2.0 sites for example, are based on AJAX and other similar approaches, not on the Flash itself, although they can embed Flash into them. For the desktop applications (i.e. running outside of the browser), I even cannot remember any _big_ application with Flash-based interface. So - selecting Flash is like selecting an old somebody's else SUV car and trying to use it on races: it was designed for different purpose, it is old and you cannot heavy-tune it.


- The same applies to the Gnash - with addition that Gnash (in the sense of development direction) is a tail of Flash pony: Gnash has to follow the Flash features and evolution. It will never be the separate.

Maybe that's true for "Flash", but Adobe's Flash is just their branded instance of the open standard SWF. GNASH is an open SWF player (and FLV server) that doesn't actually have to follow Flash, just SWF. It does, however, follow Flash without LMCE doing anything, so that's not a problem. However, SWF does indeed reflect quite a lot of what UI3, and LMCE in general, need to do, and the ways to do it. As I've posted in this thread, Sun has included OpenLaszlo in its new Java platforms that are already parts of the Blu-Ray, DVB and mobile phone specs. So lots of devices already work according to its design, and even include support for features that OpenLaszlo scripts.

OK, the SWF is said to be open (IIRC, with exclusion that GNASH developers have to reverse-engineer SWF files to build SWF player, because they cannot look into SWF standard itself, right?). The Flash itself is very popular (in certain places) and there is even OpenLaszlo utilizing SWF (or DHTML) which is pushed to some devices. But imho, we are discussing different things - I am not attacking the Flash itself, I am in doubt if it has enough flexibility to implement the UI with all the required features. For example, consider video+UI blending, when video is played by Xine_Player and menu overlays video - can Flash/Laslo do it, taking into account that Xine window is a separate window itself? I don't know. Next example: the current version of Linux Flash plugin is 32-bit only, I have not heard about 64-bit version (and even about plans releasing it). Gnash sometimes fails for me even on "not very complex" SWFs - does this mean there will be no 64-bit version of UI3 until some_not_defined_time_point? What about heavy 3D animation / OpenGL-like effects - will SWF be able to handle it and will it give a good frame rate when rendering?

SWF is open enough, given what GNASH (and other developers) have produced. As I've said, neither SWF nor GNASH (nor, to the point, the SWF supported by GNASH) might be sufficient to support all of UI3's requirements. But SWF/GNASH seems to complete many, if not most, of UI3's requirements, without introducing much of anything extra that conflicts with UI3 or LMCE's requirements. Starting with SWF/GNASH and adding development to improve it where specifically UI3 needs help seems a lot more productive than starting from scratch (or just from the TrollTech libraries). That also brings the SWF/GNASH communities. And the OpenLaszlo and other SWF development tools bring the same kinds of benefits. Plus the Adobe support. The same goes for a 64bit version of either way: upgrade 32bit GNASH vs 64bit scratch. FWIW, GNASH also already supports OpenGL (and Cairo and AGG when OpenGL isn't available). Multiply the efforts in either track by the different GUI platforms, some with severe resource constraints (eg. mobile & IP phones), and just getting the GNASH headstart is a big boost, because GNASH does already do so much of what's planned for UI3.


FWIW, just because HTML is used for a lot more GUIs than is Flash doesn't mean Flash isn't good for a GUI, or for specifically UI3. Just because most cars on the road are sedans doesn't mean that pickup trucks are inappropriate for working construction sites. LMCE is much more complex than most software, especially the Web software that's pretty crippled in its UI paradigm and widgets (and which has to get downloaded with every new page).

Seems we interpreted this "cars" analogy differently: when talking about SUVs, I mean the use of "inappropriate tool for reaching some goal", not "win by popularity" logic which is of course stupid. I still not see why the SWF is the best framework for building the UI3.

I don't know SWF is the best way to do UI3. But it seems the best alternative we're currently discussing, and quite a good one. It seems SWF's features are highly complementary to UI3, especially since UI3 is to operate a media center, which SWF is highly suited to in its many multimedia features.

1) if you want to design it from scratch, you will most probably have to hack all and every piece in whole LinuxMCE system, to propagate new ideas/design approaches - or you will restrict your "open mind" to some subset of things (say, "UI3 has to coexist nicely with applications A,B,C") thus losing half of "from scratch" benefits

Certainly not. LMCE is a very properly tiered architecture. Only the GUI needs to be touched for UI3 to take over. We're talking about the Orbiter, in various platforms (standard Orbiter, Web Orbiter, mobile Orbiter, IP phone Orbiter, etc).

There are definitely some nuances and I am not completely aware about all of them, but for example - the standard Orbiter also does "windows" handling, e.g. when Xine_Player or Pluto_Screen_Server needs to be displayed, the proper window is brought on top, including all required positioning, resizing, etc. When creating the UI3 itself, this functionality will be probably separated into another tool, say magic_windows_manager. This means that either this magic_window_manager should be started only when UI3 is selected, or the same should happen for the UI2 and UI1 Orbiters. This what I mean "design changes propagation".

Now we're talking about backporting new features from UI3 to UI1/2. Seems not just a different discussion entirely, but also that it doesn't matter whether it's backported from SWF/GNASH or from anything else. Or that it's required at all. Why shouldn't the old UIs work the same way they always did?

Within the scope of just making a new, more featureful UI3, there shouldn't be any revisions propagated through the rest of the code. Unless perhaps there are new events etc supported/generated by UI3, which the DCErouter must support. But of course delivering required new features will be worth upgrading the DCErouter to accommodate them. Though the generic nature of DCE seems to make even that unlikely. AppServer upgrade, maybe, if there are new apps, but that's the architecture's way of handling such new features, which indeed minimizes the scope of the revisions. And besides, any approach to a UI3 that supports new features like that will require such propagation, not just if it's delivered in SWF. If anything, SWF is mature enough that developing such new features, and extending them to any new coding in scopes outside the Orbiter's code, will probably have existing techniques that will make development easier than "blue sky" development of UI3 from scratch.


- Regarding the IDE: don't care too much about the big and powerful IDE at the beginning of journey: if the development be successful, the IDE will appear. Of course, certain ideas should be taken into account, such as - it is always easier to modify human-readable file that to issue SQL queries. Like with websites - webdesigners doesn't use IDE 100% of time - they spend a lot of time in notepad-like text editors, tweaking the HTML,CSS,JS,etc. :)

It's irresponsible to replace a GUI with a new system without including a GUI designer (or allowing use of an existing one). Especially when one major limit of LMCE is that its old GUI designer is acknowledged totally unusable by both its users and its authors. This requirement is a driving force in producing a UI3. "It'll come" is no way to plan a project, especially when it'll have to come from us. We have to pick the framework that will most reliably include a GUI design/composition system, like a data format that already has one, or plan to write one ourselves.

If you design the GUI system well (including the data structures, elements layout formats, etc.) it will be not hard to write a custom IDE for it (when rendering of sample UIs is already present and data format is selected carefully). But I certainly wouldn't recommend to select the UI framework putting the availability of IDE for it. 99% of time people will see the UI3, not the framework for UI3 development, that's the logic.

In addition - yep, Adobe does have an SWF editors, etc. but is all this stuff free? Is it easily extensible/customizable so we can modify it? As in file formats, the XML is better than a binary blob - with editors/creating tools it is easy when you can use free tools to make a modification. If Joe needs to buy an IDE for $300 just to change color of 1 button, what will he say? :)

There are free tools for generating SWF, not limited to OpenLaszlo, and tools for generating other formats from SWF. I expect there will be some development necessary by LMCE to bundle them into an environment best suited to LMCE. Especially if we incorporate something like sqlCVS with templates for target Orbiter devices. But that would be faster than developing a whole new GUI studio environment, especially if on a whole new GUI format, and most especially while doing both at the same time with the same developers only from the LMCE community, rather than bundle SWF tools perhaps with help from the SWF community.


- Description of objects properties inheritance and conditional properties evaluation, reminds me about HTML+CSS style of UI implementation - does anybody else see this analogy? Maybe we could utilize it and define the engine layouts in kind of XML-ish format. And use CSS-analogue approach for defining stylesheets, too.

Whatever we decide to do to produce the UI improvements, it will probably include runtime configs that style the GUI. HTML+CSS is one example of that model, but is very limited (owing mostly to the limits of HTML).

We are not restricted by the HTML standard, I was talking about "analogy", not direct use of this technology. But it is a bit funny to hear about restriction of HTML+CSS from you at the same time when you speak about the OpenLaszlo which can be also compiled into DHTML.

Well, I was agreeing with your analogy. Though pointing out that one instance, HTML+CSS, is limited. Which is one reason why I have not been pushing the DHTML generation of OpenLaszlo. Those limitations of DHTML are why SWF itself is a better target for OpenLaszlo than is DHTML, except when SWF can't be supported in a device, but SWF can. FWIW, I've also mentioned the even more limited SVG, which could be suitable in even more constrained environments (or when the GUI doesn't need more than SVG).



I'm not really sure that you really understand GUI development, or LMCE's architecture. You also seem to be wrong about SWF and how it actually does relate to UI3 (or to Flash) on LMCE. Where are you getting these perspectives from?


As far as I know, the SWF and Flash are not related to LMCE (at least yet ;-). If you are aware about some freshly developed code - please update me on this.

Well, we are relating them now in this discussion (and elsewhere on that subject). You were claiming that SWF is really different from what UI3 needs, though Aaron.b disagreed with that claim (though he'd made it too, in the original wiki UI3 article) in replies in this post.

Regarding GUI, LMCE and perspectives - good question. I never developed full-scale GUI framework, but used different kinds of them: VCL, MFC, QT, so I know what it is and what are components of. LMCE - it's a fork of Pluto, right? I am familiar with Pluto system and even submitted some code to them. Of course I don't know _full_ details architecture as it is too large to fit in my head  ;D, but DCE is for Devices,Commands,Events - right? :)

Well, our disagreements about the scopes necessarily touched by UI3 development indicated that you didn't understand how LMCE's UI architecture worked. Evidently you do, so I'm still not sure why you said what you did about LMCE's UI architecture, claiming cross-tier dependency problems. I personally don't know how LMCE's architecture changed after it forked from Pluto, and whether perhaps there were some Pluto dependency problems like that that are now fixed in LMCE.

But I've been designing and implementing GUIs for a long time. Since the early 1980s. I helped clone a Photoshop beta into a "digital prepress" (pro desktop publishing) system for extremely hirez cameras (crossplatform Mac/Windows, pre-3.0). I was part of the team that helped Apple port the MacOS Toolkit from Pascal to C++ (for Apple Developer University), specifically in a code development tool with a hypermedia API. I've been part of so many evolutions (and revolutions) that I know how easy it is to talk about GUI architectures and requirements, vs how hard it is to be right. Conversely, I know how quickly GUI experts can pick up new ones, and the importance of reusing them. I'm not just trying to brag about my resume. I'm trying to point out how I know what I know about these issues, even though my expertise in specifically LMCE is still limited, my SWF/OpenLaszlo/GNASH experience is largely from reading webpages (though I've used some free SWF tools before). A good deal of getting this kind of thing right is both predicting how GUI techniques will evolve over a few years for developers and consumers, and how the various community lifecycles both enable and constrain that evolution. YMMV.
Title: Re: UI3 Discussion
Post by: shivizard on January 22, 2008, 04:48:59 am
Transparent Flash Control in Plain C++
http://www.codeguru.com/cpp/g-m/multimedia/graphics/article.php/c12229/

Just to reply on Kir's note - can use something like to above - ofcourse customize it to our MCE needs.

Flash should be a decent solution for UI3
Title: Re: UI3 Discussion
Post by: Matthew on January 22, 2008, 04:59:59 pm
Transparent Flash Control in Plain C++
http://www.codeguru.com/cpp/g-m/multimedia/graphics/article.php/c12229/

Just to reply on Kir's note - can use something like to above - ofcourse customize it to our MCE needs.

That's an OLE/ActiveX Windows component. LMCE is Linux.


Flash should be a decent solution for UI3

I agree.
Title: Re: UI3 Discussion
Post by: danielk on January 22, 2008, 05:50:27 pm
One of the things that came up at the meeting on Saturday was that the KDE folks are already planning to add gnash support to libplasma, so we should get flash objects for free by targeting libplasma.
Title: Re: UI3 Discussion
Post by: Matthew on January 22, 2008, 07:54:39 pm
One of the things that came up at the meeting on Saturday was that the KDE folks are already planning to add gnash support to libplasma, so we should get flash objects for free by targeting libplasma.

That revelation must have been part of the stretches that were inaudible to the Internet audience.

What does "GNASH support in libplasma" mean?
Title: Re: UI3 Discussion
Post by: coley on January 22, 2008, 11:24:11 pm
Are there notes or minutes from Saturdays meeting knocking around? or will it all come to light in this thread.

-Coley.

 >:( ISP issues had me offline for it
Title: Re: UI3 Discussion
Post by: shivizard on January 24, 2008, 09:35:09 am
Save the Planet; Disable Adobe Flash
http://www.oreillynet.com/onlamp/blog/2008/01/save_the_planet_disable_adobe.html

The article says that FLASH eats up a lot of CPU power - Somebody know how much this compares to custom opengl solution?
Title: Re: UI3 Discussion
Post by: Matthew on January 24, 2008, 05:59:30 pm
Save the Planet; Disable Adobe Flash
http://www.oreillynet.com/onlamp/blog/2008/01/save_the_planet_disable_adobe.html

The article says that FLASH eats up a lot of CPU power - Somebody know how much this compares to custom opengl solution?

The LMCE development strategy (especially for UI3) is to ensure simple user features first, then optimize efficiency for cheap HW and electrical efficiency later. The UI3 proposal is to use GNASH, not Adobe Flash.


Quote from: chromatic link=http://www.oreillynet.com/onlamp/blog/2008/01/save_the_planet_disable_adobe.html
I don’t use Adobe’s Flash player (partly because I’m no fan of poorly-coded proprietary software, but also partly because Adobe lies about its Linux support), but I’ve noticed that Flash on the machines of other people seems to make fans run, and Gnash eats up a lot of CPU too.

I'd prefer to see actual data other than just chromatic's anecdotal data point (also related to some "quick calculations" about Adobe's Flash player). Especially since a GNASH feature is its higher CPU performance than Adobe's Flash, which also typically means lower electric consumption for equal results.

Quote from: chromatic link=http://www.oreillynet.com/onlamp/blog/2008/01/save_the_planet_disable_adobe.html
I’m sure a couple of hours with debugging symbols and Powertop and the source code could improve things.

As energy efficiency becomes a higher LMCE priority, improvements to the FOSS GNASH can be made. chromatic's post is mainly an argument for Adobe to open the Flash source, which is already the case with GNASH. Since LMCE now has direct access to GNASH's Rob Savoye (head of the GNASH project), we should be able to put this issue into actual GNASH context.
Title: Re: UI3 Discussion
Post by: rrambo on February 06, 2008, 10:24:40 pm
I don't know if this is even a viable option...  But is Clutter a possibility for UI3?  http://clutter-project.org/  The UI for the Entertainer project is quite slick..  http://www.entertainer-project.com/

I'm not a programmer and don't even pretend to know much about something of this magnitude..  Just wondering...
Title: Re: UI3 Discussion
Post by: dopey on February 08, 2008, 04:41:48 am
I never heard of Clutter before now, but it does at the very least look like a step in the right direction.

Still I would like to the the meeting minutes so I know the direction that we are heading...

From what I understand the options are to use and build upon GNash, use and build upon another OSS project (Clutter would be in this category), or just start from scratch. From my understanding GNash is a no go and this would likely have to be started from scratch to deal with licensing.
Title: Re: UI3 Discussion
Post by: Matthew on February 08, 2008, 03:08:44 pm
I never heard of Clutter before now, but it does at the very least look like a step in the right direction.

Still I would like to the the meeting minutes so I know the direction that we are heading...

From what I understand the options are to use and build upon GNash, use and build upon another OSS project (Clutter would be in this category), or just start from scratch. From my understanding GNash is a no go and this would likely have to be started from scratch to deal with licensing.

What makes you think GNASH is a no go?
Title: Re: UI3 Discussion
Post by: totallymaxed on February 10, 2008, 06:44:52 pm
I never heard of Clutter before now, but it does at the very least look like a step in the right direction.

Still I would like to the the meeting minutes so I know the direction that we are heading...

From what I understand the options are to use and build upon GNash, use and build upon another OSS project (Clutter would be in this category), or just start from scratch. From my understanding GNash is a no go and this would likely have to be started from scratch to deal with licensing.

Look lets get this straight... GNASH is not a no go in any way at all. You just have to look at the range, sophistication breadth of UI's being delivered as 'Flash' applications now to realise this. In UI3 the plan is to use the GNASH runtime engine to provide the runtime engine.
Title: Re: UI3 Discussion
Post by: Matthew on February 10, 2008, 08:45:57 pm
I never heard of Clutter before now, but it does at the very least look like a step in the right direction.

Still I would like to the the meeting minutes so I know the direction that we are heading...

From what I understand the options are to use and build upon GNash, use and build upon another OSS project (Clutter would be in this category), or just start from scratch. From my understanding GNash is a no go and this would likely have to be started from scratch to deal with licensing.

Look lets get this straight... GNASH is not a no go in any way at all. You just have to look at the range, sophistication breadth of UI's being delivered as 'Flash' applications now to realise this. In UI3 the plan is to use the GNASH runtime engine to provide the runtime engine.

Has the UI3 discussion gone any further than the UI3 conference 3 weeks ago? Because I see no public sign that it has.
Title: Re: UI3 Discussion
Post by: bulek on February 15, 2008, 09:41:43 am
Hi.

don't know much about GUIs, but have spotted this news. Maybe it can be of some relevance :

http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html (http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html)

Regards,

Bulek.
Title: Re: UI3 Discussion
Post by: Matthew on February 15, 2008, 07:24:40 pm
don't know much about GUIs, but have spotted this news. Maybe it can be of some relevance :

http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html (http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html)

It looks like yet another GUI system from Microsoft. But it isn't even compatible with other Windows systems, including the ones that are HW accelerated. And its independent video player component is said to suck. Plus it will run only on Windows. Sounds like DOA to me.

But it is interesting news in the Windows world, even if not to LinuxMCE. Because it's an open source GUI toolkit from MS that will compete with their proprietary toolkits. Which might be why it's got so many problems. "Defective by design", perhaps?
Title: Re: UI3 Discussion
Post by: bulek on February 16, 2008, 12:38:18 am
don't know much about GUIs, but have spotted this news. Maybe it can be of some relevance :

http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html (http://www.team-mediaportal.com/news/global/mediaportal_ii_gets_opensource_xaml_skinengine.html)

It looks like yet another GUI system from Microsoft. But it isn't even compatible with other Windows systems, including the ones that are HW accelerated. And its independent video player component is said to suck. Plus it will run only on Windows. Sounds like DOA to me.

But it is interesting news in the Windows world, even if not to LinuxMCE. Because it's an open source GUI toolkit from MS that will compete with their proprietary toolkits. Which might be why it's got so many problems. "Defective by design", perhaps?
Not sure if I understood it right, but guy has written (and still is developing) open source variant that I guess will be much better over time than Microsoft's proprietary solution. Guys from Media Portal are doing great job, they are also renewing GUIs... MAybe something can come up from both efforts...

Regards,

Bulek.
Title: Re: UI3 Discussion
Post by: jester on February 25, 2008, 12:47:41 pm
Not really my field, but: Mozilla Prism (http://wiki.mozilla.org/Prism)? (The open source opponent of: Adobe Air / Microsoft Silverlight)
Title: Re: UI3 Discussion
Post by: shivizard on April 07, 2008, 05:07:52 am
Hi All, How come this UI3 route is moving so very slowly - if not moving at all. I am checking this thread everyday to see some action. Can we have the tasks organized and responsibilities assigned - Lets stick to one solution and move ahead.
Title: Re: UI3 Discussion
Post by: shivizard on April 15, 2008, 05:40:34 am
Is this Thread Abandoned? UI3 - Mission Abandoned?
Title: Re: UI3 Discussion
Post by: tschak909 on April 15, 2008, 06:32:43 am
We are very busy trying to get the 0710 release done.

-Thom
Title: Re: UI3 Discussion
Post by: colinjones on April 15, 2008, 06:34:37 am
Thom - does that mean it isn't going to RC first after all? Straight to final release?
Title: Re: UI3 Discussion
Post by: tschak909 on April 15, 2008, 06:36:53 am
Daniel will annouce the RC very soon now, he is attempting a build this evening.

-Thom
Title: Re: UI3 Discussion
Post by: tschak909 on April 15, 2008, 06:40:54 am
I notice there are those of you, who seem a bit antsy about wanting UI3....

I also notice that these people haven't even used Designer, to do any work in UI1, or UI2....

and yet these people want to discuss future plans for an interface without taking what we already have into account?

sounds pretty silly to me.... why weren't you at the Designer conference?

Why haven't any of you downloaded Designer to use it?

If you want something to do, Do that!

Then maybe you might be in a position to better discuss future user interface improvements.

-Thom
Title: Re: UI3 Discussion
Post by: ddamron on April 15, 2008, 06:44:26 am
Thom,

I whole heartedly agree.  The UI2 layout is very well laid out.  I would think a understanding of how to use UI2 is indeed a requirement to start brainstorming about UI3

My thoughts,

Dan
Title: Re: UI3 Discussion
Post by: wsuetholz on May 23, 2008, 08:18:14 pm
Hello,
  Most of the newer fancy interfaces I've been seeing are using python, and GStreamer for the media playback.

  The Elisa media center is using python with a Fluendo built render..  Entertainer is using Python with Clutter, and Gloss http://code.google.com/p/gloss-mc/ (http://code.google.com/p/gloss-mc/) is a new MythTV frontend coded in Python using Clutter.  All three of the above are using GStreamer.  Gloss in particular looks fairly interesting.
 
  Does the DCERouter have any python bindings?

  On another note:  The documentation can be fairly vague about how the various pieces of LinuxMCE interact.  Is there a guide that would assist people with understanding the UI interactions?

Title: Re: UI3 Discussion
Post by: tschak909 on May 23, 2008, 08:28:01 pm
Hello,
  Most of the newer fancy interfaces I've been seeing are using python, and GStreamer for the media playback.

  The Elisa media center is using python with a Fluendo built render..  Entertainer is using Python with Clutter, and Gloss http://code.google.com/p/gloss-mc/ (http://code.google.com/p/gloss-mc/) is a new MythTV frontend coded in Python using Clutter.  All three of the above are using GStreamer.  Gloss in particular looks fairly interesting.
 
  Does the DCERouter have any python bindings?

  On another note:  The documentation can be fairly vague about how the various pieces of LinuxMCE interact.  Is there a guide that would assist people with understanding the UI interactions?



I would suggest....again... (people don't seem to listen when I suggest things.)... to start with HADesigner. My HADesigner screencasts in the Developers forum here: http://forum.linuxmce.org/index.php?topic=5059.0

This will let you know how the UI interacts with the system below, as I go through every square inch of designing UIs for my MAME media type FOR ALL the UI targets.

Designer and Orbiter work hand in hands. Questions in Designer are often answered by looking at Orbiter's source code....

and, AGAIN....

it is NOT a good idea to generally say, "we should do this and this, well why can't we?" before learning and understanding the underlying system, as there is a specific reason that Orbiter was designed the way it was, (the ability to handle multiple UI targets sanely).

-Thom
Title: Re: UI3 Discussion
Post by: alexman on June 20, 2008, 02:35:56 pm
I want something like this (http://www.sooloos.com/www/downloads/Sooloos-ControlOne.jpg)
And i think many will.
Title: Re: UI3 Discussion
Post by: jondecker76 on June 20, 2008, 05:24:21 pm
The more I dig into the guts of LMCE and the more I understand, the more I am happy keeping things just the way they are. The current system is very robust and powerful, and capable of a lot more than what we are seeing now. Why fix something that isn't broken? Why fix something that isn't currently being used to its potential?

I think we would be better off building better skins, adding a few screens and changing current layouts. After all, we are in COMPLETE control of what our UI will look like in Designer. Lets use what we have got, and just build a new UI with it. (If you don't like the big-ugly-button look of UI1, learn how to use designer and design something better looking - it sure would be a lot easier than recoding the thing from the ground up, and being back in the same place of having to design the graphics for it)
Title: Re: UI3 Discussion
Post by: tschak909 on June 20, 2008, 05:38:57 pm
I agree there, with one addendum... use the knowledge we get from all of the above this to build a better UI3. :-)

-Thom
Title: Re: UI3 Discussion
Post by: jondecker76 on June 20, 2008, 07:30:19 pm
yes, no doubt someday it will be needed. I just don't think that day is today... Or tomorrow

As you said, we can learn a lot from working with what we have now, so we have an even better understanding of what to watch out for when the time comes that we do need to tackle an entirely new UI schema
Title: Re: UI3 Discussion
Post by: tschak909 on June 20, 2008, 07:39:13 pm
I do want to start work on a new iteration of UI for Orbiter, which I call UI2.5. I have been talking with m1cha3l in the #linuxmce channel, started while we were at LinuxTag, but I want to seriously start work with people who are watching the screencasts, and have an idea of the tools we are working with.

-Thom
Title: Re: UI3 Discussion
Post by: jondecker76 on June 20, 2008, 08:05:23 pm
I would be interested as well. My time is a little limited right now, but will be freeing up a lot soon.

If we could come up with a good "theme" and a couple of mock-ups, then we could split the screens up between everyone involved (if we get enough people we may only need to do 500-1000 screens each!) We would definitely need the help of a good artist.
Title: Re: UI3 Discussion
Post by: HorstJ on June 23, 2008, 10:21:27 am
The possibilities LinuxMCE offers are enough for a good UI to be build (in my opinion), but maybe we should change something anyhow. Before building new UIs, it could be a good idea to seperate design and function of a DesingObject. Thus would allow a much easier reuse and a faster creation of new "skins"/UIs, I think.
Title: Re: UI3 Discussion
Post by: samuelmukoti on July 04, 2008, 03:05:13 pm
Hi,

I went through the videos a couple of days ago! And was very impressed with whats there already!  I believe with some guidance I cold work with my graphical artist (from our Company) and try do some mockups of a few ideas. 

I wanted to ask about animation, i believe i saw an "animate" checkbox in the DesignObj options, is so how does that work, and where and how has it been used in the current UIs.

Lastly, where can i get info about UI 2.5, would love to get involved, need somewhere to start, could help with Artwork maybe, and hopefully get into bare metal soon enough..

regards,

Sam

Title: Re: UI3 Discussion
Post by: tschak909 on July 04, 2008, 08:51:53 pm
Currently, yes.. any Bitmap designObj can use an MNG file as its image.. However, we have to be _VERY_ careful here...because OrbiterGen literally disassembles each frame, and renders a fully composited set of images containing each frame.... this essentially means, that strange rendering artifacts can happen if you use MNG graphics near user interface elements like Buttons for example.

As for getting into the UI2.5 discussion, this is best handled on the IRC channel.

-Thom
Title: Re: UI3 Discussion
Post by: los93sol on July 11, 2008, 04:00:51 pm
Is there anywhere I can just download HADesigner and whatever else I need to start making a new UI for LMCE?  I'm willing to watch the screencasts and learn, but I don't wanna muck around for two days compiling stuff just to start working and learning.
Title: Re: UI3 Discussion
Post by: tschak909 on July 11, 2008, 04:20:24 pm
"I wanna learn, but I wanna do it now!"

patience, my son.

you can get a copy of Designer, from the wiki.
http://wiki.linuxmce.org/index.php/Installing_HADesigner

-Thom
Title: Re: UI3 Discussion
Post by: eNoodle on July 11, 2008, 06:09:36 pm
@tschak

If the OrbiterGen is generating all the screen, wouldn't it be possible to not generate images, but XML files?

These files could then be used with a kind of Proxy-Orbiter to build an Ajax Orbiter or even a Flash Orbiter. Am I totally wrong here?
Title: Re: UI3 Discussion
Post by: ineedme on December 06, 2008, 02:15:57 am
Why doesnt make a new UI using SVG not trying to use Flash or similar
Another good alternative is Ajax
Title: Re: UI3 Discussion
Post by: tschak909 on December 06, 2008, 02:49:07 am
Have you not paid attention to this thread at all?

*hmm*

I'm not going to explain it.

Go watch my screencasts, and you'll see why this isn't possible. Don't post again until you're able to make an intelligent argument.

-Thom