LinuxMCE Forums

General => Feature requests & roadmap => Topic started by: garagevibes on October 14, 2016, 05:37:36 pm

Title: Alexa and amazon echo skill
Post by: garagevibes on October 14, 2016, 05:37:36 pm
Hi all,

The amazon echo has final made it way to the UK and I'm excited to get my hands on one, just wondering if anyone had looked in to adding a Linuxmce skill to control basic home automation like lights and heating ect, it seems like someone has created a skill for our slightly common sister vera but it annoyingly connects via there in house web interface so even though the code has familiar terminology like Device Ids ect it just wont simply port over.

the amazon alexa system looks simple enough at creating a skill using node.js the issue i have is just trying to think of the best way of implementing a secure external api type interface so that the external amazon alexa voice service can send and receive commands to our internal DCErouter.

Just thought id put it out there and see if anyone had looked into this or had any ideas of the best way of implementing a solution.

cheers
Title: Re: Alexa and amazon echo skill
Post by: posde on October 14, 2016, 09:41:41 pm
golgoj4 has been looking into it. Maybe he can chime in.
Title: Re: Alexa and amazon echo skill
Post by: golgoj4 on October 14, 2016, 11:19:21 pm
Hi all,

The amazon echo has final made it way to the UK and I'm excited to get my hands on one, just wondering if anyone had looked in to adding a Linuxmce skill to control basic home automation like lights and heating ect, it seems like someone has created a skill for our slightly common sister vera but it annoyingly connects via there in house web interface so even though the code has familiar terminology like Device Ids ect it just wont simply port over.

the amazon alexa system looks simple enough at creating a skill using node.js the issue i have is just trying to think of the best way of implementing a secure external api type interface so that the external amazon alexa voice service can send and receive commands to our internal DCErouter.

Just thought id put it out there and see if anyone had looked into this or had any ideas of the best way of implementing a solution.

cheers

Whats your development skillset?
I was looking into creating some interfaces for this, but it needs to be a 2 part system.

i have been working on work and an xbmc device so ive been busy. but if you are serious, we can attack this problem and get something basic like light control going.

-lb
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on October 24, 2016, 10:58:58 am
Hey sorry for the delay had a few family issues, my echo arrived yesterday though so really keen to have a go at setting this up, my skill set is in web design really so I'm sure i could help at least with the http interface and designing of the api. my main struggle would probably be with the actual connecting up to the dcerouter, how to announce available devices ect.

You mentioned you found some nice code from google that would handle this don't suppose you could send a link  and ill see if I can get this intergraded as a starting point maybe?

cheers
Title: Re: Alexa and amazon echo skill
Post by: phenigma on October 24, 2016, 04:40:09 pm
I currently have Alexa running on an RPi and I am also interested in connecting with LMCE.  Any existing info people have would be great to share here.  I'm still trying to figure out how we would interface with it.  As far as I can tell once we have a secure authentication that happens automatically I think we simply need to create a 'skill' for Alexa to interface to LMCE, and then an associated LMCE device/plugin to handle interaction.

J.
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on October 27, 2016, 11:19:48 am
Im still playing around with this, from a lot of reading up on the amazon developer site i think I've now created a basic skill that I can enable and send and receive requests from but the responses are NULL as I haven't actually done any linking to linuxmce yet thats the next step.

Did i see once on this forum that someone had created a JSON dcedevice before? I don't seem to be able to find it or any documentation on the api for it, just thought it might be an easy way to plug in.

Otherwise ill probably need to create my own scripts, but unfortunately my only coding skills are in php, javascript and html so id probably write an interface in php using a connection to the mysql db to get the details of controllable devices and there device ids etc and then use the same script to fire off exec commands to send the required dcemessages as they are received from amazon.

Obviously i need to add in a secure connection between the core and the amazon web service as well.

Does this seem the right approach or at last a workable approach?
Title: Re: Alexa and amazon echo skill
Post by: phenigma on October 27, 2016, 02:23:02 pm
Golgo has already made a lot of additions to enable the secure login/connection.  Likely an addition needs to be made for the Amazon login/authentication vs. google/etc.

There was a JSON device created, it's quite limited iirc.  As for an lmce interface, my opinion is that it should be a proper plug-in to the dcerouter so that the interface has access to lmce information and data.  This will require a c++ plug-in to be written.  My opinion is that it would be counter productive to move forward any other way, except for initial testing.  Others may have different opinions.

How do we access the skill you have created?  i.e. is it available for use/testing?

Jason.
Title: Re: Alexa and amazon echo skill
Post by: golgoj4 on October 27, 2016, 06:55:22 pm
High Level observations after working with google, smartthings

*Does every lmce box now support https connections?
*We cannot do this for only one service. We must develop a linuxmce web api. period! The upside is all these cloud connected services can utilize this one api, so we only have to do it the one time. Until they invent the next thing besides json...

My showstopper: user credentials. If you think about it for a second, we would basically be storing very sensitive account information for multiple services for a multi-tude of people. That makes users a very juicy target, plus we need to respect privacy between users. Im sure people know how to do this, I dont have much experience there but im confident as a group we could figure it out. kids shouldn't be able to sneak into the parents google account via linuxmce db diving, etc.

Ive settled on using cakephp in addition to webadmin. I know most people would think that this is something that would be rolled into the webadmin, but frankly the webadmin is big, ugly and it doesn't make sense from my standpoint to tack on a massive feature like this. cakephp would be able to  integrate into our existing database with some work but it provides a nice clean framework to provide a gui frontend if we choose, but a json api next to it from the same codebase.

I should also add that a c++ adapter would be in the mix purely for passing event into the linuxmce system (unless something better already exists) for sending events / anything else not directly connected to web api.

Kudos to you guys for also attacking this!

golgoj4
Title: Re: Alexa and amazon echo skill
Post by: posde on October 27, 2016, 09:44:54 pm
A quick note on the JSON Plugin (there is a wiki page about it). Its sole purpose is to provide information to the outside world that lives within DCERouter and can't be accessed using database lookup means, namely, Now Playing and Media Playlist - both are not stored anywhere, so a plugin was the only way to access it. Thanks go to Radu for creating it.
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on November 02, 2016, 09:42:25 am
Ok so after some more digging to secure our API we need to use something like OAuth2 and in fact to use it with alexa you can only use OAuth2. There are a number of ways of doing this as I'm quite fluent in php id probably prefer to go down that route. I haven't used cakePHP before but I've heard of it and ill gladly take a look especially if it simplifies the DB connections, I'm still getting my head round the changes in the depreciated mysql php functions which I've become accustomed to for years :).

There is a nice example and demo of implementing an OAuth2 server over on this site: https://bshaffer.github.io/oauth2-server-php-docs/ I don't think it would be to much work to implement this on to the core.

In terms of security I believe keys are stored encrypted anyway so a person with access to the Linuxmce DB wouldnt get simple access to peoples accounts but if it really is an issue I guess a new schema just for this could be created with a password or something? But this is about giving an application like Alexa access to LinuxMCE not google giving LinuxMCE access to google if that makes sense.

Another option would be to implement a framework like this: https://apigility.org designed to help create api's and offers a few different security options for securing them, this would give better flexibility and future proof with regular updates but there are obviously disadvantages as well relying on yet another 3rd party piece of software.

Im confident i can implement any one of these on to the core to secure our apis the next stage would then i guess be what do we want our api to look like what commands would we send and what would our structured responses look like to give the best usability and flexibility with multiple services?

Ill continue digging and see if i can implement something on my core here just as a test and come back to you all. Cheers for the help and advice as well, I will be back I'm sure  ;)
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on November 29, 2016, 11:44:57 am
Hi all,

Sorry this has taken me a while to get going but it is still on my radar, I've got an oauth2 server running now on my core and looks to be working fine and issuing out tokens and all that nice stuff. I've brought a cheap domain name in the black friday sales and linked that into my core and installed an SSL certificate as well which is also working fine however after 2 days of pulling my hair out trying to get amazon to accept and link my skill to my core it seems that amazon are extremely picky on what SSL certs they will accept from what i can see it has to come from either, digicert, versign, twane or one of the other over overpriced big players you cant use just any old cheap certificate that you can get nowadays I got mine from godaddy but i tried a trial for geotrust as well and that also wouldn't work.

To verify that it was a certificate issue I copied my code over from my core on to one of the web servers at my work which has an enhanced certificate from digicert and it linked fine so this leads me to a problem either we all have to buy a £250+ a year SSL certificate and link an amazon skill to each of our cores so that there is no concern in security or the other option I'm looking at is develop a web interface on a server with an expensive certificate and then use this as a middle man to receive the commands from amazon or any other similar service link them to a user in a DB which has that persons core IP address or domain name and then route the command to there core.

To keep this as secure as possible ill still use Oauth2 on the core to link to this intermediate site but be a little more lenient on the certificate type allowing like a selfsigned one the good thing about this as well will be that the skill I produce should be allowed to be published publicly as its linking to a single secured service and although there will be concerns maybe with a third party site holding core details by using oauth2 a user can revoke access at any point.

To test this approach I'm making the intermediate site on a server i have access to at work where I'm currently building an internal web application, This will be fine to test but it cant obviously stay there for all to connect to I cant see my manager being to impressed but if I can get this working and others find it useful then perhaps we can group fund a certificate for the linuxmce.org domain and move the page built over to this server I think people would full more secure that way as well.

But anyway just thought id give an update and let people know its not been forgotten about.

Cheers
Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 01, 2016, 02:11:21 pm
Just a note, an OAuth2 server was just added to LMCE for initial integration and testing.  It'd be great if you could now shift to using the provided server and ensure it is functioning properly.  This server will be used for multiple cloud service authentications so needs to remain generic. 

Keep on playing and having fun!
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on December 06, 2016, 11:28:15 am
So last night the magic happened 8)
I had Alexa discovering all my lighting devices from my core, added them to the Alexa app and more importantly spent the night pissing off my family by repeatedly requesting Alexa to turn my living room lights on, off and up and down. It was amazing.

Now i need to tidy up my code mess along the way and strengthen the oauth2 server I added to my core as at the moment for testing there is no password or anything to authorise which kind of defeats the point but I note phenigma that you said we now have one already on LinuxMCE? I've updated and I cant see any sign of one? I'm happy to use the one built in one though I mean who whats 2? :) its a basic web standard so should require no more than a url change in my code to switch anyway.

At the moment I've only linked in lights but over the next few days ill try and add thermostats and scenarios although I only have a z-wave boiler control not a thermostat so may be in the dark a bit there so may need some help to test.

I'm excited about scenarios but I've read that amazon will only allow up to 12 "scenes" as they call them to be discovered which may be a pain.

Ive tried my best to make the api as simple and flexible as possible and not have it only for Alexa but indeed any web service its a bit rough at the moment so will probably no doubt need changes and tweaks that I'm sure all you guys on here can offer advice and assistance on.

At the moment because I'm using my own oauth2 server on the core the interface requires the following:
Note also that the intermediate site I've built will store your cores IP address and the required access tokens there is no way around this as I said before this site should i believe be stored on this linuxmce.org domain securely but for the moment I have it on a test site I can promise you Im not interested in accessing your core, but at the moment this is the only way I can see it being possible, you still have full control of the outh server on your core and revoke all access at any point.

So anyway I guess the fun question is once I've ironed out the few bits today who has a system they don't mind playing with that wants to test this?
Title: Re: Alexa and amazon echo skill
Post by: posde on December 06, 2016, 02:33:18 pm
Great news! Check with polly here on the forum to get some git credentials. We don't want to loose stuff.
Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 06, 2016, 04:37:27 pm
I'll pass on the existing LMCE OAuth2 information when I get home later today.  You will be the first to use it so it is very likely to not be completely set up properly.  I will leave it to you to let me know what needs to be changed/altered in gitlab issues.  Thanks for this!

The other thing that we're going to need to look at is setting up a proper LMCE interface with Alexa.  I understand you have created some 'skills' or whatever they are called on the Amazon side and we'll need to get an official LMCE setup with Amazon.  We can discuss this more as we go forward.

My intent is to implement Alexa on the RPi for use with LMCE, although my time is split between a whole lot of things.

Thanks again!  Don't forget to get some git credentials and we'll setup a branch to store this stuff in.

J.
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on December 06, 2016, 05:32:36 pm
Thanks guys for getting back to me, Yes phenigma if you could send over the details of the LMCE oauth2 server details ill get it integrated into that and ill ask polly for some git login details so I can store the code there.

I've created an interface for a central web api portal which is used an intermediate for web services to connect to so for the moment the system works a bit like the quick diagram I've attached.

So there are basically 2 Oauth2 servers in use 1 on the core and 1 one this new web portal the idea being that a number of different web services can connect through the same web portal and it will route the messages to the correct users core and a user can break the chain at either end at any point if they ever need to, this means also that any app we produce can hopefully be released in to the public store's.

Looking on the amazon developer console I believe i can add people as developers to my account if I have there email address so they can add the skill for testing and view the code behind it, the web portal site is simply a router I wanted to keep all the api itself on each users core so that updates and things will stay as they are and hopefully the site will never have to change.

Ive had the idea of including alexa as a voice service as well but thats another task once I have this one solved :)
                                 
Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 06, 2016, 05:42:44 pm
Awesome stuff!  I think you're right in line with conversations in the irc dev channel!  I'll get you some more info and a git account if polly/ochorocho hasn't been able to yet.  Send me a PM with your name/username/email and I'll get an account up.

Thanks!

J.

Sent from my Nexus 5 using Tapatalk

Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 06, 2016, 06:12:52 pm
Okay the OAuth2 implementation that was chosen some time ago is described here: https://bshaffer.github.io/oauth2-server-php-docs/cookbook/

I believe the appropriate files are all packaged into the lmce package 'lmce-oauth2-server'.  "apt-get install lmce-oauth2-server" on a core.

This will install the OAuth2 files in /var/www/lmce-admin/include/OAuth2/ and two files "oauth2-server.php" and "oauth2-token.php" in /var/www/lmce-admin/.  When the package installs it creates the database and the appropriate tables.

That's as far as that has made it.  Anything can change as far as configs, etc.

J.
Title: Re: Alexa and amazon echo skill
Post by: golgoj4 on December 06, 2016, 06:34:57 pm
link to code?
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on December 06, 2016, 08:48:19 pm
Haha hilarious it just so happens thats the exact oauth2 server scripts I've been using anyway so should be quite simple to change over to this and my steps to add in the database tables wont be needed either.

I'll just have to change some of the required paths, Ill give this a go and see how i get on once thats done ill upload all the code. I haven't contacted polly yet for a git account ill do that in a min.

cheers all.
Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 06, 2016, 08:49:07 pm
:D

Sent from my Nexus 5 using Tapatalk

Title: Re: Alexa and amazon echo skill
Post by: garagevibes on December 07, 2016, 11:00:52 am
Hi phenigma,

Ive removed the parts I've added and now installed the new built in OAuth2 server, it all went through fine but it doesn't look to have created the database? hunting through the code I can see there should now be a database called lmce_oauth2_db but I don't seem too be able to find it in my system? I can manually create it for the time being but just to give you a heads up.

Also in the 'oauth2-server.php' file there needs to be the following line added to the bottom otherwise you wont get refresh tokens given and that will be really rubbish if we have to relink our systems every hour when the token expires :) I set the refresh expiration to 14 days here this means that you will have to relink if no commands are sent for more that 14 days? This may want to be extended or made indefinitely.

Code: [Select]
// Add the "Refresh Token" grant type
$server->addGrantType(new OAuth2\GrantType\RefreshToken($storage, array(
    'always_issue_new_refresh_token' => true,
'refresh_token_lifetime'         => 2419200
)));

Whats the best way of going about this just make all the changes need to make this work on these scripts and then upload then to git or zip them and post them? Or am I better just sending you each amendment, I don't think there are that many?
Title: Re: Alexa and amazon echo skill
Post by: phenigma on December 07, 2016, 02:16:34 pm
Neato.  Definitely create an Issue in gitlab to track this stuff.  For minor changes you can simply pass the info or files to me.  I'll try to have a look at these initial issues when I get home from work later today.  Thanks!

J.
Title: Re: Alexa and amazon echo skill
Post by: garagevibes on January 11, 2017, 03:52:48 pm
Hi phenigma,

Hope you had a merry xmas and a good new year, sorry I've left this for a few weeks over christmas and things.

Im now all up on git and ready to post some initial code but if I'm honest I've never used it before and i have no clue what I'm doing.

I'm assuming I have to clone the current code, add my amendments and adjustments and add in any new files then submit to have it added in to the live repository but I really cant figure it out. I'm probably just being stupid but I seem to be able to create my own projects and add issues into Linuxmce's repo but I don't know if I'm missing something.

Do we have a guide anywhere?

Cheers
 
Title: Re: Alexa and amazon echo skill
Post by: phenigma on January 11, 2017, 05:16:18 pm
Hmm.  We haven't made a how-to since moving to git.  Essentially you'll clone the repo, create a new branch, add/change code in that branch, push that updates to the new branch.  Someone can have a look to ensure things look good and in proper locations.  Then it can be merged into the master branch so it is built.  There are lots of how-to for using git from the command line, if you need specific assistance then feel free to ask in -devel. 

Sent from my Nexus 5 using Tapatalk