Alexa and amazon echo skill

[garagevibes]

Hi all,

The amazon echo has final made it way to the UK and I'm excited to get my hands on one, just wondering if anyone had looked in to adding a Linuxmce skill to control basic home automation like lights and heating ect, it seems like someone has created a skill for our slightly common sister vera but it annoyingly connects via there in house web interface so even though the code has familiar terminology like Device Ids ect it just wont simply port over.

the amazon alexa system looks simple enough at creating a skill using node.js the issue i have is just trying to think of the best way of implementing a secure external api type interface so that the external amazon alexa voice service can send and receive commands to our internal DCErouter.

Just thought id put it out there and see if anyone had looked into this or had any ideas of the best way of implementing a solution.

cheers

[Marie.O]

golgoj4 has been looking into it. Maybe he can chime in.

[golgoj4]

Hi all,

The amazon echo has final made it way to the UK and I'm excited to get my hands on one, just wondering if anyone had looked in to adding a Linuxmce skill to control basic home automation like lights and heating ect, it seems like someone has created a skill for our slightly common sister vera but it annoyingly connects via there in house web interface so even though the code has familiar terminology like Device Ids ect it just wont simply port over.

the amazon alexa system looks simple enough at creating a skill using node.js the issue i have is just trying to think of the best way of implementing a secure external api type interface so that the external amazon alexa voice service can send and receive commands to our internal DCErouter.

Just thought id put it out there and see if anyone had looked into this or had any ideas of the best way of implementing a solution.

cheers

Whats your development skillset?
I was looking into creating some interfaces for this, but it needs to be a 2 part system.

• Simple http interface for doing the initial security handshake. google has some nice code for doing this, i did it successfully in my tests. It would need to be handled per cloud service: Alexa, Smartthings, google, iftt, etc
• A proper http api exposing proper endpoints for these things to communicate.

i have been working on work and an xbmc device so ive been busy. but if you are serious, we can attack this problem and get something basic like light control going.

-lb

[garagevibes]

Hey sorry for the delay had a few family issues, my echo arrived yesterday though so really keen to have a go at setting this up, my skill set is in web design really so I'm sure i could help at least with the http interface and designing of the api. my main struggle would probably be with the actual connecting up to the dcerouter, how to announce available devices ect.

You mentioned you found some nice code from google that would handle this don't suppose you could send a link  and ill see if I can get this intergraded as a starting point maybe?

cheers

[phenigma]

I currently have Alexa running on an RPi and I am also interested in connecting with LMCE.  Any existing info people have would be great to share here.  I'm still trying to figure out how we would interface with it.  As far as I can tell once we have a secure authentication that happens automatically I think we simply need to create a 'skill' for Alexa to interface to LMCE, and then an associated LMCE device/plugin to handle interaction.

J.

[garagevibes]

Im still playing around with this, from a lot of reading up on the amazon developer site i think I've now created a basic skill that I can enable and send and receive requests from but the responses are NULL as I haven't actually done any linking to linuxmce yet thats the next step.

Did i see once on this forum that someone had created a JSON dcedevice before? I don't seem to be able to find it or any documentation on the api for it, just thought it might be an easy way to plug in.

Otherwise ill probably need to create my own scripts, but unfortunately my only coding skills are in php, javascript and html so id probably write an interface in php using a connection to the mysql db to get the details of controllable devices and there device ids etc and then use the same script to fire off exec commands to send the required dcemessages as they are received from amazon.

Obviously i need to add in a secure connection between the core and the amazon web service as well.

Does this seem the right approach or at last a workable approach?

[phenigma]

Golgo has already made a lot of additions to enable the secure login/connection.  Likely an addition needs to be made for the Amazon login/authentication vs. google/etc.

There was a JSON device created, it's quite limited iirc.  As for an lmce interface, my opinion is that it should be a proper plug-in to the dcerouter so that the interface has access to lmce information and data.  This will require a c++ plug-in to be written.  My opinion is that it would be counter productive to move forward any other way, except for initial testing.  Others may have different opinions.

How do we access the skill you have created?  i.e. is it available for use/testing?

Jason.

[golgoj4]

High Level observations after working with google, smartthings

*Does every lmce box now support https connections?
*We cannot do this for only one service. We must develop a linuxmce web api. period! The upside is all these cloud connected services can utilize this one api, so we only have to do it the one time. Until they invent the next thing besides json...

My showstopper: user credentials. If you think about it for a second, we would basically be storing very sensitive account information for multiple services for a multi-tude of people. That makes users a very juicy target, plus we need to respect privacy between users. Im sure people know how to do this, I dont have much experience there but im confident as a group we could figure it out. kids shouldn't be able to sneak into the parents google account via linuxmce db diving, etc.

Ive settled on using cakephp in addition to webadmin. I know most people would think that this is something that would be rolled into the webadmin, but frankly the webadmin is big, ugly and it doesn't make sense from my standpoint to tack on a massive feature like this. cakephp would be able to  integrate into our existing database with some work but it provides a nice clean framework to provide a gui frontend if we choose, but a json api next to it from the same codebase.

I should also add that a c++ adapter would be in the mix purely for passing event into the linuxmce system (unless something better already exists) for sending events / anything else not directly connected to web api.

Kudos to you guys for also attacking this!

golgoj4

[Marie.O]

A quick note on the JSON Plugin (there is a wiki page about it). Its sole purpose is to provide information to the outside world that lives within DCERouter and can't be accessed using database lookup means, namely, Now Playing and Media Playlist - both are not stored anywhere, so a plugin was the only way to access it. Thanks go to Radu for creating it.

[garagevibes]

Ok so after some more digging to secure our API we need to use something like OAuth2 and in fact to use it with alexa you can only use OAuth2. There are a number of ways of doing this as I'm quite fluent in php id probably prefer to go down that route. I haven't used cakePHP before but I've heard of it and ill gladly take a look especially if it simplifies the DB connections, I'm still getting my head round the changes in the depreciated mysql php functions which I've become accustomed to for years .

There is a nice example and demo of implementing an OAuth2 server over on this site: https://bshaffer.github.io/oauth2-server-php-docs/ I don't think it would be to much work to implement this on to the core.

In terms of security I believe keys are stored encrypted anyway so a person with access to the Linuxmce DB wouldnt get simple access to peoples accounts but if it really is an issue I guess a new schema just for this could be created with a password or something? But this is about giving an application like Alexa access to LinuxMCE not google giving LinuxMCE access to google if that makes sense.

Another option would be to implement a framework like this: https://apigility.org designed to help create api's and offers a few different security options for securing them, this would give better flexibility and future proof with regular updates but there are obviously disadvantages as well relying on yet another 3rd party piece of software.

Im confident i can implement any one of these on to the core to secure our apis the next stage would then i guess be what do we want our api to look like what commands would we send and what would our structured responses look like to give the best usability and flexibility with multiple services?

Ill continue digging and see if i can implement something on my core here just as a test and come back to you all. Cheers for the help and advice as well, I will be back I'm sure 

[garagevibes]

Hi all,

Sorry this has taken me a while to get going but it is still on my radar, I've got an oauth2 server running now on my core and looks to be working fine and issuing out tokens and all that nice stuff. I've brought a cheap domain name in the black friday sales and linked that into my core and installed an SSL certificate as well which is also working fine however after 2 days of pulling my hair out trying to get amazon to accept and link my skill to my core it seems that amazon are extremely picky on what SSL certs they will accept from what i can see it has to come from either, digicert, versign, twane or one of the other over overpriced big players you cant use just any old cheap certificate that you can get nowadays I got mine from godaddy but i tried a trial for geotrust as well and that also wouldn't work.

To verify that it was a certificate issue I copied my code over from my core on to one of the web servers at my work which has an enhanced certificate from digicert and it linked fine so this leads me to a problem either we all have to buy a £250+ a year SSL certificate and link an amazon skill to each of our cores so that there is no concern in security or the other option I'm looking at is develop a web interface on a server with an expensive certificate and then use this as a middle man to receive the commands from amazon or any other similar service link them to a user in a DB which has that persons core IP address or domain name and then route the command to there core.

To keep this as secure as possible ill still use Oauth2 on the core to link to this intermediate site but be a little more lenient on the certificate type allowing like a selfsigned one the good thing about this as well will be that the skill I produce should be allowed to be published publicly as its linking to a single secured service and although there will be concerns maybe with a third party site holding core details by using oauth2 a user can revoke access at any point.

To test this approach I'm making the intermediate site on a server i have access to at work where I'm currently building an internal web application, This will be fine to test but it cant obviously stay there for all to connect to I cant see my manager being to impressed but if I can get this working and others find it useful then perhaps we can group fund a certificate for the linuxmce.org domain and move the page built over to this server I think people would full more secure that way as well.

But anyway just thought id give an update and let people know its not been forgotten about.

Cheers

[phenigma]

Just a note, an OAuth2 server was just added to LMCE for initial integration and testing.  It'd be great if you could now shift to using the provided server and ensure it is functioning properly.  This server will be used for multiple cloud service authentications so needs to remain generic. 

Keep on playing and having fun!

[garagevibes]

So last night the magic happened
I had Alexa discovering all my lighting devices from my core, added them to the Alexa app and more importantly spent the night pissing off my family by repeatedly requesting Alexa to turn my living room lights on, off and up and down. It was amazing.

Now i need to tidy up my code mess along the way and strengthen the oauth2 server I added to my core as at the moment for testing there is no password or anything to authorise which kind of defeats the point but I note phenigma that you said we now have one already on LinuxMCE? I've updated and I cant see any sign of one? I'm happy to use the one built in one though I mean who whats 2? its a basic web standard so should require no more than a url change in my code to switch anyway.

At the moment I've only linked in lights but over the next few days ill try and add thermostats and scenarios although I only have a z-wave boiler control not a thermostat so may be in the dark a bit there so may need some help to test.

I'm excited about scenarios but I've read that amazon will only allow up to 12 "scenes" as they call them to be discovered which may be a pain.

Ive tried my best to make the api as simple and flexible as possible and not have it only for Alexa but indeed any web service its a bit rough at the moment so will probably no doubt need changes and tweaks that I'm sure all you guys on here can offer advice and assistance on.

At the moment because I'm using my own oauth2 server on the core the interface requires the following:

• A small folder of web pages to be placed in the  /var/www/lmce-admin folder on your core.
• The creation of 7 small mySQL tables to handle the access tokens ect.
• A SSL cert but it can be self signed we just want an encrypted connection for security - I think this is now done by default anyway.?
• Allow remote access to the web admin via https on the standard port and ensure you can then access your web admin from the web.
• An Alexa enabled product and the Alexa app

Note also that the intermediate site I've built will store your cores IP address and the required access tokens there is no way around this as I said before this site should i believe be stored on this linuxmce.org domain securely but for the moment I have it on a test site I can promise you Im not interested in accessing your core, but at the moment this is the only way I can see it being possible, you still have full control of the outh server on your core and revoke all access at any point.

So anyway I guess the fun question is once I've ironed out the few bits today who has a system they don't mind playing with that wants to test this?

[Marie.O]

Great news! Check with polly here on the forum to get some git credentials. We don't want to loose stuff.

[phenigma]

I'll pass on the existing LMCE OAuth2 information when I get home later today.  You will be the first to use it so it is very likely to not be completely set up properly.  I will leave it to you to let me know what needs to be changed/altered in gitlab issues.  Thanks for this!

The other thing that we're going to need to look at is setting up a proper LMCE interface with Alexa.  I understand you have created some 'skills' or whatever they are called on the Amazon side and we'll need to get an official LMCE setup with Amazon.  We can discuss this more as we go forward.

My intent is to implement Alexa on the RPi for use with LMCE, although my time is split between a whole lot of things.

Thanks again!  Don't forget to get some git credentials and we'll setup a branch to store this stuff in.

J.