LinuxMCE Forums

General => Developers => Topic started by: bherbie on September 03, 2014, 08:17:40 pm

Title: [SOLVED] Device Template using SOAP
Post by: bherbie on September 03, 2014, 08:17:40 pm
Recently I had to retire my OPPO Blu Ray Player with its RS232 interface and went looking for something new.  Of all the Blu Ray choices out there, I settled on a Sony for its cost and features - one of which is its ability to accept network commands via SOAP.  I have been able to send commands to my Sony using command line and have fleshed out exactly what it can and cannot do and now I wish to create a device template to integrate this into my LMCE system and I have a few questions...

First of all, I have looked in the GSD source code and cannot quite figure out if it is possible to send headers to use SOAP in Ruby.  In my testing (using cURL) there is a header that needs sent in order for this to work - if it is not present, the player disregards the message and returns a 404.  So my question is - can I send a header to make this work and if so - how?

I have seen other posts about using SOAP in a device template but I do not see where it has been successful and I do not see any templates that currently use SOAP...

thanks in advance,
-herb
Title: Re: Device Template using SOAP
Post by: mkbrown69 on September 04, 2014, 04:26:52 am
Hi bherbie!

I'm the author of the ISY994i template (device #2276) which has some SOAP calls.  I only play at being a programmer (more of a SysAdmin/integrator type) so take anything I write with a grain of salt...

My device driver uses GSD.  I have a function to "subscribe" to an events channel, to receive a stream of status events from the ISY controller.  As part of the function, it does a base64 encoding of the Username and password as part of the call (parameters 114 & 115). It looks like this...

Code: [Select]
def subscribeToISY()
# Prepare to send a subscription SOAP request to the device.
# Subscription is a one-way stream of events from the ISY to
# LMCE using the LMCE GSD conn_ method.
$port = device_.devdata_[69].to_s
auth_s=Base64.encode64(device_.devdata_[114]+":"+device_.devdata_[115])
soapBody ='<s:Envelope><s:Body><u:Subscribe xmlns:u="urn:udi-com:service:X_Insteon_Lighting_Service:1"><reportURL>REUSE_SOCKET</reportURL><duration>infinite</duration></u:Subscribe></s:Body></s:Envelope>'
s = "POST /services HTTP/1.1 \x0D\x0A"
s+= "Host: "+$host+":"+$port+" \x0D\x0A"
s+= "Content-Length: "+soapBody.length.to_s+" \x0D\x0A"
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
s+= "Authorization: Basic "+auth_s+" \x0D\x0A"
s+= "NT:upnp:event \x0D\x0A"
s+= "SOAPACTION:\"urn:udi-com:service:X_Insteon_Lighting_Service:1#Subscribe\" \x0D\x0A\x0D\x0A"
s+= soapBody.to_s
log("Attempting to Subscribe to ISY Events")
debuglog(s.to_s)
# Attempt to subscribe and retrieve data from the ISY
subscribe = conn_.Send(s.to_s)
log(subscribe)
end

I send commands back to the ISY asynchronously using it's REST interface, like so:
Code: [Select]
def sendISYcommand(command)
# This function sends commands to the ISY using it's REST interface.
# Does not use the subscription channel conn_ method.
# Commands are sent asynchronously, and success/fail is returned using
# this function.  Device feedback is received via the subscription.
#
# $username=device_.devdata_[114].to_s
# $password=device_.devdata_[115].to_s
$port = device_.devdata_[69].to_s
res = Net::HTTP.start($host) {|http|
req = Net::HTTP::Get.new(command)
req.basic_auth device_.devdata_[114].to_s, device_.devdata_[115].to_s
response = http.request(req)
xml_data = (response.body)
debuglog($yellow + "====== / sendISYcommand response body ======" + $grey)
debuglog(xml_data)
debuglog($yellow + "====== sendISYcommand response body / ======" + $grey)
doc = REXML::Document.new(xml_data)
s = ""
s = REXML::XPath.first(doc, "RestResponse/status")
if s != nil then
if s.text != "200" then
# Didn't get the HTTP OK :-(  return error code
return s.text
end
end
p = ""
p = REXML::XPath.first(doc, "properties/property")
if p != nil then
return p.text
end
return doc.to_s
}
end

Feel free to peruse the code in the template if this looks like something useful for you.  The interesting stuff is in "355 Process Initialize" and "373 Private Method Listing".

The Basic flow of the device start-up is:

355 Process Initialize (all the set-up stuff, calls functions in 373 Private Method Listing).
350 Process Incoming Data (basically, loops processing data coming in from the subscription channel).  Calls functions in 373 Private Method Listing.

Events from LMCE heading outbound to the ISY are serviced by 384 Process Receive for Child, which parses the LMCE commands, formulates the appropriate REST URL , and then calls my function sendISYcommand(command), where command is the URI encoded REST URL.  That code looks like this:

Code: [Select]
cmdId           = cmd.id_                                   # Command ID: ON, OFF, SET LEVEL
cmdTo           = cmd.devidto_                              # Device ID in LinuxMCE
devPort         = device_.childdevices_[cmdTo].devdata_[12] # 12 contains a port/channel
childType       = device_.childdevices_[cmdTo].devtemplid_  # Template ID to know type of device: switch or dimmer


case cmdId
      when 192 #192 is ON                     
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DON"
      when 193 #193 is OFF                       
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DOF"
      when 184 #184 is Level of dimmer
      dim_level = percenttodecihex(cmd.params_[76])
log("------dce--- Dim Level:" + dim_level.to_s)
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DON/#{dim_level}"
else

end
log(command)
response = ""
response = sendISYcommand(command)
log(response)

You can find the commands you'll be interested in via the LMCE web GUI, under Advanced -> DCE -> Commands.  Look for the media commands.  Hopefully this provides some food for thought...

Good luck with your device driver!

/Mike
Title: Re: Device Template using SOAP
Post by: bherbie on September 04, 2014, 03:39:21 pm
Thanks Mike!

This is exactly what I was looking for - I read your post about it when you were developing it but didn't see a device template for it.  I tried out your code and I am getting back a `bad request` error which in my experience has been a malformed SOAP request so I am still working with it to see if I can iron out the issues but I am by no means a ruby expert.

Again, thanks for the nudge in the right direction!
-herb
Title: Re: Device Template using SOAP
Post by: mkbrown69 on September 04, 2014, 11:47:15 pm
Hi bherbie!

Have you seen this thread on Sony SOAP commands?  Might be helpful...

http://www.remotecentral.com/cgi-bin/mboard/rs232-ip/thread.cgi?171,3

Cheers!

/Mike
Title: Re: Device Template using SOAP
Post by: bherbie on September 05, 2014, 02:50:20 am
Quote
Have you seen this thread on Sony SOAP commands?  Might be helpful...

http://www.remotecentral.com/cgi-bin/mboard/rs232-ip/thread.cgi?171,3

Yes, I have seen this and my code is based on a hybrid of that and yours.  It works flawlessly from the command line using php cURL but I must be doing something incorrect when translating it to ruby.  The logfile shows the command being sent and it looks correct but the response looks like this:

Code: [Select]
04-09-2014  18:43:44  HTTP/1.1 400 Bad Request
Connection: close
Date: Fri, 05 Sep 2014 00:43:44 GMT
Server: Linux/2.6
04-09-2014  18:43:44  UPnP/1.0 Sony-BDP/2.0

-herb
Title: Re: Device Template using SOAP
Post by: mkbrown69 on September 05, 2014, 03:48:09 am
Hi bherbie!

Why don't you post the code segment and the logs, and maybe we all can figure it out?

Keep fighting the good fight!  You can always bend technology to your will  ;)

/Mike
Title: Re: Device Template using SOAP
Post by: bherbie on September 05, 2014, 04:10:01 am
thanks for the help!  I know it is close and probably something I am overlooking as I am a noob in ruby.

here is the code

Code: [Select]
def log(word)
   logTime = Time.now
   timeStr = logTime.strftime("%d-%m-%Y  %H:%M:%S  ")
   $logFile.print(timeStr + word + "\n" )
   $logFile.flush()
   print(word + "\n")
end

def RemoteKey(key)

soapBody ='<?xml version="1.0"?>'
soapBody ='<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">'
soapBody+='<SOAP-ENV:Body>'
soapBody+='<m:X_SendIRCC xmlns:m="urn:schemas-sony-com:service:IRCC:1">'
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">' +key+ '</IRCCCode></m:X_SendIRCC>'
soapBody+='</SOAP-ENV:Body>'
soapBody+='</SOAP-ENV:Envelope>'

s = "POST /upnp/control/IRCC HTTP/1.0 \x0D\x0A"
        s+= "User-Agent: MediaRemote \x0D\x0A"
s+= "SOAPAction: \"urn:schemas-sony-com:service:IRCC:1#X_SendIRCC\" \x0D\x0A"
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
s+= "Content-Length: "+soapBody.length.to_s+" \x0D\x0A"
        s+= "Content-Transfer-Encoding: text \x0D\x0A"
        s+= "Connection-Close: close \x0D\x0A\x0D\x0A"
s+= soapBody.to_s

log("*****sending " +key+ " ******")
        log(s.to_s)
conn_.Send(s.to_s)
end

and then to send the key:
Code: [Select]
RemoteKey("AAAAAwAAHFoAAAAWAw==")
Title: Re: Device Template using SOAP
Post by: mkbrown69 on September 06, 2014, 04:25:36 am
Hi beherbie!

I'm self-taught in ruby, so we could have a case of the blind leading the blind here...  :P


thanks for the help!  I know it is close and probably something I am overlooking as I am a noob in ruby.

here is the code

Code: [Select]
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">' +key+ '</IRCCCode></m:X_SendIRCC>'


and then to send the key:
Code: [Select]
RemoteKey("AAAAAwAAHFoAAAAWAw==")

I suspect an issue with type conversion.  I think you'll have to explicitly force it to string to merge it into the soapBody variable, like so...

Code: [Select]
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">'+key.to_s+'</IRCCCode></m:X_SendIRCC>'

I also generally don't place spaces between quotes and joined variables for string concatenation;  shouldn't make a difference, but some programming languages are more finicky than others (also prevents hidden line feed characters from sneaking in and causing hair loss  ;) ).

Try that, and see how it goes... Hope it helps!

Good Luck!

/Mike
Title: Re: Device Template using SOAP
Post by: bherbie on September 06, 2014, 06:20:27 pm
I made the suggested changes and still get the same bad request error.  Here is the log:

Code: [Select]
06-09-2014  10:05:50  *****sending AAAAAwAAHFoAAAAWAw== ******
06-09-2014  10:05:50  POST /upnp/control/IRCC HTTP/1.1
Host: 192.168.80.151:50001
User-Agent: MediaRemote
Content-Type: text/xml; charset="utf-8"
Content-Length: 374
Content-Transfer-Encoding: text
SOAPAction: "urn:schemas-sony-com:service:IRCC:1#X_SendIRCC"
Connection-Close: close

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP-ENV:Body><m:X_SendIRCC xmlns:m="urn:schemas-sony-com:service:IRCC:1"><IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">AAAAAwAAHFoAAAAWAw==</IRCCCode></m:X_SendIRCC></SOAP-ENV:Body></SOAP-ENV:Envelope>
06-09-2014  10:05:50  HTTP/1.1 400 Bad Request
Connection: close
Date: Sat, 06 Sep 2014 16:05:49 GMT
Server: Linux/2.6
06-09-2014  10:05:50  UPnP/1.0 Sony-BDP/2.0

and sending the code from my php script works fine and it looks identical to the ruby code so I am not sure where to go.  I tried pulling up your template and when I click on the ruby codes it kicks me out to the admin home page - I completed an sqlCVS update and get the same thing so I am not sure if I am missing something in the initialize section or not.

any help you can provide is greatly appreciated!

-herb
Title: Re: Device Template using SOAP
Post by: bherbie on September 08, 2014, 03:06:23 am
it always comes down to the details

Code: [Select]
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
changed to
Code: [Select]
s+= "Content-Type: text/xml; charset=utf-8 \x0D\x0A"
fixes my problem and it now works as expected.  Thanks for the help!

-herb
Title: Re: [SOLVED] Device Template using SOAP
Post by: tschak909 on September 08, 2014, 04:07:06 am
Are we putting this in a wiki page? :)

-Thom
Title: Re: [SOLVED] Device Template using SOAP
Post by: bherbie on September 08, 2014, 04:36:46 am
Quote
Are we putting this in a wiki page? :)

Absolutely.  Just need to work out a few more kinks...

-herb
Title: Re: [SOLVED] Device Template using SOAP
Post by: RayBe on September 23, 2014, 10:16:06 pm
Absolutely.  Just need to work out a few more kinks...

-herb

Any progress on the subject?
I am very interested in how to create SOAP requests using GSD.
I want to try and make a DT for a panasonic plasma with it.
thanks in advance,

br,
Raymond
Title: Re: [SOLVED] Device Template using SOAP
Post by: bherbie on September 23, 2014, 11:56:15 pm
Quote
Any progress on the subject?

I have been able to build a template that works using SOAP although I am a ways from being finished.  The code provided by mkbrown69 was enough to get me pointed in the right direction as the Sony interface expects information in its header that is different then what mkbrown69 needed for his.  Also, I am not sure if it is just my setup, but the buttons work intermittently with the this template.  I see a lot of 'Could Not Handle This Message' errors in the logs and after a quick reload it works fine for a click or two and then it does it again.  Once again - not sure if it is my lack of knowledge in Ruby or if there is another issue at play.

Also, my device uses WOL as its 'power on' and I have hard coded the device number into the template (i.e. system('sudo /usr/pluto/bin/WakeMD.sh --dev 125') and once I get a more elegant solution, I will add it (or could this be part of the issue noted above?)

So long story short, see if you can make any progress using the code snippets outlined in this thread or post back if you need more of a start as I am in the middle of getting a wiki page put together once I know more myself.

-herb
Title: Re: [SOLVED] Device Template using SOAP - Registering your Control device
Post by: totallymaxed on October 22, 2014, 04:06:52 pm
Hi,

All Sony IP controllable TV's/BluRay Players require that the controlling device register with the TV/BluRay Player before reliable control can commence. You need to place the Sony device in 'Registration Mode' - a function that will be found in one of the Settings screens. You then need to register your control device by sending a 'register' command to the Sony device you want to control. In the Dianemo Sony TV & BluRay player device templates we have a 'Register' command implemented for this purpose. On receipt of the 'Register' command the BluRay player or TV should then confirm the registration on screen...and your all set.

Reading the thread above I can't see any mention of the registration process - without it you may get intermittent or unreliable control capability.

All the best

Andy

Title: Re: [SOLVED] Device Template using SOAP
Post by: bherbie on October 23, 2014, 03:03:39 am
Quote
Reading the thread above I can't see any mention of the registration process - without it you may get intermittent or unreliable control capability.
Correct.  I did this simply by using php cURL from the command line. The issues I was running into is my particular model does not use the same ports as the Sonys that I could find the protocols for. I did register my core as a device before I began setting up the template for the device itself and I still have some kinks to work out such as hard coding in the magic packet for WOL.

-herb
Title: Re: [SOLVED] Device Template using SOAP
Post by: totallymaxed on November 06, 2014, 10:27:22 am
Correct.  I did this simply by using php cURL from the command line. The issues I was running into is my particular model does not use the same ports as the Sonys that I could find the protocols for. I did register my core as a device before I began setting up the template for the device itself and I still have some kinks to work out such as hard coding in the magic packet for WOL.

-herb

Sorry for my slow response on this! Sony TV's (at least all the ones we've ever tested or built device templates for) do not work with WOL via the network port. If the TV is placed into 'Stand-by Off' the NIC is powered completely off in this mode and so cannot be communicated with. The only way to 'wake' a Sony TV (and any other brand with IP control) from 'Stand-by Off' is to either use the IR remote or to send an 'Stand-by On' via HDMI-CEC control. This is one of the functions we use Raspberry Pi's for in our systems (we used to use PulseEight CEC Adapters but they need to be controlled from an MD whereas the Rpi can do HDMI-CEC control directly)

All the best

Andy