Author Topic: [SOLVED] Device Template using SOAP  (Read 13802 times)

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
[SOLVED] Device Template using SOAP
« on: September 03, 2014, 08:17:40 pm »
Recently I had to retire my OPPO Blu Ray Player with its RS232 interface and went looking for something new.  Of all the Blu Ray choices out there, I settled on a Sony for its cost and features - one of which is its ability to accept network commands via SOAP.  I have been able to send commands to my Sony using command line and have fleshed out exactly what it can and cannot do and now I wish to create a device template to integrate this into my LMCE system and I have a few questions...

First of all, I have looked in the GSD source code and cannot quite figure out if it is possible to send headers to use SOAP in Ruby.  In my testing (using cURL) there is a header that needs sent in order for this to work - if it is not present, the player disregards the message and returns a 404.  So my question is - can I send a header to make this work and if so - how?

I have seen other posts about using SOAP in a device template but I do not see where it has been successful and I do not see any templates that currently use SOAP...

thanks in advance,
-herb
« Last Edit: September 08, 2014, 03:07:08 am by bherbie »

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Device Template using SOAP
« Reply #1 on: September 04, 2014, 04:26:52 am »
Hi bherbie!

I'm the author of the ISY994i template (device #2276) which has some SOAP calls.  I only play at being a programmer (more of a SysAdmin/integrator type) so take anything I write with a grain of salt...

My device driver uses GSD.  I have a function to "subscribe" to an events channel, to receive a stream of status events from the ISY controller.  As part of the function, it does a base64 encoding of the Username and password as part of the call (parameters 114 & 115). It looks like this...

Code: [Select]
def subscribeToISY()
# Prepare to send a subscription SOAP request to the device.
# Subscription is a one-way stream of events from the ISY to
# LMCE using the LMCE GSD conn_ method.
$port = device_.devdata_[69].to_s
auth_s=Base64.encode64(device_.devdata_[114]+":"+device_.devdata_[115])
soapBody ='<s:Envelope><s:Body><u:Subscribe xmlns:u="urn:udi-com:service:X_Insteon_Lighting_Service:1"><reportURL>REUSE_SOCKET</reportURL><duration>infinite</duration></u:Subscribe></s:Body></s:Envelope>'
s = "POST /services HTTP/1.1 \x0D\x0A"
s+= "Host: "+$host+":"+$port+" \x0D\x0A"
s+= "Content-Length: "+soapBody.length.to_s+" \x0D\x0A"
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
s+= "Authorization: Basic "+auth_s+" \x0D\x0A"
s+= "NT:upnp:event \x0D\x0A"
s+= "SOAPACTION:\"urn:udi-com:service:X_Insteon_Lighting_Service:1#Subscribe\" \x0D\x0A\x0D\x0A"
s+= soapBody.to_s
log("Attempting to Subscribe to ISY Events")
debuglog(s.to_s)
# Attempt to subscribe and retrieve data from the ISY
subscribe = conn_.Send(s.to_s)
log(subscribe)
end

I send commands back to the ISY asynchronously using it's REST interface, like so:
Code: [Select]
def sendISYcommand(command)
# This function sends commands to the ISY using it's REST interface.
# Does not use the subscription channel conn_ method.
# Commands are sent asynchronously, and success/fail is returned using
# this function.  Device feedback is received via the subscription.
#
# $username=device_.devdata_[114].to_s
# $password=device_.devdata_[115].to_s
$port = device_.devdata_[69].to_s
res = Net::HTTP.start($host) {|http|
req = Net::HTTP::Get.new(command)
req.basic_auth device_.devdata_[114].to_s, device_.devdata_[115].to_s
response = http.request(req)
xml_data = (response.body)
debuglog($yellow + "====== / sendISYcommand response body ======" + $grey)
debuglog(xml_data)
debuglog($yellow + "====== sendISYcommand response body / ======" + $grey)
doc = REXML::Document.new(xml_data)
s = ""
s = REXML::XPath.first(doc, "RestResponse/status")
if s != nil then
if s.text != "200" then
# Didn't get the HTTP OK :-(  return error code
return s.text
end
end
p = ""
p = REXML::XPath.first(doc, "properties/property")
if p != nil then
return p.text
end
return doc.to_s
}
end

Feel free to peruse the code in the template if this looks like something useful for you.  The interesting stuff is in "355 Process Initialize" and "373 Private Method Listing".

The Basic flow of the device start-up is:

355 Process Initialize (all the set-up stuff, calls functions in 373 Private Method Listing).
350 Process Incoming Data (basically, loops processing data coming in from the subscription channel).  Calls functions in 373 Private Method Listing.

Events from LMCE heading outbound to the ISY are serviced by 384 Process Receive for Child, which parses the LMCE commands, formulates the appropriate REST URL , and then calls my function sendISYcommand(command), where command is the URI encoded REST URL.  That code looks like this:

Code: [Select]
cmdId           = cmd.id_                                   # Command ID: ON, OFF, SET LEVEL
cmdTo           = cmd.devidto_                              # Device ID in LinuxMCE
devPort         = device_.childdevices_[cmdTo].devdata_[12] # 12 contains a port/channel
childType       = device_.childdevices_[cmdTo].devtemplid_  # Template ID to know type of device: switch or dimmer


case cmdId
      when 192 #192 is ON                     
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DON"
      when 193 #193 is OFF                       
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DOF"
      when 184 #184 is Level of dimmer
      dim_level = percenttodecihex(cmd.params_[76])
log("------dce--- Dim Level:" + dim_level.to_s)
           command = "/rest/nodes/#{URI.encode(devPort)}/cmd/DON/#{dim_level}"
else

end
log(command)
response = ""
response = sendISYcommand(command)
log(response)

You can find the commands you'll be interested in via the LMCE web GUI, under Advanced -> DCE -> Commands.  Look for the media commands.  Hopefully this provides some food for thought...

Good luck with your device driver!

/Mike

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: Device Template using SOAP
« Reply #2 on: September 04, 2014, 03:39:21 pm »
Thanks Mike!

This is exactly what I was looking for - I read your post about it when you were developing it but didn't see a device template for it.  I tried out your code and I am getting back a `bad request` error which in my experience has been a malformed SOAP request so I am still working with it to see if I can iron out the issues but I am by no means a ruby expert.

Again, thanks for the nudge in the right direction!
-herb

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Device Template using SOAP
« Reply #3 on: September 04, 2014, 11:47:15 pm »
Hi bherbie!

Have you seen this thread on Sony SOAP commands?  Might be helpful...

http://www.remotecentral.com/cgi-bin/mboard/rs232-ip/thread.cgi?171,3

Cheers!

/Mike

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: Device Template using SOAP
« Reply #4 on: September 05, 2014, 02:50:20 am »
Quote
Have you seen this thread on Sony SOAP commands?  Might be helpful...

http://www.remotecentral.com/cgi-bin/mboard/rs232-ip/thread.cgi?171,3

Yes, I have seen this and my code is based on a hybrid of that and yours.  It works flawlessly from the command line using php cURL but I must be doing something incorrect when translating it to ruby.  The logfile shows the command being sent and it looks correct but the response looks like this:

Code: [Select]
04-09-2014  18:43:44  HTTP/1.1 400 Bad Request
Connection: close
Date: Fri, 05 Sep 2014 00:43:44 GMT
Server: Linux/2.6
04-09-2014  18:43:44  UPnP/1.0 Sony-BDP/2.0

-herb

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Device Template using SOAP
« Reply #5 on: September 05, 2014, 03:48:09 am »
Hi bherbie!

Why don't you post the code segment and the logs, and maybe we all can figure it out?

Keep fighting the good fight!  You can always bend technology to your will  ;)

/Mike

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: Device Template using SOAP
« Reply #6 on: September 05, 2014, 04:10:01 am »
thanks for the help!  I know it is close and probably something I am overlooking as I am a noob in ruby.

here is the code

Code: [Select]
def log(word)
   logTime = Time.now
   timeStr = logTime.strftime("%d-%m-%Y  %H:%M:%S  ")
   $logFile.print(timeStr + word + "\n" )
   $logFile.flush()
   print(word + "\n")
end

def RemoteKey(key)

soapBody ='<?xml version="1.0"?>'
soapBody ='<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">'
soapBody+='<SOAP-ENV:Body>'
soapBody+='<m:X_SendIRCC xmlns:m="urn:schemas-sony-com:service:IRCC:1">'
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">' +key+ '</IRCCCode></m:X_SendIRCC>'
soapBody+='</SOAP-ENV:Body>'
soapBody+='</SOAP-ENV:Envelope>'

s = "POST /upnp/control/IRCC HTTP/1.0 \x0D\x0A"
        s+= "User-Agent: MediaRemote \x0D\x0A"
s+= "SOAPAction: \"urn:schemas-sony-com:service:IRCC:1#X_SendIRCC\" \x0D\x0A"
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
s+= "Content-Length: "+soapBody.length.to_s+" \x0D\x0A"
        s+= "Content-Transfer-Encoding: text \x0D\x0A"
        s+= "Connection-Close: close \x0D\x0A\x0D\x0A"
s+= soapBody.to_s

log("*****sending " +key+ " ******")
        log(s.to_s)
conn_.Send(s.to_s)
end

and then to send the key:
Code: [Select]
RemoteKey("AAAAAwAAHFoAAAAWAw==")

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Device Template using SOAP
« Reply #7 on: September 06, 2014, 04:25:36 am »
Hi beherbie!

I'm self-taught in ruby, so we could have a case of the blind leading the blind here...  :P


thanks for the help!  I know it is close and probably something I am overlooking as I am a noob in ruby.

here is the code

Code: [Select]
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">' +key+ '</IRCCCode></m:X_SendIRCC>'


and then to send the key:
Code: [Select]
RemoteKey("AAAAAwAAHFoAAAAWAw==")

I suspect an issue with type conversion.  I think you'll have to explicitly force it to string to merge it into the soapBody variable, like so...

Code: [Select]
soapBody+='<IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">'+key.to_s+'</IRCCCode></m:X_SendIRCC>'

I also generally don't place spaces between quotes and joined variables for string concatenation;  shouldn't make a difference, but some programming languages are more finicky than others (also prevents hidden line feed characters from sneaking in and causing hair loss  ;) ).

Try that, and see how it goes... Hope it helps!

Good Luck!

/Mike

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: Device Template using SOAP
« Reply #8 on: September 06, 2014, 06:20:27 pm »
I made the suggested changes and still get the same bad request error.  Here is the log:

Code: [Select]
06-09-2014  10:05:50  *****sending AAAAAwAAHFoAAAAWAw== ******
06-09-2014  10:05:50  POST /upnp/control/IRCC HTTP/1.1
Host: 192.168.80.151:50001
User-Agent: MediaRemote
Content-Type: text/xml; charset="utf-8"
Content-Length: 374
Content-Transfer-Encoding: text
SOAPAction: "urn:schemas-sony-com:service:IRCC:1#X_SendIRCC"
Connection-Close: close

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP-ENV:Body><m:X_SendIRCC xmlns:m="urn:schemas-sony-com:service:IRCC:1"><IRCCCode xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="string">AAAAAwAAHFoAAAAWAw==</IRCCCode></m:X_SendIRCC></SOAP-ENV:Body></SOAP-ENV:Envelope>
06-09-2014  10:05:50  HTTP/1.1 400 Bad Request
Connection: close
Date: Sat, 06 Sep 2014 16:05:49 GMT
Server: Linux/2.6
06-09-2014  10:05:50  UPnP/1.0 Sony-BDP/2.0

and sending the code from my php script works fine and it looks identical to the ruby code so I am not sure where to go.  I tried pulling up your template and when I click on the ruby codes it kicks me out to the admin home page - I completed an sqlCVS update and get the same thing so I am not sure if I am missing something in the initialize section or not.

any help you can provide is greatly appreciated!

-herb

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: Device Template using SOAP
« Reply #9 on: September 08, 2014, 03:06:23 am »
it always comes down to the details

Code: [Select]
s+= "Content-Type: text/xml; charset=\"utf-8\" \x0D\x0A"
changed to
Code: [Select]
s+= "Content-Type: text/xml; charset=utf-8 \x0D\x0A"
fixes my problem and it now works as expected.  Thanks for the help!

-herb

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5549
  • DOES work for LinuxMCE.
    • View Profile
Re: [SOLVED] Device Template using SOAP
« Reply #10 on: September 08, 2014, 04:07:06 am »
Are we putting this in a wiki page? :)

-Thom

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: [SOLVED] Device Template using SOAP
« Reply #11 on: September 08, 2014, 04:36:46 am »
Quote
Are we putting this in a wiki page? :)

Absolutely.  Just need to work out a few more kinks...

-herb

RayBe

  • Guru
  • ****
  • Posts: 315
    • View Profile
Re: [SOLVED] Device Template using SOAP
« Reply #12 on: September 23, 2014, 10:16:06 pm »
Absolutely.  Just need to work out a few more kinks...

-herb

Any progress on the subject?
I am very interested in how to create SOAP requests using GSD.
I want to try and make a DT for a panasonic plasma with it.
thanks in advance,

br,
Raymond
When you were born, you were crying and everybody else was laughing.
Live your life so when you die, you are laughing and everybody else is crying.

bherbie

  • Veteran
  • ***
  • Posts: 54
    • View Profile
Re: [SOLVED] Device Template using SOAP
« Reply #13 on: September 23, 2014, 11:56:15 pm »
Quote
Any progress on the subject?

I have been able to build a template that works using SOAP although I am a ways from being finished.  The code provided by mkbrown69 was enough to get me pointed in the right direction as the Sony interface expects information in its header that is different then what mkbrown69 needed for his.  Also, I am not sure if it is just my setup, but the buttons work intermittently with the this template.  I see a lot of 'Could Not Handle This Message' errors in the logs and after a quick reload it works fine for a click or two and then it does it again.  Once again - not sure if it is my lack of knowledge in Ruby or if there is another issue at play.

Also, my device uses WOL as its 'power on' and I have hard coded the device number into the template (i.e. system('sudo /usr/pluto/bin/WakeMD.sh --dev 125') and once I get a more elegant solution, I will add it (or could this be part of the issue noted above?)

So long story short, see if you can make any progress using the code snippets outlined in this thread or post back if you need more of a start as I am in the middle of getting a wiki page put together once I know more myself.

-herb

totallymaxed

  • LinuxMCE God
  • ****
  • Posts: 4660
  • Smart Home Consulting
    • View Profile
    • Dianemo - at home with technology
Re: [SOLVED] Device Template using SOAP - Registering your Control device
« Reply #14 on: October 22, 2014, 04:06:52 pm »
Hi,

All Sony IP controllable TV's/BluRay Players require that the controlling device register with the TV/BluRay Player before reliable control can commence. You need to place the Sony device in 'Registration Mode' - a function that will be found in one of the Settings screens. You then need to register your control device by sending a 'register' command to the Sony device you want to control. In the Dianemo Sony TV & BluRay player device templates we have a 'Register' command implemented for this purpose. On receipt of the 'Register' command the BluRay player or TV should then confirm the registration on screen...and your all set.

Reading the thread above I can't see any mention of the registration process - without it you may get intermittent or unreliable control capability.

All the best

Andy

Andy Herron,
CHT Ltd

For Dianemo/LinuxMCE consulting advice;
@herron on Twitter, totallymaxed+inquiries@gmail.com via email or PM me here.

Get Dianemo-Rpi2 ARM Licenses http://forum.linuxmce.org/index.php?topic=14026.0

Get RaspSqueeze-CEC or Raspbmc-CEC for Dianemo/LinuxMCE: http://wp.me/P4KgIc-5P

Facebook: https://www.facebook.com/pages/Dianemo-Home-Automation/226019387454465

http://www.dianemo.co.uk