Purps,

Speaking with my sysadmin hat on, you'll need to create the user joggler first on the core, using the password that you used to create the joggler user on the Joggler device.  A group jogger will be created automatically, based on the standard behavior for useradd. Then chown will work properly.  You may need to play with making the UID's and GID's match on the joggler and the core, if there are going to be shared files over NFS; otherwise you may run into permission issues.  This is based on a quick read of the wiki page; I don't own a joggler.  

HTH!

/Mike

Good day folks!

I just thought I'd point out a home automation controller with a well developed REST and SOAP/WSDL implementation that might provide some food for thought.  It also has an AJAX library specific to it that leverages it's REST/SOAP interfaces, that's used as an embedded controller Web UI (it's main Admin/configuration app is Java based, but will likely be ported to something more HTML5 based in the future)

http://www.universal-devices.com/isy-developers/   look under the Official SDK section.

I'm pointing it out because it exposes lighting control, climate control and weather, energy measurement, etc via those REST/SOAP interfaces using XML.  It exposes enough information that various GUI's can determine it's configuration and that of all the child devices, and present a GUI based on that information.  It might give you folks some ideas...

HTH!

/Mike

P.S.  that's the device I'm struggling to write a GSD driver for, so if Hari feels like whipping one up for Ago Control, I'd be happy to test.  I can't 'C' my way out of a wet paper bag ;-). I've been playing with Ago a bit on my Pi.  Want to have some free time for hardware hacking so I can play with your Blink-M stuff (once I build a few clones). /M

Hi Thom,

I'm not that IRC saavy/equipped; I should probably add that to my list of things to learn.  I'm also not sure how well it'd work via my iPad since I bounce between Safari, Textastic, and iSSH.  Long story...

Anyway, based on my understanding of the devices' SDK, it needs a channel for the subscription for the SOAP events with a periodic heartbeat response, and commands are sent via the REST interface (so, two connections).  Based on my understanding of the conn_object, it's a single channel implementation, but bi-directional.  This all is with the caveat/disclaimer that I don't play a programmer on TV or the Internet;  I'm a sysadmin who's managed to learn enough about programming to be able to glue various bits and pieces together.

So, I have this little private method which works beautifully, and I can control manually-defined children lights of the device quite nicely.

Code:

def sendISYcommand(command)
#$username=device_.devdata_[114].to_s
#$password=device_.devdata_[115].to_s
#$host=device_.devdata_[?].to_s

$host = "192.168.80.148"
$port = device_.devdata_[69].to_s
res = Net::HTTP.start($host) {|http|
req = Net::HTTP::Get.new(command)
req.basic_auth device_.devdata_[114].to_s, device_.devdata_[115].to_s
response = http.request(req)
xml_data = (response.body)
log("====== / sendISYcommand response body ======")
log(xml_data)
log("====== sendISYcommand response body / ======")
doc = REXML::Document.new(xml_data)
log(doc)
s = REXML::XPath.first(doc, "RestResponse/status")
log(s)
return s.text
}
end

I'll worry about http/https via the port data later when the driver is close to completion.  Pulling device data like username & password works well.  I'm just looking to find a means to obtain the IP cleanly for sending these REST commands.  My next project is the conn_object for the subscription, and figure out all the other xml parsing that'll go along with it.

Thanks again for your time and guidance.

/Mike

Hi Thom!

Why?  To be different, of course! 

Seriously, though, the device in question has a SOAP interface used to subscribe to events emitted by the device, and a REST interface to control it.  I have a basic set of commands working to control it via REST, using a hard-coded variable for the IP.  Before I go writing code to query the device table directly, I'm asking for a less ugly, more Thom-approved method.   

I plan on getting to the subscription stuff next using the conn_object.  It's slow-going teaching myself SOAP, XML parsing, and ruby all at once, plus figuring out the relevant LMCE infrastructure.

Thanks for your time!

/Mike

Good day folks!

I'm working on a device template using GSD for an IP enabled device, and I was wondering if it's possible to obtain the IP address of the device from device data?  I'm building various URL's for device control, and it would be cleaner to be able to retrieve the IP address from the device's data...

Thanks for your time!

/Mike

It works with MythTV, but not the version in LMCE 10.04 (which is MythTV 0.23).  MythTV 0.25 is minimum requirement, which is still doable on LMCE 10.04.  I believe that MythTV 0.26 requires versions of libraries that are in Ubuntu 12.04, and not the Ubuntu 10.04 that LMCE is based on.

http://www.mythtv.org/wiki/Ceton

HTH!

/MIke

Thom,

Actually, if you're able to make the C++ 1-wire device template more flexible, to accommodate more of the OWFS supported devices, that may solve the problem.  Right now it's hard-coded to expect a serial type device, which rules out all the network, i2c, and other types of OWFS bus masters <http://owfs.org/index.php?page=bus-masters>

In my case, I'd like to be able to use my Raspberry Pi as a networked OWFS control node.  I have some quick and dirty bash scripts running that do owreads into a variable, and then do a messagesend to a manually created device on the core.  Works well enough for proof-of-concept, and I can see the little Thermometer icon on the floor plans.

Just a thought...

Thanks!

/Mike

Thom,

Thanks for the detailed explanations and good advice.  I'll keep it all in mind as I continue to learn about LMCE, and work towards implementing some of the things I'm working on.

Thanks for your time, and all the best for the New Year!

/Mike

Happy New Year folks!

As I'm poking around under the hood trying to understand how some things bolt together so I can look at implementing some things I've been thinking about, I've got a few questions about how things work in the device table.

1. What is the purpose of the 'state' field?

2. Is the intention of the 'status' field to show a near-real-time status of the device in question?  Typically, what is responsible for updating that field?  Based on what's shown in the wiki, it's almost as if each device class or plugin is responsible for updating status. http://wiki.linuxmce.org/index.php/Device_Status

3. As there is a MAC address and an IP address as part of the device's data, would it also not be prudent to have a hostname field as part of the device data?  It would make some aspects of managing the LMCE internal network a lot easier if that data were present in the device table, rather than derived and hard coded in external scripts or elsewhere in the database.  I'm looking at playing around with the 'workstation' aspect, and it would sure make my life easier if the hostname were there. (Note: there is a "#188 computer name" data point, but that seems like a WINS name field to me, but I could be mistaken).

If the core devs don't see an issue with that, I'll put a feature request ticket in for that schema change.  The core devs would be the ones with a full understanding of the possible effects that change would have, so I would think they'd want to handle any change to a core db table like device.

Also, one final question:  I manually changed a MD from a MD template to a workstation template in the Device table, and that seems to work.  Is that data recorded anywhere else besides the device table?

Thanks for your time, and happy home automation for 2013!

/Mike

Newguy2,

On your core, run

sudo dpkg -s mythtv-frontend | grep Version

If Version is greater than 0.23, then you've installed a newer mythtv version than is presently supported on LinuxMCE.  Some surgery may be required to restore it to a working version...

Hope that helps!

/Mike

Ted,

The IBM card is coming up in raid mode (meaning it wasn't flashed into IT mode, which is a passthrough mode).  So, you'll want to set the I/O scheduler to 'noop' rather than deadline, and let the raid card handle the re-ordering of I/O operations.  

/etc/rc.local

Code:

echo noop > /sys/block/sda/queue/scheduler

The default CFQ scheduler just adds unnecessary latency and I/O operations when feeding HBA's and RAID controller cards.  It's more useful on desktops/laptops with SATA disks.

Presuming those are consumer SATA drives, not SAS or enterprise SATA, you'll be limited by the spindle speeds.  Hopefully those aren't green drives; 7200 RPM and higher would be necessary as the root file system.

That raid card supports four ports, so depending on how it was cabled at the factory, you might have the other two ports available to you, in which case you could tuck an SSD inside the server case and Velcro it in.  It might be using SFF-8087 or 8088 cables, in which case you'll be out of luck for adding internal disks.  Take a look at http://public.presalesadvisor.com/LiteratureUploads/Literature-482.pdf which is for the IBM server you identified, and see how it compares to what you have.

Use the GigE switch for production usage.  It'll help, but your core issue is disk related.  You may also need to tune MySQL, as 5.1, InnoDB tables, and ext4 don't necessarily play well together, resulting in I/O waits which you'll definitely notice as everything is hitting the same set of spindles.

HTH!

/Mike

(with a small SSD disk inside things could be very fast)
To make my life easier, when I first started designing the project specs, I was planning to install Linuxmce 810 final, but (on installation time) it would not see the IBM controller (serveraid M1015)..

As Thom said, you're I/O bound.  Your issues are the result of 10 NFS clients using the NFS shares as their root file system, so you have all sorts of random I/O's happening, including lots of writes due to logging.  I'm also presuming that the MySQL database which powers the application logic on the core is residing on the same spindle, so you're probably maxing out the IOPS of a single drive, and hitting wait states and timeouts as a result.  'top' and 'iotop' run on both the core and MD's will tell you for sure.

Things you can do to improve the situation:

• Install your core's filesystem to an SSD.  You'll be going from 100 IOPS to 4K+ IOPS in one easy step.  If you can't re-install or move it, then add an SSD to the system using LVM, and create logical volumes which get mounted at /var/lib/mysql and /usr/pluto/diskless to host the IO intensive workloads.  Obviously, you'll need to move the contents of those directories while in single-user mode, then reboot.
• If you can't do SSD, then use the RAID card in RAID 10 mode for the core's file system.  SSD would be better though for random I/O.  Either way, set the I/O scheduler for that /dev/sdX to 'deadline'.
• If you have lots of memory on your MD's, consider making /tmp a tmpfs ramdisk
• Consider setting vm.swappiness=1 i sysctl.conf on your MD's, to reduce swapping, or install a local disk for swap if you don't have enough memory on the MD's.
• Tune the TCP stack for throughput on the core, and increase your memory buffers.  http://www.cyberciti.biz/faq/linux-tcp-tuning/
• Use a really good network card for the internal network.  Intel GigE cards are typically the best on Linux; Realtek and NVIDIA on-boards are crap under heavy loads.  If you have a capable managed switch and a multi-port NIC (or multiple NIC's), you can look at port aggregation for increased bandwidth.  Or go 10G between the core and the switch if you have money to spare.  No matter what, you need a switch with a non-blocking fabric, capable of full wire-speeds for the number of ports available.

Hope that helps!

/Mike

sad face. all i can say. bums me out to hear that we are basically suffering because of software we dont maintain. awesome :|

The problem with MythTV in LMCE is that it's 0.23, and that version had known stability issues.  When I had upgraded my (non-LMCE) MythTV, it was considerably more stable on 0.24 and 0.25 than it ever was on 0.23.  If we could upgrade the LMCE MythTV parts to support 0.25-fixes, that would likely address some people's stability issues.  0.26 introduces UTC time in the database, so the Level of effort there might be a bit higher at this point in time.  Besides, it's never a good idea to upgrade to a point zero release on any software... Let it get to 0.26.2 before considering it for inclusion when stability is key.

My $0.02 worth before HST.

HTH!

/Mike

Good day folks!

I'm having some issues with the stock firewall settings (I added the "domain" TCP/UDP 53 ones via the LMCE firewall page to play with it).  Lots of packet denied messages, including localhost.

Code:

[1062546.521689] IPv4 packet denied: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=127.0.0.1 DST=127.0.0.1 LEN=57 TOS=0x00 PREC=0x00 TTL=64 ID=39621 DF PROTO=TCP SPT=47142 DPT=3450 WINDOW=770 RES=0x00 ACK PSH URGP=0

[1062570.285733] IPv4 packet denied: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:52:54:00:a8:7f:eb:08:00 SRC=192.168.80.245 DST=255.255.255.255 LEN=68 TOS=0x00 PREC=0x00 TTL=128 ID=14050 PROTO=UDP SPT=52330 DPT=52330 LEN=48

[1062522.277283] IPv4 packet denied: IN=eth1 OUT= MAC=54:52:00:6a:5b:4b:00:0c:29:b5:fd:73:08:00 SRC=192.168.80.4 DST=192.168.80.1 LEN=136 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=703 DPT=2049 LEN=116

Interfaces are correct, eth0 is outside, eth1 is inside (LMCE network).

Code:

# iptables -L
Chain INPUT (policy DROP)
target     prot opt source               destination
LOG        all  --  anywhere             anywhere            limit: avg 5/min burst
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISH
ACCEPT     all  --  anywhere             anywhere            mark match 0x1
ACCEPT     udp  --  anywhere             anywhere            udp dpt:bootps
ACCEPT     all  --  192.168.80.0/24      anywhere
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:www
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Any ideas why localhost and the internal network are denying so many packets?

Code:

dcerouter_1021496:/var/log# cat syslog | grep "IPv4 packet denied" |wc -l
6283
dcerouter_1021496:/var/log# cat syslog | grep "IPv4 packet denied" |grep 127.0.0.1 |wc -l
2776
dcerouter_1021496:/var/log# cat syslog | grep "IPv4 packet denied" |grep 192.168.80 |wc -l
3243

Thanks for your time!

/Mike

P.S.  disabling the firewall via the web admin doesn't actually disable or clear the firewall.   I had to clear with iptables -F and iptables -Z.

Good day folks!

I've got a question about the sambahelper user.  Obviously, it's used on the core and on the MD's.  It's being assigned UID's in the "normal" unix user range, which can conflict with existing core and NIS users.  As you see below, it's inconsistent between what was created on moon72 and what was created on the core.

Code:

root@moon72:~# cat /etc/passwd | grep samba
sambahelper:x:1000:1000:Pluto Samba Share Helper:/tmp:/bin/false

root@moon72:~# ypcat passwd
mkbrown:x:1000:1000:Michael Brown,,,:/home/mkbrown:/bin/bash
sambahelper:x:1001:1001:Pluto Samba Share Helper:/tmp:/bin/false

The sambahelper user is created in PlutoStorageDevices/mkr_postinst.sh. It is created using the following code starting on lines 31

Code:

## Samba Share Helper
if [[ "$(id -u sambahelper)" == "" ]] ;then
        useradd -c "Pluto Samba Share Helper" -d /tmp -s /bin/false sambahelper
fi

Wouldn't it be better to have it created as a "system" user using -r, or to statically assign something like -u 999 -g 999 ?

I'm throwing this out there for someone much more knowledgeable about this subsystem to educate me, or suggest I file a Trac ticket.

Thanks for your time!

/Mike