Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - mkbrown69

Pages: 1 ... 11 12 [13] 14 15
Feature requests & roadmap / Re: Monitoring
« on: January 23, 2012, 02:07:40 am »

What I think Thom is getting at is that if you're going to champion Cacti integration, you'll have to extend Cacti a bit to send DCE events, so that DataLogger can record them, and the rest of LMCE can act on them.  So, instead of Cacti sending an e-mail or SMS when a threshold is exceeded, it would use MessageSend to emit an LMCE event which Datalogger would record, and that the system could act on if a scenario was defined.  Like so...

Power usage goes over 5kw, Cacti generates a MessageSend to LMCE, Datalogger records the threshold exceeded event, and an "excessive power use" scenario is triggered, which broadcasts a message to all orbitors, and a "chi-Ching" sound is played over all audio devices.

So, it means some work extending Cacti, some work extending Datalogger, and some work figuring out events.  Plus, doing all that in such a way that someone else can extend your work to graph other things (soil moisture readings, humidity, Squid proxy results for domains, etc).  It would also probably involve integrating Cacti's web GUI into the LMCE web GUI, and use LMCE defined users.

I agree with you that Cacti and rrd's are great for recording some types of data, like multiple temperature sensors.  Assuming OWFS as a sensing infrastructure, it samples by default every 10 seconds per sensor.  So, that works out to 8640 samples per sensor per day.  Over time, those samples can be averaged because the further away they are from the present, the less need there is for precision (which is what RRD's do).   Cacti is also great at dealing with all the noise that syslog can generate.  Same with other types of repetitively sampled data, where the need for precision becomes less important over time.

Where I think Datalogger integration could really shine is for correlating events from multiple subsystems into a "timeline of events".  It's the concept of federated data; one overseer of other more specialized reporting/monitoring systems. It could be useful for debugging complex scenerios (this motion event triggered that lighting event when this other condition existed) as well as for security auditing (think alarm system event log), telephony logging, MythTV events, etc.  It would be especially nice if the Datalogger events could be viewed from an orbitor.  What would be really cool is heuristics to mine the Datalogger for patterns.  Think a "vacation mode" which operates lights based on the inhabitants past behaviours.

Just my $0.02 CDN as someone who's presently dealing with Federating configuration and compliance management systems at work (in addition to other things).


Users / Re: Core Only with TV Tuner?
« on: January 14, 2012, 05:16:37 am »
Good day!

Someone more knowledgeable will correct me if I'm wrong, but disabling the MD/Orbitor on the core will only affect your ability to VIEW media on the core.  The tuners are managed by MythTV or VDR (whichever you chose to set up when you installed LinuxMCE) and will run as background services.  I'm more knowledgable about Myth, as that's my present "production" environment at home (I'm testing and doing some infrastructure proof-of-concepts on LMCE right now, so it's not "prod" yet).

My MythTV backend (which would be a LMCE core) runs headless, and has two dual-tuner PVR-1600's in it, and it can use the two PVR-150's that are installed in a slave backend/front-end in the family room.  Myth is really great for running a distributed system, and you can run any combination of backends and front ends as your situation may require.  Since LMCE leverages Myth, you inherit those capabilities.

Hope that helps!


Users / Re: Is sysklogd needed?
« on: January 14, 2012, 04:31:11 am »
Nice work brononi!

Are you using the MySQL backend for rsyslog, or is cacti just parsing the logs?  If you want to get fancy, you can enable remote syslog sending on your MD's, listening on the core, and then you can run a central syslog server on the core.  We do that at work for the security guys.  Linksys and OpenWRT routers (and a few others) support sending their syslog output to a syslog host, so you could incorporate that if you wanted...

Just an FYI, Ubuntu 10.04 and newer use rsyslog.  I think most modern distros have ditched ksyslogd for rsyslog.

You might want to put up a wiki page (even if it's on your own user page) so that others can follow in your footsteps and play around with it.  It might stir up the creative juices...

In my case, I've been running collectd on my KVM virtual host and in the guests, as it can hook into the stats provided by libvirt on the host, and I can also see what's happening inside the guest via the collectd daemon running there.  To keep the stats fresh and the system responsive, I put the RRD's on a 4 GB logical volume on an SSD (I do the same with my existing MythTV/MySQL production system, and the MySQL used by my LMCE 10.04 virtual core that I'm testing (/var/lib/mysql for each of them is it's own logical volume on the SSD on the host).  Keeps the databases performing nicely and the system responsive.  Your Cacti implementation looks interesting, as some have used Cacti as a front end for the RRD's provided by collectd.

Thanks, and Happy New Year!


Installation issues / Re: Phone line admin page is missing in 1004
« on: December 20, 2011, 10:07:27 pm »
OK, maybe not an asterisk problem. I tried to reinstall and watch things more closely. Looks like there were download errors. If someone could help me out with the --fix missing syntax then I could investigate further.

sudo apt-get update
sudo apt-get install --fix-missing

That should do it for you!


Users / Re: mythconverg_bac out of control
« on: December 20, 2011, 05:18:41 am »
Here's the permissions for the binaries in /usr/bin {stock 0.23}

-rwxr-xr-x 1 root root   34632 2010-04-23 22:04 mythavtest
-rwxr-xr-x 1 root root 1433788 2010-04-23 22:04 mythbackend
-rwxr-xr-x 1 root root  588276 2010-04-23 22:04 mythcommflag
-rwxr-xr-x 1 root root  359088 2010-04-23 22:04 mythfilldatabase
lrwxrwxrwx 1 root root      31 2011-10-07 21:46 mythfrontend -> ../share/mythtv/
-rwxr-xr-x 1 root root 2209232 2010-04-23 22:04 mythfrontend.real
-rwxr-xr-x 1 root root   38688 2010-04-23 22:04 mythjobqueue
-rwxr-xr-x 1 root root  227312 2010-04-23 22:04 mythlcdserver
-rwxr-xr-x 1 root root   84164 2010-04-23 22:04 mythreplex
-rwxr-xr-x 1 root root   83764 2010-04-23 22:04 mythshutdown
-rwxr-xr-x 1 root root  359576 2010-04-23 22:04 mythtranscode
-rwxr-xr-x 1 root root   18052 2010-04-23 22:04 mythtvosd
lrwxrwxrwx 1 root root      31 2011-10-07 21:42 mythtv-setup -> ../share/mythtv/
-rwxr-xr-x 1 root root  363528 2010-04-23 22:04 mythtv-setup.real
-rwxr-xr-x 1 root root  141736 2010-04-23 22:04 mythwelcome

And the running processes...

 ps -ef |grep myth
mythtv    1309  1308  0 Dec17 ?        00:01:13 /usr/bin/mythbackend --logfile /var/log/mythtv/mythbackend.log --user mythtv
mythtv    3414  7578  0 23:13 ?        00:00:00 sleep 1
mythtv    7577  1309  0 Dec18 ?        00:00:00 sh -c /usr/pluto/bin/  >>/var/log/mythtv/mythfilldatabase.log 2>&1
mythtv    7578  7577  0 Dec18 ?        00:10:34 /bin/bash /usr/pluto/bin/

And the permissions on /var/log/mythtv
drwxrwsr-x  2 mythtv    mythtv       4096 2011-10-08 22:21 mythtv

Hope that helps!


Users / Re: mythconverg_bac out of control
« on: December 16, 2011, 06:17:43 pm »
You can stop an upstart job with:

sudo stop {service name}

Restart it with sudo start {service name}.

To disable it once already stopped, you can either

sudo mv /etc/init/{service}.conf /etc/init/{service}.conf.noexec


sudo update-rc.d {service} disable

(doing it again with enable at the end will re-enable it)

Hope that helps!


Users / Re: mythconverg_bac out of control
« on: December 14, 2011, 05:57:28 pm »
A couple of possibilities for you:

Disable the upstart job that starts mythtv-backend.  Upstart has service supervision, so if the backend keeps dying, upstart will restart it (kind of like monit or daemontools).

Rename the mythtv-backend binary.  Then you can launch it from a terminal with debugging turned on.  I'm not in front of my system right now, so I can't say for certain what the binary is called.  I think it's mythtv-backend.

Hope that helps!


Users / Re: Live Tv Menu
« on: November 21, 2011, 10:17:37 pm »
I have a 19 month old and 3 month old. The 19 month thinks his bedtime isn't until 11:00 :)

It'll get a bit better when they get into kindergarten... it tuckers them out pretty good.  Until then, you're it!  ;-)

Thom, Posde, or gogol4,

Given that it's undergoing inital dev in 10.04, is the plan to make qOrbiter the new orbiter for 12.04?  Will the old one be deprecated or removed when qOrbiter becomes the default?

Just curious...

/Mike (aka parental play structure unit known to two as DAD ;-)

Users / Re: MythTV version
« on: November 14, 2011, 04:36:13 am »

Thanks for the info.  Ticket 1329 has been created in Trac for tracking purposes, and referenced this thread.

I'm not a C programmer, so someone more talented in that department with a vested interest will likely get something done before I can figure out the basics.

Thanks for the pointers on DCE and Device Data.  I'll read up on that for some plumbing I'm looking into.


Users / Re: MythTV version
« on: November 13, 2011, 11:07:34 pm »
The MythTV backend protocol has changed.

Due to brain dead changes in MythTV's backend protocol by truly idiotic developers, You will need to change the code that sends the MYTH_PROTO command, to also output the appropriate version token. This is a random hex number that changes with each official released version. This was a deliberate attempt to keep unofficial clients from just randomly connecting to the backend.  it is explained here.
I have here a patch file for doing it for 0.24. No this will not be folded in until the official mythTV packages upstream change to 0.24, as this completely breaks 0.23 compatibility.

Code: [Select]
Index: MythBackEnd_Socket.cpp
--- MythBackEnd_Socket.cpp (revision 25010)
+++ MythBackEnd_Socket.cpp (working copy)
@@ -17,7 +17,7 @@
 using namespace std;
 using namespace DCE;
-#define MYTH_PROTOCOL 56
+#define MYTH_PROTOCOL 63
 #include "Gen_Devices/AllCommandsRequests.h"
 MythBackEnd_Socket::MythBackEnd_Socket(MythTV_PlugIn *pMythTV_PlugIn,string sIPAddress)
@@ -157,7 +157,7 @@
  string sResponse;
- if( !InternalSendMythString("MYTH_PROTO_VERSION " TOSTRING(MYTH_PROTOCOL),&sResponse,"ACCEPT") )
+ if( !InternalSendMythString("MYTH_PROTO_VERSION " TOSTRING(MYTH_PROTOCOL) " 3875641D",&sResponse,"ACCEPT") )
  LoggerWrapper::GetInstance()->Write(LV_CRITICAL,"MythBackEnd_Socket::Connect couldn't send MYTH_PROTO_VERSION m_bConnected=false");


Could the MythTV Player be extended to read in MYTH_PROTO_VERSION from a configuration file in /etc/pluto?  That way people who need to run a newer version of Myth could simply change the Protocol number.  Obviously, they couldn't leverage newer MythTV features that aren't supported in the Player, but they'd have the basic functionality.

Something for your consideration...


Installation issues / Re: Installation hang on second step. Please help))
« on: October 26, 2011, 07:33:59 pm »
i start install again   on monitor   file /etc/apt/sources.list   and LaunchManager.progress.log
when shown
        Downloading radvd - IPv6 Router Advertisement Daemon...
        Failed to get radvd - IPv6 Router Advertisement Daemon

the lucid  entry is addeded  on sources.list
i edited this fille and remove lucid line .
and now installation is finished .  and show  Sara

radvd is also affecting 10.04 installs, may be an upstream issue.,11988.msg84827.html#msg84827,11988.msg84895.html#msg84895

Hope that helps!


Installation issues / Re: New 1004 Installer Testing
« on: October 24, 2011, 07:06:35 pm »
oops maybe spoke too soon.
apt-get apdate/upgrade today on the virtual core and I'm getting an error installing radvd, error parsing config file.


I had the same issue.  I solved it by copying radvd.conf from /usr/share/doc/radvd/examples/simple-radvd.conf to /etc/radvd.conf, and then doing apt-get install to resume the upgrade.  I would concur with Murdock as it's likely an upstream issue.  The radvd.conf file that was installed by apt had two blank lines in it, which is why apt barfed an error on trying to restart the service.


P.S.  I'm not typing this from home; The path is from memory, so it'll get you in the ballpark if it's not quite right! ;-)

P.P.S.S. I'm running LMCE in a KVM based VM, installed by the new installer.  Core installed fine, I'm having issues with MD's.  Neither has a keyboard attached normally.  One is a Intel Atom 330 with Intel graphics (X fails to configure), the second is an M2NPV-VM using component out 720p (which doesn't initialize).  I expect the M2NPV-VM is fine, and I just need to use the keyboard to pick the output and resolution.  I'll see if I can work it out with the Atom/Intel graphics, and submit a ticket/patches when I get some free time.  Not much of that when you have kids ;-)

Developers / Re: The Vision
« on: October 12, 2011, 05:51:32 am »
Just some thoughts to share with respect to golgoj4's post...

What is LinuxMCE now, and what is it meant to become?  What doesn't it do that it should, what does it do that it shouldn't, and what will it never do?  That's where Vision comes into play... Vision provides scope and direction, which are really important when developers (and developer's time) are limited.  It also affects decisions about architecture and infrastructure, frameworks and integration points...

For example, the requests forum is a real smorgasbord of people's wants and desires for LinuxMCE.  What makes it in, and what doesn't?  The usual answer is, those that write the code can put it in; but does it always make sense to put some something in, just because you can?  Many of the requests are for portal and app server functionality, like the stuff Amahi does.  Is that a space LinuxMCE wants to or should be playing in?  If so, what framework(s) get used/re-used to provide that functionality to maximize on the features delivered vs time invested by developers.  Another example would be Dianemo using MythTV and saying that it provides everything that VDR could do.  Does that mean VDR support is a duplication of effort?  Should it be deprecated/removed for 10.04?  (Note: I'm not saying that, but with a common vision and direction, these kinds of things have to be examined and decided upon).  Vision provides scope and direction, which influence quality, time and effort.

I bring this up only because one of my roles in my day job is to encourage the DBA's and app admins I support to consider the full life-cycle of the services they support, both for the here and now and for the end state, because they do effect one another.  The end state (the vision) effects the here and now, and the attitudes towards the here and now (scope and direction, and consequently time, effort, and quality) effects the end state.  I do this as someone who inherits the "lost turds"; the services that were rushed out to meet the immediate needs of the here and now, with no consideration for the long-term life-cycle needs of that vital service.  Specifically, how to keep it alive and well, and evolve it as needed.

LinuxMCE is different from most projects or applications, and not just because of it's complexity.  When fully integrated into a home environment, it becomes a daily use tool, one that people can literally grow up with; in some cases, children may become teenagers or adults, not remembering a time before LinuxMCE was a part of their lives.  That's why Vision is so important... As a collection of OpenSource components, LinuxMCE's foundations have a life of their own, which will drive the evolution of LinuxMCE to a large extent.  As the LinuxMCE project itself grows and evolves, more people will come about to try to help out in some way.  With a shared collective vision and direction, it becomes easier to rally the troops and make strategic choices; without, it may be like herding cats...

I'm going to duck back out now and continue to examine the plumbing in 10.04... still got a lot to figure out before I can hope to make a contribution, but I hope to make a dent in posde's points 1 & 2 eventually...


Users / Re: LMCE newb... what do I need?
« on: September 09, 2011, 06:44:03 pm »
any good docs you could point us to.
Currently I am running KVM on a proxmox ve server
I do have an 80gb ssd I could add for mysql use any hints on that


I think I'll write up a wiki page of tips and tricks for virtualization that are specific to LMCE, so it doesn't get lost in this thread.  Mostly, it's paying attention to a whole bunch of little details, where the aggregate cause a significant performance boost.

Proxmox VE is a Debian-based distro, with KVM and their own perl-based management GUI.  I haven't used it myself other than quickly trying it out, but I believe it uses LVM as an option with local storage.  If that's the case, then what you might want to do is carve off a 20G LVM slice of the SSD as your LMCE boot/root disk, and mount other spindle-based slices as /var/log, /tmp and /home.  That way, you're placing your I/O workload onto hardware that will lend itself well to the types of I/O that will be hitting it.  I think you'll find Orbiter Regens will just fly!  It'll take some work to do it that way; you'll either have to install from scratch to the SSD, doing a manual partitioning of the virtual disks, or restore a clonezilla backup to the SSD, mount the spindle slices to temporary mount points and rsync the data over (and then delete the original copy), then pivot the disks into their final mount points.  There are some SSD specific optimizations that can be done in the guest OS, plus the disk I/O schedulers need to be disabled for the virtual disks.

Basically, the kernel disk drivers assume that it's writing to real disks, so it queues up and re-orders operations in order to take advantage of where the heads are over the disk platters.  The host OS is already doing this, so we don't need the guest OS doing it also, because it will simply be working at odds to what the host (who actually controls the access to the disks) is doing.  So, we add "elevator=noop" to the kernel boot parameters of the guest.  Where to do it varies between grub and grub2, so I'll leave that as an exercise for the wiki.  You can change it on the fly with echo noop > /sys/block/[s,v]d[a-z]/queue/scheduler.  You can also cat that file to see the one in brackets that is presently selected.  You can do that change on a per-disk basis in /etc/rc.local, by echoing the appropriate scheduler to the appropriate disk.  SSD's like the deadline scheduler better, as it round-robins all the processes seeking I/O time.

You also want to configure the device on the virtualisation host as a "Virtual disk" rather than an emulated IDE or SCSI, as that will leverage the paravirtualized VirtIO drivers.  You'll also want to configure the network adapter as a VirtIO device as well, as the drivers for both network and disk are included in the 8.10 and 10.04 kernels.  VirtIO gives a huge performance boost (near native, 95~99% to the physical hardware), as the hypervisor is not having to emulate various hardware registers in software.  Networking between VM's using VirtIO is simply a memory-to-memory copy, and that occurs at orders of magnitude faster in RAM than on wirespeed.

There are some other parameters I've put in my libvirt config files to disable caching, mount point options in the guest OS's to optimize for the underlying slices, and some other application specific tweaks, so I'll go through my stuff at home and make a proper wiki page for a virtual LMCE core.  I've got some half baked ideas for  some infrastructure work on LMCE that will do some auto-detection of the underlying core and md hardware (physical and virtual), and will make optimizations based on what it finds.  It's something I'm already working on at the day job, so I'll need to work up a proof-of-concept at home for how I can abuse it for use in LMCE.  It's going to take a while to get there, as I have to poke around under the hood in LMCE to see how things are working presently, and how (and when) this new infrastructure could be integrated non-disruptively.

Hope that helps!


Users / Re: LMCE newb... what do I need?
« on: September 08, 2011, 09:46:25 pm »
I have found vm's work fine for testing for both core and MD's but as far as a production system It don't cut the mustard.

Do you guys have stock in your electric companies ;)
You can run lmce on a 35 watt core

just my 2cents


In my day job, 95% percent of what I work on is virtualized, and I work with 4 different hypervisors on three different hardware platforms.  The stuff that doesn't get virtualized is the stuff that will keep a 32-core box running flat out all the time on it's own. VM's run fine when the OS is tweaked to use paravirtualized drivers and some of the default behaviours of it's I/O are changed.  It's also helpful to have an end-to-end understanding of the hosting hardware platform, the hypervisor and it's various schedulers, and the underlying infrastructure like SAN and network, plus where you need to tweak under the hood to optimize for the workload.  At work we regularly get 80-100 VM's onto a big honkin server, and the client's don't know the OS instance is virtualized.

As for my home system, I'm running LMCE plus an average of 5 other VM's on a 45W Dual core CPU.  LMCE (8.10) actually places the highest load on the system, in part do the the age of the virtualized drivers and the kernel itself, plus inefficiencies in ext3 filesystems running in virtuals.  10.04 with ext4 file systems and VirtIO drivers plays a lot nicer in a virtualized environment.  I'm actually trying to avoid having to upgrade the CPU, but if I end up running Windows as a virtual I'll have no choice; Windows (even 7) takes up wayyyy more resources than an equivalent Linux install.  Energy efficient CPU's in Socket AM3 are getting harder to find unless you special order them from NewEgg or something like that...

LMCE Core (1vCPU, 1.7G RAM)
Zarafa Mail Server (1vCPU 1.5G RAM)
Misterhouse Home Automation (1vCPU 512M RAM)
Astaro VPN endpoint (1vCPU 512M RAM)
Ubuntu Virtual Hosted Desktop (1vCPU 784M RAM)
LMCE MD and other test VM's (1 vCPU and various RAM sizes, keeping under 5G total to leave a gig for the host OS)

Plus MythTV and other external network services that are running on the host OS.

At this point in time, I'm more I/O bound than CPU bound, but less so since I added a 40G SSD which I've carved up through LVM and presented as separate disks to instances running MySQL.  It gets mounted inside the guest at /var/lib/mysql, and the db files sit on it.  I'm seeing ~1400 IOPS using the VirtIO drivers in the guest to the LVM'd SSD, compared to ~100 IOPS on my RAID-1 set on the host, using the Oracle Orion test tool.  Disks are better for sequential I/O (like media files and logging), and SSD's are better for Random I/O (like OS drives and databases).  Using Orion on the raw SSD block device from the host nets me ~25K IOPS (that's avoiding the FS and the FS cache).

One thing that tends to foul up most people using VM's is throwing more vCPU's at an instance in order to improve performance.  More often than not, more vCPU's will hobble you, as the hypervisor has to find the free number of cores available at the same time as you have vCPU's configured, before it will schedule workload on the cores.  So, on a dual core host, a VM with 2 vCPU's would need both cores free _at the same time_ before the hypervisor's scheduler will dispatch the guest onto the cores.  So, host processes will be in competition with the guest for CPU time, as those usually get dispatched individually onto cores.  If you have more than one 2vCPU guest, then they start to get into contention for CPU time, with each other and the other guests.  If they have heavy I/O, then the host is competing with the guests for CPU time in order to perform the I/O, and all are starved out as a result.  Then people complain that virtualization sucks...  ;)

Food for thought...


Pages: 1 ... 11 12 [13] 14 15