DHCP - combine with other DHCP - HOWTO?

[Curtis]

Hello all -

I'm completely new to LMCE, but have been reading everything in the posts that I can, and have tried installing a 710 B3/4 a few ways on test equipment to see how it works (or not), and then take notes.

I'm an IT veteran, a home automation nut, a videophile, a song-lover, a game-player, and an amateur digital photographer.  When I saw the LMCE video, I was also slack-jawed in amazement, because there, for the first time, was everything I've ever wanted my main Linux box to do...  and then some.  *cough* But it's not quite ready just yet.

Unfortunately -- as has been mentioned before in these forums -- the video really is somewhat misleading, as the project is nowhere near the implied "It all Just Works (tm) Today!".  It will indeed work, but to do so you must provide it very specific hardware, and give up your previous networked environment.   DCERouter replaces your previous networking, and for some good reasons.  (I applaud the idea of MAC discovery PNP -- very clever.)

For some folks -- heck, many folks -- this will work fine.  Most people can't spell DHCP let alone care what it is, or what's doing it.  For the hard-core geeks among us however (and you know who you are), being forced to change everything you've done up to now is a rather prickly pear to pick.  (LMCE developers have the same justifiable reaction when someone utterly new staggers into the forums with "You should have done it THIS way!".)

Now, I will likely never use LMCE for both my authoritative DNS and DHCP, for many reasons.  Some involve security, some involve design, and some involve inertia. I don't want all my network-config eggs in one server basket, and I've used BIND and DHCPD for many years at work, but at home they are simply overkill.  I use a wonderful little app named "dnsmasq" ( http://www.thekelleys.org.uk/dnsmasq/doc.html ) which does DNS and DHCP (and more) very nicely, using one simple config file.

So for those of us who would prefer to see LMCE do just the media stuff, and no network management, we should be able to do so.  I should be able to eliminate the firewall, DNS, and DHCP from LMCE, leaving it simply another single-nic server on my network, delightfully performing those things I wish it to.

Then the question becomes "How do I get the device discovery incorporated?"  And it's needed too: try configuring an HDHomeRun manually, or letting the discovery scripts do it, and pick which *you'd* prefer.

I'd say there are two ways: the simpler way is to run a script on the DCERouter which processes a MAC address text file you've put together.  This would especially be nice for test environments, to debug and polish some specific installation script you want to repeatedly run -- even if the device really isn't there.

The slightly more difficult, but better-automated method would be to obtain those MAC address from something else.  I saw a good thread here talking about "arpwatch" supplying MACs to the core.  Another way I myself could do it is via using a keyword in my dnsmasq.conf file which will launch a script when it assigns an IP address.  Either of these methods would work, and if they use MessageSend as a hook, should be able to present DCERouter with a new mac address.

As a nice seque, that brings up another point.  Everyone who has ever posted about the message-handling  features in LMCE being the real show of this project is correct: No other app does this, and having all your disparate applications unified by this brings up mind-boggling possibilities that would make any geek croon with delight.

I would like to suggest that this message-handling can also be used in another way:  If I can compile "MessageSend" on another system -- pick a box, any box -- and use it to send messages to LMCE, then I can add whatever activities that box is doing to LMCE in a gentle way.  My little DebianSlug box running "heyu" ( http://www.heyu.org/ ) for all my X10 needs can be told to spew that activity to my new LMCE box, where I can Do Stuff based on that information, and also tell my little Slug box what to tell my house to do.

Yes, of course I can add that into LMCE directly using the CM11 code, but what if I don't want to?  What if I legitimately don't *need* to?  What if LMCE can do HA itself, or can use messaging to tell whatever other system I use for HA (or DNS, or DHCP, or...) to do something?  One of messaging's greatest strengths is that, by definition, not everything has to be done on one system.  (In fact, this "messaging hook" allows anyone to develop new functionality for LMCE -- without changing any code!)

I can choose to keep them separate, or integrate some HA scenes into LMCE, leaving others in my HA server, or I can choose to let LMCE do all my HA.  It's all about choice -- and choice, folks, is what FOSS is all about.  I can gracefully ease my existing applications into LMCE as little, or as much as I want.

Additionally, I may have other servers or devices all over the place -- all over the world, if I wish -- using "MessageSend" with my LMCE box.  The sky's the limit.  (Ok, if you want to get technical, XML-RPC, Jabber, or even SMTP would work better there, but hey -- the possibility exists.)  Maybe I want my LMCE system to find me wherever I might be in the house to tell me that my annoying relative has logged into chat by playing "Comfortably Numb".  Maybe I want my LMCE system to send some server a message to page me a letting me know it's run out of space.  If LMCE and other servers can discuss anything together, then what can be done with the result?  Anything.


Ok, so I've rambled a bit (especially for a first post!), but hopefully I've stayed on topic enough to show that there are (or should be) easy ways to use LMCE without it having to be your one-stop-shopping for network management, etc.  It *can* be, and likely should still default to doing so, but it doesn't *have* to be.  It should work fine so long as it has all the information it needs -- by whatever means it can be supplied.  The messaging hooks should allow both concepts to co-exist peacefully.


So! <rubbing hands briskly>  Is there any reason I can't compile LMCE messaging code on an ARM processor?

Thanks so very much to the LMCE team for all their hard work, and to Pluto for releasing the code.  This project is the most exciting, innovative, and delightful application I've ever seen!

Thanks,
Curtis

[tschak909]

not at all, in fact, the DCE Router and its partners in crime are often compiled for embedded systems. This is how Pluto makes their money.

But if you're going to start working on splitting apart the whole appliance, work with us on it. We are all working on this together, and would appreciate the extra hands. This project can not be developed in isolation, and requires that we all work together.

-Thom

[gazzzman]

OK just my 2p (lets see if anyone can see anything wrong here?)
one point first
if you install without DHCP running on your router Lmce will assign itself 2 iIP addresses 192.168.80.1
and internal 192.168.80.254
it cannot connect to the internet like this initiallly and brings it's own problems so
(again in my opinion)
assuming you dont need Lmce as a router device for your internet and you only want 1 NIC
here is what SEEMS to work out...
during install have DHCP on on the router WITH ONLY LMCE BOX CONNECTED!
once install is complete set up the WAN side to static at its current IP settings IE if your router is at 192.168.0.1 then set your LMCE ext IF to 192.168.0.254 then turn OFF DHCP server on the router...
now at your roter/switch you have 2 subnets available...
192.168.0.X original LAN and 192.168.80.X LMCE lan (with DHCP)
at this point if you want to share anything belonging to the LMCE LAN you must either turn off LMCE's firewall or configure it accordingly..
dont forget LMCE is INSIDE your existing firewall entirely so turning off LMCE's firewall should be fine
now.. anything you don't want to commit to LMCE's control should be set with a static IP in the subnet 19.168.0.X LMCE will not see nor try to control this
anything you want LMCE to use.. you enable DHCP on
that will end up on the 192.168.80.X subnet
this way both networks can share one switch and  one lot a cabling and there should be no unwanted interaction
if you need to access something inside LMCE's subnet from your original network (say a NAS)
you can do so by it's IP address (or presumably it's DNS name) THROUGH LMCE's router 
anyone see anything wrong with this solution??
(my background is in  networking NOT Linux)
cheers
Gazzzman

[1audio]

In a few short messages the extremes of LinuxMCE's concepts have been touched. At one end its a very customizable platform for doing many complex things. At the other end it was designed to be a consumer appliance with all of the complex gears hidden and hopefully as user friendly as an iPhone. It was designed to support distributed control and it some of the future power will come specifically from that. It's conceivable that, for example, a security panel vendor would build a DCE engine into his box that can discover another DCE engine in the network and automagically integrate. Another future goal would be to have redundant systems for higher reliability. Important for larger installations.

All of the code for the core engines are C++ so they will go on to Arm or Mips or whatever. Some stuff really won't, like the Asterisk app, for now. But its modular and you use what you need.

However networking has been one of the most challenging aspects of supporting the system. Consumers barely know what a network is let alone what DHCP is. So for an appliance it was necessary to hatch the 2 nic solution. There were some discussions about having the box automatically reconfigure a router but thats way too difficult for a practical product. Too many different routers. There was a draft spec for having DHCP servers communicate with each other in a network but that was dropped for lack of interest. Here is where it would really help.

Its entirely possible to to the advanced configuration defined at the beginning of this thread, but it requires a firewall/router that can be configured. The generic Netgear/d-link/2wire router doesn't support 10% of that configurability. The big issues surface with the PXE booting. The next level with the PNP discovery. The process that is supposed to update the firmware in the Cisco phone steps into all of the stages of the process. And the HDHomeRun tuner is another example of exploiting the network smarts. In the future there will be more devices like these. And then there is the VOIP and NAT transversal, another thing that works best at the edge of the network.

The security of having all the functions in one box is an issue but it may be resolvable to some level of confidence. And having one box running should be more "green" than two. However I would not suggest pitching a lot of work configuring a complext system if it can be massaged to work as described.

[royw]

Architecturally, I think some of these issues could be addressed with a relatively minor change.  Let's take the DHCP PNP discovery as an example.  There is a script that runs that tails then parses the dhcpd log file.  When a device discovery is noticed, then the script adds the device to the database.  This script pretty much couples the dhcpd to the core box.

What if instead, we set up a web servlet (soap, rest, whatever) to have the CRUD API for managing devices in the database.  Then change the Dchcpd-Plugin.sh to access this API using curl or whatever instead of directly accessing the database.

Same functionality as currently, but with the added benefit of having an external API.

This would enable allowing device discovery from other boxes or even using alternative discovery procedures (ex, polling internal network with nmap).

Just a thought.

Have fun,
Roy

[tschak909]

cool, implement it!

-Thom

[royw]

OK, but it'll be a couple of weeks to perk to the top of my todo list (finish building replacement workstation, finish building sewing machine cabinet to keep wife happy, finish my media manager rails app).   

[Curtis]

Just a quick reply here in the thread for completeness' sake:

Roy - Cool!  (and, FWIW, I understand completely about time, priorities, and wifely requests coming first.)

Thom - Not to worry, my understanding of the project is nowhere near the level you and others have -- I'm still repeatedly looking at the docs, code, and overview trying to understand where things hook together.  I understand and appreciate (!) the *potential* I see here... but I must start with tiny baby-steps.  I plan on simply trying to compile MessageSend for ARM so that I can send DHCP notifies into the DCERouter once I have it back up, and see what happens.

As an aside: If anyone else sees this -- and I may just post it into the "I'm New here how do I start" thread -- don't try to play with LMCE on old hardware.   I mean, most of us Unix geeks are used to trying a new app on some older hardware "sandbox" where it's fairly isolated and we can poke it with a stick to see what happens.  We know it will likely be slow, but that's ok because we're just "checking it out".  After trying this with LMCE a few times -- and reading yet more in the forums -- I've come to the conclusion that doing this is a recipe for frustration with bouts of hysteria.  I will find me some good (current/fast) hardware, with an Nvidia card that can handle at least UI2, and *then* I will poke it with a stick.  This should lessen any histrionics.  (It won't eliminate them -- this is a computer project, after all.)

Thanks!
Curtis

[blacklotus]

without networking experience, yes your right...

I guess hari, my problem is this.  I have this problem of assuming anyone seeking help from the forums is also seeking to educate themselves.  If I don't put this information out there, then they will never educate themselves.  You see, I don't like giving people simple solutions, that reminds of the old ass saying "Give a man a fish, feed him for a day, teach a man to fish, feed him for a lifetime"

Basically what i'm trying to say, is I treat everyone equal, so if someone asks about getting something working, I will give them my opinion based on my own expertise, maybe dumb down some of the terminology.  And I have no problem whatsoever explaining or walking people through my proposed solutions.  Maybe they will learn something in the long run...

THANK YOU thank you thank you.
that's my point as well. why does everyone assume that no one here knows anything about networking here? i know a lot about networking but my *nix skills are lacking. anyways, i have my home network divided into 4 segments and 2 of them are vpn'd back to my office (lan+voip). the other one is the rest of the home + wireless.

i don't mind having the lmce box act as dhcp on the insecure vlan, but i will not replace my firewall. i would like to set up some of my cisco ip phones on the lmce box on the untusted vlan.
i also would like to add some access from one of my pc's on the trusted lan to the core so i can't (or don't want to) use nat'ing on the lmce. i also don't like the idea of the lmce dhcp server forcing its own ip as the gateway address. is there any way i can change the dhcp config to hand out the firewall ip as the gateway?

i don't want to take out the internet just because i take down the lmce core for maint, etc.

why can't they make a FULLY CONFIGURABLE single nic install where you can change all the settings during setup?

[Lexje]

Hi folks,

Here's some more 2cents:
Since the beginning of this year I've had my fair chair of LMCE. Fortunately I'm in a position I've a lot of hardware lying around and I can afford to spend some on testing LMCE. I realize very well that LMCE is a SUM of quite a few other fine projects s.a. MythTV, Pluto, Asterisk just to name a few. Before LMCE I didn't have any experience with MythTV which already offers quite some possibly 'complicated' capabilities.
It is also true that the 'average' hobbyist will far from spontaneously realize the intrinsic interwovenness of these components. Therefore it is logic and natural for a lot of people to get frustrated, in spite of the beauty of this project.

I, for one, purely out of practicality find it extremely annoying having LMCE hand out DHCP.
At my place lives a 'small' network holding amongst others 3 networked printers, a networked scanner, NAS etc.
For various reasons it is just plain 'easier' to have some of these have fixed IP.
Until now I have not met a solution that will provide seamless DNS in conjunction with DHCP specifically for those printers & scanner. Also it is not obvious at all to have samba serve out printer drivers for these specific printers...

To accomodate LMCE at various moments I have shut down my firewall DHCP etc.
Each time I've ran into situations where e.g. the TV card (HVR 1100) would not work or, as it is a growing system, that I have to reboot or reorganize the LMCE machine.
At such moments it is a plain misery to have all of your 'small' network disfunction because IP's have changed or subnets have been altered..

Also I have tested +/- 5 VGA cards, all nVidia type, and have found them not easy at all to configure using the AVwizard. I feel the AVwizard is marvelous, but not yet finished. How do you know what connector you have to connect to? Often you have one DVI and one VGA connector, sometimes they have 2 DVI, sometimes you have an extra S-Video output, at other times you have one connector with a special cable splitting in 2 to connect multiple monitors. It is often very difficult if not impossible to find out what will be the primary output a specific card will default to.

Now my suggestion:
I have been working with a specific server type Linux distro named Syn-3. It's Slackware based. It is a Dutch group of guys that have put together this specific distro. It has various interesting points into which I will not digg into any deeper. Just to name a couple: installs a complete server in less than 15 minutes! (it's simly the fastest I know...) and now it gets interesting:
It has a completely configurable firewall and DHCP server aboard, off course DNS etc are also present.
Now what is intresting about this?
As the system will, depending on number of nic's present, divide the setup in several zones, like LAN, WAN, DMZ, it has to communicate with the user which adapter is which.
This is solved in a way that the PC will beep once, twice, or three times (at various tones) if you connect / disconnect a network cable.
For a starter this is a foolproof way of determining what is connected to what.
I don't have the technical knowledge, but could this be conceived for VGA adaptors as well?
Can the system 'measure' if a device is connected to output 1 or 2? I'm not 100% sure this is possible for all devices but maybe this is a way to make the 'Wizards' more fool proof?

What is also interesting in Syn-3 is the fact that it has a graphical control interface onboard allowing you to monitor various services an shut them down or restart.
I really think there should be a more integrated control interface (perhaps webmin(?))
Also it would be interesting to have a little more feedback from the system, as it quite often happens that reaction time is longer than the average users patience, leading to unclear situations.

Just my 2cents,

Erwin

[tschak909]

These are great suggestions, but we need more people on the dev team to help investigate and integrate these solutions.

Our only concern right now, is to create a particular hardware integration path that works well, and that is the hardware configuration we recommend.. If you go outside of that, you're on your own, we do not have the man-power to support it.

-Thom

[Lexje]

Thom,

Thanks for your input.
I'm not a developer, but I'm sure willing to help where help is possible and acceptable.
Suggestions always welcome!
After all, it's all about doing it together, isn't it?

Erwin

[hari]

that's my point as well. why does everyone assume that no one here knows anything about networking here? i know a lot about networking but my *nix skills are lacking. anyways, i have my home network divided into 4 segments and 2 of them are vpn'd back to my office (lan+voip). the other one is the rest of the home + wireless.

not no one, but ask 100 people that run a computer at home about subnetting or static routes. I'd assume only one or two could answer the question.

why can't they make a FULLY CONFIGURABLE single nic install where you can change all the settings during setup?

who is they? can you?

best regards,
Hari

[blacklotus]

that's my point as well. why does everyone assume that no one here knows anything about networking here? i know a lot about networking but my *nix skills are lacking. anyways, i have my home network divided into 4 segments and 2 of them are vpn'd back to my office (lan+voip). the other one is the rest of the home + wireless.

not no one, but ask 100 people that run a computer at home about subnetting or static routes. I'd assume only one or two could answer the question.

why can't they make a FULLY CONFIGURABLE single nic install where you can change all the settings during setup?

who is they? can you?

best regards,
Hari

ok i'll agree with you about the basic home user not knowing about subnets, vlan's etc BUT i'm not even asking the LinuxMCE team to make the core work with vlan's and multiple subnets, although that would be nice. all i'm asking is that you give us the ability to allow the core to work on ONE subnet, us who know networks can take care of the rest.

for example:
during the initial setup process it could ask for networking setup with a simple progression of menu's with this type of structure:
-----------------------------------------------------------------------

1:automatic (current config, continues on its merry way)
2:manual
    1:Firewall Mode (NAT enabled)
         set LAN nic and ip address/mask
         set Public nic and ip address/mask/gateway (options for DHCP, PPPoE/adsl also available)
         the dhcp server is automatically enabled with the usual options
         other things like RIP config/static routes could be configured through web admin
         DONE
    2:Router Mode (no NAT)
         set up interfaces and ip addresses/masks (VLAN capability would be nice for future!)
         1 enable DHCP on interface(s)
              choose interface(s) to enable dhcp on
              under each dhcp server be able to set at the very least:
                  set ip address pool/range
                  set netmask (could be auto assigned to same as interface's)
                  GATEWAY ADDRESS (could point to another gateway/firewall or be
                  this machine if routing to NAT Firewall)
                  set dns server(s)
                  any other dhcp options as fancy as you want to get like like custom scope options, WINS, etc
                  it will auto configure itself as the boot server
              other things like RIP config/static routes could be configured through web admin
        2 do not enable DHCP
        DONE
    3:Single Adaptor
        set up ip address/mask and default gateway
        do you want to set up DHCP server?
             1. yes
                  set ip address pool/range
                  set netmask (could be auto assigned to same as interface's)
                  GATEWAY ADDRESS (could point to any other gateway/firewall)
                  set dns server(s)
                  any other dhcp options as fancy as you want to get like like custom scope options, WINS, etc
                  it will auto configure itself as the boot server
             2. no
        other things like RIP config/static routes could be configured through web admin
        DONE
-----------------------------------------------------------------------------

what do you think? i think a solution like this would satisfy and take care of the needs or many more people than right now.

the other thing you have to think about when it comes to network knowledge are these:
a. even if some people don't know about all the networking options, it doesn't mean that they wouldn't learn to use it more if it were easier to configure and more accessible to them
b. once LinuxMCE matures more these systems will be installed more by professionals in high-end homes perhaps. these installers would definitely need more setup sonfigurability to get the systems to work on the myriad of setups out there.

Are these things i'm requesting THAT impossible to do? before you tell me to code it myself i'll apologize that i lack the skills and the time to dedicate to learn those skills at that moment.

What i can do is lend my assistance in helping to layout configuration options and (hopefully) explain what would make greater flexibility more accessible to the general user AND more the knowledgeable ones.

[hari]

ok i'll agree with you about the basic home user not knowing about subnets, vlan's etc BUT i'm not even asking the LinuxMCE team to make the core work with vlan's and multiple subnets, although that would be nice. all i'm asking is that you give us the ability to allow the core to work on ONE subnet, us who know networks can take care of the rest.

yeah, thats a much wanted feature. It also would enable us to provide a Live CD. But atm pluto and the lmce dev guys use it as an appliance. The 2nic setup fits good for that. It's not that we don't want a solution for that, but we have limited resources. As you have good networking experience you maybe want to help. I could assist if you run into troubles.

what do you think? i think a solution like this would satisfy and take care of the needs or many more people than right now.

I think the biggest problem is getting rid of dependencies to the actual "assumptions" done in many scripts/pieces of code, not the installer itself.

a. even if some people don't know about all the networking options, it doesn't mean that they wouldn't learn to use it more if it were easier to configure and more accessible to them

thats an support issue. We don't have the manpower to guide everybody through basic networking in irc or on the forum. Guys like Orionsune really attacked us for not trying to push his single NIC/DHCPless setup ideas. But he does not really supports those users, either. We have to focus on LMCE issues and can't assist with thousands of possible home network setups. Thats the reason why the dual nic approach was chosen initially by pluto. And most of the people targeted by the pluto appliance don't care about the network specifics in their setup. They want things to simply work and don't insist on mixing the LMCE network with some already running business/asterisk/dhcp/whatever server.

Maybe its no ideal solution but the 2nic setup is a good compromise.
Btw, you don't have to give up your existing firewall, just daisy chain them.

b. once LinuxMCE matures more these systems will be installed more by professionals in high-end homes perhaps. these installers would definitely need more setup sonfigurability to get the systems to work on the myriad of setups out there.

We fully agree on that. It's also needed for a Live CD.

Are these things i'm requesting THAT impossible to do? before you tell me to code it myself i'll apologize that i lack the skills and the time to dedicate to learn those skills at that moment.

What i can do is lend my assistance in helping to layout configuration options and (hopefully) explain what would make greater flexibility more accessible to the general user AND more the knowledgeable ones.

you don't have to apologize, feature requests are always welcome. But at the end of the day somebody has to develop the code. Your feature wish is not impossible but i'm not aware that sb. working on the code has dedicated time to that. Besides bashing the actual way things are done there were no working contributions to the ethernet plug and play problem without DHCP besides my arpwatch patch. And one has to check every feature of LMCE in both setups and protocol which parts exactly break on single nic and which without DHCP. There are also some hardcoded bits of "192.168.80.1" in some scripts iirc.

A good start would be to identify the pieces of code that need change. Everybody touching one of those subsystems then could try to come up with solutions for that part. So we can solve the issue over time.

best regards,
Hari