Single -> Double NIC... No Adminsite [RESOLVED]

[Matthew]

I switched back from a single NIC setup to double NIC, and now I can't access (with some weird exceptions) the LMCE hybrid on its external interface.

I had switched my LMCE to a Single NIC and it was working fine (no LMCE devices, just a hybrid). But I want to start testing/using my Cisco 7970 IP hardphone, which needs to netboot, so I decided to go with the (LMCE default) double-NIC setup. I installed a 3Com 3C905C-TX PCI card in the machine (which shows up in lspci):

Code: [Select]

# lspci |grep Ethernet
05:08.0 Ethernet controller: Intel Corporation 82801DB PRO/100 VM (LOM) Ethernet Controller (rev 81)
05:0a.0 Ethernet controller: 3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 30)
Then I started following the "Single to Double NIC" wiki instructions. In the first step, "Web Admin: Advanced menu > Network > Network Settings", I saw that LMCE saw 2 network cards, though it offered to config them as eth0 (configurable with either DHCP or static IP/netmask/gateway) and eth0:0 (just a static IP# config), which was strange, because eth0:0 is virtual, and I've got 2 physical cards. But Linux (ifconfig) does see the 2nd ethernet interface as eth0:0 . Following (by inference) the wiki instructions, I set the eth0:0 IP# to 192.168.1.10 (leaving the eth0 interface manually set to 192.168.0.10/255.255.255.0). When I submitted the change, the page did not return, though the eth0:0 IP# is changed and apache is still running. The Adminsite is gone, as the external interface IP# no longer seems to have a webserver listening.

Code: [Select]

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0B:CD:01:72:4C 
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20b:cdff:fe01:724c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2033061 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10922155 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1591419024 (1.4 GiB)  TX bytes:2071456480 (1.9 GiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0B:CD:01:72:4C 
          inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:9936648 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9936648 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2823888630 (2.6 GiB)  TX bytes:2823888630 (2.6 GiB)

Code: [Select]

# ps aux |grep apache
root     20509  0.0  0.3   2880   756 pts/15   R+   08:38   0:00 grep apache
root     25556  0.0  2.7  23348  6728 ?        Ss   Dec17   0:00 /usr/sbin/apache2 -k start
www-data 25574  0.0  1.4  23348  3484 ?        S    Dec17   0:00 /usr/sbin/apache2 -k start
www-data 25575  0.0  1.4  23348  3480 ?        S    Dec17   0:00 /usr/sbin/apache2 -k start
www-data 25576  0.0  1.4  23348  3480 ?        S    Dec17   0:00 /usr/sbin/apache2 -k start
www-data 25577  0.0  1.4  23348  3480 ?        S    Dec17   0:00 /usr/sbin/apache2 -k start
www-data 25578  0.0  1.4  23348  3480 ?        S    Dec17   0:00 /usr/sbin/apache2 -k start
I expect that apache is now listening on the eth0:0 interface. I haven't tested that, because I don't want to believe that LMCE exposes the Adminsite to only its internal LAN segment, rather than to other devices on the external LAN segment with the hybrid. I hope someone can explain otherwise.

Code: [Select]

# cat /etc/network/interfaces
auto lo
        iface lo inet loopback

auto eth0
iface eth0 inet static
        address 192.168.0.10
        netmask 255.255.255.0
        gateway 192.168.0.1
auto eth0:0
iface eth0:0 inet static
        address 192.168.1.10
        netmask 255.255.255.0
Meanwhile, there are other weird symptoms: I'm still logged into a working ssh session started before I changed the eth0:0 IP#, but I can't initiate new ones to the old IP# (which should still be configured on eth0). Pinging that old IP# fails with 100% packet loss.

I'm not even sure where to change the eth0:0 IP# back to the old 192.168.0.10 IP#, to undo that last (catastrophic) change, with my ssh or console access. I guess I can just reinstall LMCE from scratch, but there should be a way to switch back & forth between single and double NIC setups. Any advice?

[Hagen]

The Adminsite is gone, as the external interface IP# no longer seems to have a webserver listening.

true, it should be blocked by the firewall. (in my experience)
Try switching off the firewall?

I expect that apache is now listening on the eth0:0 interface. I haven't tested that, because I don't want to believe that LMCE exposes the Adminsite to only its internal LAN segment, rather than to other devices on the external LAN segment with the hybrid. I hope someone can explain otherwise.

Well... believe
In my experience, and logic would also suggest this, the LMCE core was supposed to be the house router, there should be no reason for 'outsiders' to be able to connect to the admin site. But I do believe you can open up for this in the LMCE firewall settings. (I have not seen a need for this myself though)

[Matthew]

The Adminsite is gone, as the external interface IP# no longer seems to have a webserver listening.

true, it should be blocked by the firewall. (in my experience)
Try switching off the firewall?

I expect that apache is now listening on the eth0:0 interface. I haven't tested that, because I don't want to believe that LMCE exposes the Adminsite to only its internal LAN segment, rather than to other devices on the external LAN segment with the hybrid. I hope someone can explain otherwise.

Well... believe
In my experience, and logic would also suggest this, the LMCE core was supposed to be the house router, there should be no reason for 'outsiders' to be able to connect to the admin site. But I do believe you can open up for this in the LMCE firewall settings. (I have not seen a need for this myself though)

Ah, pointing those services to the internal segment makes sense if LMCE is the router for the entire LAN, which the firewall provides, and the external interface just connects to the WAN.

So I switched around the topology to do that. But there's no DHCP server running on the internal segment (dhclient on a host connected to a switch to the internal interface fails with "DHCPREQUEST ... DHCPDISCOVER ... No DHCPOFFERS received."). How do I switch on the LMCE DHCPd without access to anything but the LMCE hybrid console?

[Hagen]

You should be able to get the web admin up on a hybrid if you go to the KDE desktop.
I have been playing around a bit with the lan settings myself, and it's very logical.
Remember that the order of the nics in the 'list' is important as to internal and external NIC

[tkmedia]

eth0 is a physical net card
eth0:0 is a virtual net card bound to eth0

If you add another physical net card it becomes eth1
I also have seen the new card become eth0 and move the original net card to eth1
/etc/network/interfaces should no longer have virtual cards.

simple interface example
auto lo
   iface lo inet loopback
   
auto eth0
iface eth0 inet dhcp
auto eth1
iface eth1 inet static
   address 192.168.80.1
   netmask 255.255.255.0


Hope this helps

[Matthew]

I rebooted LMCE, and it turns out that there are several problems, not all solved.

Linux switched the motherboard ethernet interface from eth0 to eth1, and added the PCI ethernet card as eth0. Not only nonintuitive, but I had of course rebooted before after installing the PCI ethernet, and ifconfig didn't show it - now it does. And how is a script to determine which ethN interface is assigned to a given card? Is it arbitarily numbered? Is the MAC address required to determine it? Can the numbering/order be specified somewhere? Linux questions, not necessarily LMCE, I know, but I haven't had to deal with them before, and others doing Single -> Double NIC are bound to later. "Solved" in this instance, but unresolved in general.

Evidently the bug (Mantis#3469) that corrupts the dhcpd.conf file is active when rewriting the eth0:0 config I changed in the Adminsite (that got me into this mess). So I had to edit dhcpd.conf to specify the proper same subnet in all the ranges dhcpd is managing. "Solved" in this instance but unresolved until that bug is fixed.

The real problem is that hosts on the inside LAN segment are not properly routing to hosts on the outside LAN segment. A host on the inside gets its net config from LMCE DHCP. I can ping from the inside host to the cablemodem's IP# that's connected to the outside LAN segment. But I cannot connect to the cablemodem's httpd at that same IP#. However, I can ssh from the inside host to the LMCE, and from that LMCE commandline I can connect to the cablemodem's httpd. I can reach the LMCE Adminsite from the internal host to the LMCE internal IP#.

Until inside hosts can route to hosts on the outside, this net topology won't work. But that seems to be the only problem that I can't currently work around.

[tkmedia]

I believe the eth numbering is based on the way the hardware is scanned  pci slots first.
In the web admin  you can "swap interfaces" if you need to. 

[Matthew]

I believe the eth numbering is based on the way the hardware is scanned  pci slots first.
In the web admin  you can "swap interfaces" if you need to. 

Actually, the Adminsite is pretty confused about the networking. It claims that there are 2 interfaces, that external interface is eth0 and the internal is eth0:0. But there are 3 interfaces in ifconfig: eth1 is the external, eth0 is the internal, and eth0:0 is a virtual interface mapping an IP# to the eth0 interface:

(My cablemodem is 192.168.100.1 , running the DHCPd that assigned 192.168.100.100 to the LMCE:eth1 .

Code: [Select]

# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:0B:CD:01:F2:AC 
          inet addr:192.168.101.250  Bcast:192.168.101.255  Mask:255.255.255.0
          inet6 addr: fe80::20b:cdff:fe01:a24c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6672 errors:0 dropped:0 overruns:0 frame:0
          TX packets:321955 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:547313 (534.4 KiB)  TX bytes:13986042 (13.3 MiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0B:CD:01:F2:AC 
          inet addr:192.168.101.10  Bcast:192.168.101.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth1      Link encap:Ethernet  HWaddr 00:01:02:71:FF:A8 
          inet addr:192.168.100.100  Bcast:192.168.100.255  Mask:255.255.255.0
          inet6 addr: fe80::201:2ff:fe71:af28/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:47991 errors:0 dropped:0 overruns:0 frame:0
          TX packets:63044 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:51129892 (48.7 MiB)  TX bytes:4866261 (4.6 MiB)
          Interrupt:21 Base address:0xe800

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:479826 errors:0 dropped:0 overruns:0 frame:0
          TX packets:479826 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:92414382 (88.1 MiB)  TX bytes:92414382 (88.1 MiB)
conflicts with

Code: [Select]

EXTERNAL_IFACE  eth0
EXTERNAL_MAC 00:0B:CD:01:F2:AC
EXTERNAL_IP 192.168.101.250
EXTERNAL_NETMASK 255.255.255.0
EXTERNAL_DHCP 0
INTERNAL_IFACE eth0:0
INTERNAL_MAC 00:0B:CD:01:F2:AC
INTERNAL_IP 192.168.101.10
INTERNAL_NETMASK 255.255.255.0
GATEWAY 192.168.100.1
DNS1 24.29.103.10
DNS2 24.29.103.11
Domain     Computer name 
DHCP server on Core:
[x] Enable DHCP server
  Range of IP addresses for Pluto devices: 192.168.101.2 - 192.168.101.128
  [x] Provide IP addresses for anonymous devices not in Pluto's database.
          Range of IP addresses for non-Pluto devices: 192.168.101.129 - 192.168.101.254
Number of network adapters: 2

Your core has the following network adapters:
1. External network card eth0
[x] Obtain an IP address from DHCP
Use a static IP address
  Core's IP address: ...
  Subnet mask: ...
  Gateway: ...
  Nameserver (DNS) #1: ...
  Nameserver (DNS) #2: ...
2. Internal network card eth0:0
  IP address: 192.168.101.10
  Subnet mask: 255.255.255.0
[ ] OfflineModeAnd I don't see where I can swap interfaces.

Also, do you know why I can't route from an inside host to an outside host's webserver (I can route to the LMCE webserver), but I can ping from the inside host to the outside host's IP#? FWIW, I can ssh into LMCE on either interface from hosts outside and inside.

[Hagen]

in the "device" tab select the core, at the bottom there is a listing of eths and their IP (or if they are set to receive DHCP). Delete it from the list (the eth0:0) or change it around in sequence. That is in my experience the one place that rules them all when it comes to changing setup of nics. just remember to use the same names as you would see on the KDE desktop and you should be golden. the sequence does matter, but I can't remember wich i wich now. (I do believe though that I have mentioned this to you earlier?)

[tkmedia]

First rem out any settings for eth0:0 ( you may have missed that in the single to double nic instructions).


After you fix eth0:0 and want to swap interfaces select "swap interface" from the Advanced - Network - Network settings on the top of the screen in Web -Admin

[Matthew]

First rem out any settings for eth0:0 ( you may have missed that in the single to double nic instructions).


After you fix eth0:0 and want to swap interfaces select "swap interface" from the Advanced - Network - Network settings on the top of the screen in Web -Admin

Actually, the order of the instructions was wrong. Editing the "Web Admin: Advanced menu > Network > Network Settings" fields before editing /etc/network/interfaces screwed things up. I revised the wiki article to the order I just tested to work. The "Swap Interfaces" appeared correctly in time to use it if I wanted, but the interfaces file specified properly gave me the eth0/external and eth1/internal I'd expect. And I had to ensure /etc/default/dhcp3-server pointed at eth1.

Now everything seems to be working. Now to try the 7970 netboot on the internal network, for more adventures . Thanks for helping out.