LinuxMCE Forums

General => Installation issues => Topic started by: Marper on December 25, 2007, 10:33:51 am

Title: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 25, 2007, 10:33:51 am
System Stats:

CORE:

P4 3.2 Intel
(2) 500 Gb HDD
4 Gb RAM
DVD Drive
(2) Gb realtek ethernet adapters


MD -

P4 - 3.2 Intel
320 Gb HDD
Onboard YUKON Gb Network interface
DVD Drive
SBLive PCI card - Not able to get sound
MCE Remote with USB IRT
MS USB Keyboard and mouse


Gigabyte switch for internal network

Problems
a.  MD Boot time is approximately 20 Mins (Not sure why)
b.  Playing DVD's goes to black screen for about 4 mins and then plays
c.  Getting to KDE - takes at lease 4 minutes

What do I change ?  Where can I start ?  If everything runs this slow it won't be worth the effort I've put into setting this up.

I've purchased pretty much all of the recommeded hardware or at least where I've seen on the forums compatible H/W.

Help if possible
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: totallymaxed on December 25, 2007, 12:00:04 pm
System Stats:

CORE:

P4 3.2 Intel
(2) 500 Gb HDD
4 Gb RAM
DVD Drive
(2) Gb realtek ethernet adapters


MD -

P4 - 3.2 Intel
320 Gb HDD
Onboard YUKON Gb Network interface
DVD Drive
SBLive PCI card - Not able to get sound
MCE Remote with USB IRT
MS USB Keyboard and mouse


Gigabyte switch for internal network

Problems
a.  MD Boot time is approximately 20 Mins (Not sure why)
b.  Playing DVD's goes to black screen for about 4 mins and then plays
c.  Getting to KDE - takes at lease 4 minutes

What do I change ?  Where can I start ?  If everything runs this slow it won't be worth the effort I've put into setting this up.

I've purchased pretty much all of the recommeded hardware or at least where I've seen on the forums compatible H/W.

Help if possible

Hmmm... well I can 100% confirm that what you are experiencing is not 'normal' at all. Typical boot up times for an MD even over 100mbit on our systems 90secs approx to the Orbiter, MD playing ripped DVD from Core is essentially instantaneous and going to the KDE desktop takes approx 10secs the first time... and then again is instantaneous subsequently.

Based on the specs of your Core & MD I would say that they are more than upto the job so I would say the likely problem lays in your network... packet loss due to electrical noise maybe or bad physical connection.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 25, 2007, 06:48:48 pm
I've checked the network cables and all that and have isolated the Core and the MD and still have the same issue.  I'm all new to this, is there a log I can look at to see if there is some problem ?  Also, it appears that the  MD rebuilds everytime it boots.  It seems that it's either creating a new disk image each time or does not have all of the files necessary to run.  I only see one fail message screen and that is with nfsmount, but the system continues to boot.

Thanks for any help.

Another note. I've reinstalled a number of times both core and MD.  However when setting up the MD I've had to manually add it and then set up the MAC address prior to it being recognized.  only once have I been able to boot a MD without having to first set it up under Add Device.  Yes, I've always push the button to setup diskless boot.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: 1audio on December 25, 2007, 07:15:11 pm
You can open a command line terminal on the core w/ cnt-alt-F2. user linuxmce password linuxmce. First run "top" which will tell you what the load on the system is. If its near 100% CPU look to see which programs are using the resources in the list.

The logs are available under var/log and var/log/pluto. There are a lot of logs. It may be best to start with the var/log/pluto/ logs The pluto and the dcerouter logs may give clues. Use the command tail (name of log) to see the end of the logs. They can be enormous.

Is this a clean install from the DVD's? Is there anything non-standard (like running on a virtual machine etc.)? Can you watch DVD's from the core OK? Are the network interfaces identified correctly and configured correctly? Run ifconfig in the terminal and look for the settings of all the interfaces.

Open the webadmin either from the core (computing-pluto admin) or from a pc in the network (192.168.80.1 in a browser) and look at the media director listings and the orbiter listings. Make sure they are correct. (No odd devices that don't really exist.)

I hope this helps
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 25, 2007, 10:14:25 pm
1audio - Great feedback and thanks a million to you all.  I will try this and look to see what gives.  I've reloaded from the DVD, clean install and yes I was always able to play DVD's from the Core.

I hate to keep asking questions, but since downloaded 3 months ago, I've really been intrigued with the Linux Distro and have changed all of my WHENDOZ systems to Linux (various flavors).  This is really a hot system and I am going to try and connect every module it has.

Thanks and Merry Christmas - a shame to be doing this on this day, but hey, my present to me.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: 1audio on December 25, 2007, 10:46:57 pm
I would suggest setting up the core/hybrid and making sure its working fine. Then add an MD but don't use the Web interface. Just set the MD's bios for PXE boot and start it. If you have an existing instance of the second MD delete it. Boot the MD and let it set itself up. On the core you will see a screen asking which room its in. Follow the on screen directions. That seems to work the best. It may take a while to configure but it should work. Once it reboots it will come up in the video wizard. You should not need to interact before then.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 25, 2007, 11:29:26 pm
Thanks, I'll do that.  I do have a clean Core.  So what you appear to be saying is that I should "NOT" need to manually add the MD.  If I click on the Setup Diskless" button that will do everything I need for the MD ?  I've done that a number of times and it has not worked but once.  I've installed this system at least 20 times so far, but loving all the learning.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Greg on December 25, 2007, 11:53:42 pm
Thanks, I'll do that.  I do have a clean Core.  So what you appear to be saying is that I should "NOT" need to manually add the MD.  If I click on the Setup Diskless" button that will do everything I need for the MD ?  I've done that a number of times and it has not worked but once.  I've installed this system at least 20 times so far, but loving all the learning.
No.  Don't do anything on the core.  Simply set the MD to PXE boot, making sure it is connected to the internal network.  The core will detect the new MD and everything should work automatically.  Note that the initial MD boot takes a LONG time, and it may even reboot automatically before it is done.  Just let it do it's own thing.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 26, 2007, 12:03:33 am
What a great group of guys you are.  So, I've started the MD and it is configuring itself.  Do I need to do click on "Setup Diskless" for each MD ? 

OK, I'll let this configure itself and go from there.  NO ERRORS SO FAR

CROSS MY FINGERS - WOOOOOOOHOOOOOOOO
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 26, 2007, 12:20:50 am
With regards to sound.  I have another post where I am not able to get sound out of my MD.  I have a SBLive Card and have tried to use Alsa to set the Optical (coax/dig) but no good results.  Any sugesstions ?
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: rrambo on December 28, 2007, 04:43:05 pm
System Stats:

CORE:

P4 3.2 Intel
(2) 500 Gb HDD
4 Gb RAM
DVD Drive
(2) Gb realtek ethernet adapters


MD -

P4 - 3.2 Intel
320 Gb HDD
Onboard YUKON Gb Network interface
DVD Drive
SBLive PCI card - Not able to get sound
MCE Remote with USB IRT
MS USB Keyboard and mouse


Gigabyte switch for internal network

Problems
a.  MD Boot time is approximately 20 Mins (Not sure why)
b.  Playing DVD's goes to black screen for about 4 mins and then plays
c.  Getting to KDE - takes at lease 4 minutes

What do I change ?  Where can I start ?  If everything runs this slow it won't be worth the effort I've put into setting this up.

I've purchased pretty much all of the recommeded hardware or at least where I've seen on the forums compatible H/W.

Help if possible

Hmmm... well I can 100% confirm that what you are experiencing is not 'normal' at all. Typical boot up times for an MD even over 100mbit on our systems 90secs approx to the Orbiter, MD playing ripped DVD from Core is essentially instantaneous and going to the KDE desktop takes approx 10secs the first time... and then again is instantaneous subsequently.

Based on the specs of your Core & MD I would say that they are more than upto the job so I would say the likely problem lays in your network... packet loss due to electrical noise maybe or bad physical connection.

Whoa....   you're kidding me... 90 secs to boot a MD to the Orbiter??  I don't think he's having a network problem... I know I don't have a network problem.. everything else on my network runs fine...  and once a MD is up, playing  movies, or anything else works fine...  I think he (and myself) must have a configuration problem...  my MD's have Athlon 3000's and 784 MB Ram...  it takes 9 minutes to boot to the Orbiter and 4 minutes to bring up the kde desktop the first time after booting...  we must be doing something wrong.....  can't be a network issue...
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: totallymaxed on December 28, 2007, 07:10:39 pm
System Stats:

CORE:

P4 3.2 Intel
(2) 500 Gb HDD
4 Gb RAM
DVD Drive
(2) Gb realtek ethernet adapters


MD -

P4 - 3.2 Intel
320 Gb HDD
Onboard YUKON Gb Network interface
DVD Drive
SBLive PCI card - Not able to get sound
MCE Remote with USB IRT
MS USB Keyboard and mouse


Gigabyte switch for internal network

Problems
a.  MD Boot time is approximately 20 Mins (Not sure why)
b.  Playing DVD's goes to black screen for about 4 mins and then plays
c.  Getting to KDE - takes at lease 4 minutes

What do I change ?  Where can I start ?  If everything runs this slow it won't be worth the effort I've put into setting this up.

I've purchased pretty much all of the recommeded hardware or at least where I've seen on the forums compatible H/W.

Help if possible

Hmmm... well I can 100% confirm that what you are experiencing is not 'normal' at all. Typical boot up times for an MD even over 100mbit on our systems 90secs approx to the Orbiter, MD playing ripped DVD from Core is essentially instantaneous and going to the KDE desktop takes approx 10secs the first time... and then again is instantaneous subsequently.

Based on the specs of your Core & MD I would say that they are more than upto the job so I would say the likely problem lays in your network... packet loss due to electrical noise maybe or bad physical connection.

Whoa....   you're kidding me... 90 secs to boot a MD to the Orbiter??  I don't think he's having a network problem... I know I don't have a network problem.. everything else on my network runs fine...  and once a MD is up, playing  movies, or anything else works fine...  I think he (and myself) must have a configuration problem...  my MD's have Athlon 3000's and 784 MB Ram...  it takes 9 minutes to boot to the Orbiter and 4 minutes to bring up the kde desktop the first time after booting...  we must be doing something wrong.....  can't be a network issue...

Hmmm... well we can that kind of boot time reliably on the Cores & MD's we build. We even get that kind of performance on Via based MD's so its not a processor bound measurement at the MD.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: rrambo on December 28, 2007, 09:34:10 pm
is there a certain way to setup these MD's??  Other than run the 'setup diskless md's' from the web admin and then boot them from the network to set them up?  The initial setup of a new MD takes FOREVER..  then they are listed as Generic PC as MD.... then subsequent boots once they are setup always takes about 9 minutes or so... the one VIA based machine I have takes even longer to boot then the AMD machines...  I haven't really thought about it much as I was under the impression that this was normal for a diskless MD...  but after reading that you're getting 90 second boot times on the VIA based MD's I've got to do something!
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Greg on December 28, 2007, 09:47:51 pm
I'm pretty sure the optimal way to get an MD up and running is to simply boot the MD without doing anything on the core, not even the "setup diskless md's" in the web admin.  As you said the intial setup for a new MD takes FOREVER, and I'm pretty sure there's at least one automatic reboot in the process.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: colinjones on December 28, 2007, 11:39:18 pm
What you describe sounds VERY much like a networking issue of some kind, as totallymaxed suggested - that performance is nowhere near normal even for 100Mb.

1. auto speed or duplexing issue either on the NICs or switch ports - although this is usually less of an issue now with Gb LANs. Is your switch manageable - meaning you can connect to it and configure it, or is it black-box?

2. try getting a cross-over cable (usually red) and connect the MD directly to the core, thus eliminating the switch and normal cables, to see if that makes any difference. this is a good first test to try.

3. packet loss somewhere between core and MD, due to cables, connections, switch or NICs. if over a few percent, this will cause TCP to have to retransmit and backoff - this will cause a dramatic reduction in performance. can be caused easily by either of the 2 points above.

4. incompatibility between the NICs and the switch hardware - test 2 will also highlight this immediately.

5. some issue with the NICs themselves, or more likely the drivers - check the versions, and the model of Realtek you are using with the ubunutu hardware wiki. EDIT2: another thought, I read somewhere in the last couple of days that the Realtek 8169(?) Gb NIC was small and cheap, but that person had a very hard time getting Gb speeds out of it with Ubuntu... more like 100Mb, that being said, you don't even seem to be getting 100Mb speeds!

6. bit difficult with a diskless MD, but an ideal troubleshooting test is to navigate to a remote share on the core from the MD, or vice versa, and copy a large (50MB+) file over and time it. work out the rate from this. the linux dudes will have to help you with where to copy it too - perhaps a ramdisk?

7.failing the above test, you could use a PC (with disk, say Windows) navigate to the core and test, then the MD and test again - this doesn't give you end to end, but it could be indicative...

8. use the performance monitoring tool on the desktop, and add the bytes in/out per second counters and watch the traffic profile whilst copying a file, and booting from LAN. Whilst copying the file, you should see it ramp up fairly quickly then stay reasonably stable (assuming no other traffic on the network) until the end of the file. not sure what to expect for the MD boot as this uses TFTP which is a UDP protocol and so doesn't use the same congestion control mechanisms - at a guess, it would ramp up much quicker, but then be fairly erratic - guesstimate an average throughput

oh, finally, I forgot to say, auto duplexing and speed issues in your configuration will often only manifest a throughput performance issue in one direction, or heavily tilted in one direction. So when you perform the file copies, etc, test both directions, but especially from core to MD, obviously!

EDIT: one more thing, there will be linux tools for capturing information like TCP resets, retransmits, errors, etc, which would point you to the problem as all of these dramatically impact network performance, but I do not know linux at all, so again the linux dudes may be able to give you pointers there.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: hari on December 29, 2007, 01:52:48 am
you can set/verify speed and duplex with ethtool and/or mii-tool. You can watch the error counters with ifconfig.

best regards,
Hari
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 29, 2007, 09:22:24 pm
Thanks to everyone for the info and trying to help.  Here is my current status.

1.  Reinstalled Core software (as usuall setup was a breeze)
2.  Re-configured MD

Used all standard settings without changing anything -

Added VOIP with Teliax
Added Rooms and Names

System up and running.

Reboot time for MD = 10 Mins

    Allowed software updates to take place - restarted all systems -

    Core boot time = 3 mins
    MD boot time = 9/10 mins
    Copied files from Core to MD - 100Mb file = 2/3 mins - 1Gb NIC
    Copied files from Core and MD to other connected systems - 3 or so mins - 1Gb NICs
    All logs are clean - no errors

When the MD boots - it takes a long time to get past the OBD Shell - about 4 mins for that to pass

Where do I go from here


Lost - Don't know where to go or what to do
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: colinjones on December 30, 2007, 12:57:40 am
Marper - sorry I don't have a working system at the moment to work this through, and as I say, networking on Linux is not my comfort zone!

However, I can say that you definitely appear to have a significant network problem from these stats... you are getting less than 5Mbit/s with one large file, so lots of small files don't stand a chance. Given this was a simple file copy, the LMCE doesn't seem to be implicated in any way. It could still be the speed of your systems, but very unlikely as the data rate is SO low, and as totallymaxed pointed out, these processes (booting) are not particularly CPU bound.

Another point is that your file transfer is TCP, but the boot process is UDP, so it is unlikely to be TCP congestion control issues, which eliminates the network stack, leaving us with the NIC drivers and the network itself.

The next "high value" test you should do is bypass the switching and network cables with the cross-over cable direct from NIC-to-NIC (core to MD). They are only a few bucks from your computer store. Do that and try the boot again.... we can go from there...
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: jeff_rigby on December 30, 2007, 01:22:59 am
I can confirm that after the initial install MD's boot in about 2 min on a Gigabit lan.  Not much longer on a 100 megabit lan either but I have noticed some lag for certain screens. (lag= 2-3 seconds)
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 30, 2007, 11:14:31 pm
I'm going to purchase new NIC's for both the Core and MD and I'll re-post my results.

Thanks for all of the support, comments and help.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: colinjones on December 31, 2007, 12:01:33 am
If that's what you want to do, but that wasn't the next suggestion I made. If there is a problem with the switch/speed/duplex/NIC drivers then you will have wasted your money on 2 more NICs, plus you will not be any closer to identifying whether it is the driver/configuration or the external network - so you will need to get the cross over cable anyway. Will leave it to you now...
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: 1audio on December 31, 2007, 12:26:55 am
NIC's for the MD's are hard to find. They must have internal support (BIOS extensions) for PXE boot. Only a few do. If your hardware is known to work for other people then its either defective hardware (which does happen) or some configuration fault.

If you can run ifconfig on the core and the MD and post the results so we can see how it thinks its configured that would help.

Also make sure you have enough memory in the MD. 512 MB is what you need. And set the video for no more that 128 if you are using integrated graphics. The last time I ran into this type of problem is was a memory issue. Especially on MD's since they swap to the core and use the same pipe as for the content.

And make sure the BIOS is current.
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 31, 2007, 06:24:02 am
Here is the ifconfig from Core:

dcerouter_52637:/home/linuxmce# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:80:AD:80:20:BE
          inet addr:192.168.0.2  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::280:adff:fe80:20be/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:54955 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14841 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:23931626 (22.8 MiB)  TX bytes:1632474 (1.5 MiB)
          Interrupt:16 Base address:0xd400

eth1      Link encap:Ethernet  HWaddr 00:80:AD:03:5D:02
          inet addr:192.168.80.1  Bcast:192.168.80.255  Mask:255.255.255.0
          inet6 addr: fe80::280:adff:fe03:5d02/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:581892 errors:0 dropped:0 overruns:0 frame:0
          TX packets:812575 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:172095480 (164.1 MiB)  TX bytes:809557645 (772.0 MiB)
          Interrupt:17 Base address:0xd000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:22569 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22569 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4476468 (4.2 MiB)  TX bytes:4476468 (4.2 MiB)
Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 31, 2007, 06:26:47 am
Sorry for the multiple posts, just learning my way around Linux.


This is MD ifconfig:

root@moon29:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:D8:C4:01:02
          inet addr:192.168.80.2  Bcast:192.168.80.255  Mask:255.255.255.0
          inet6 addr: fe80::211:d8ff:fec4:102/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:389668 errors:0 dropped:0 overruns:0 frame:0
          TX packets:281477 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:399529691 (381.0 MiB)  TX bytes:78302809 (74.6 MiB)
          Interrupt:20

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:304 errors:0 dropped:0 overruns:0 frame:0
          TX packets:304 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24576 (24.0 KiB)  TX bytes:24576 (24.0 KiB)

Title: Re: MD Runs extremely slow over Gb etho - Why
Post by: colinjones on December 31, 2007, 08:02:13 am
There's nothing obviously wrong in those stats. The core does seem to be transmitting a lot more data than the MD is receiving, and receiving a fair amount as well, but that in itself doesn't highlight much without knowing more about what else is on the LAN.

I will try asking one more time before giving up - have you got the cross over cable? This is a 5min test costing a couple of bucks and will immediately tell you if the issue is on your network or in the PCs.
Title: SOLVED - Re: MD Runs extremely slow over Gb etho - Why
Post by: Marper on December 31, 2007, 08:18:56 am
Thanks to you all for helping with this.  1audion, when I returned to the BIOS to check the settings, the Graphics Aperture Size was set to 248.  I changed it to 64 Mb and WOW, the system boots under 2 mins.  Now, that's more like it.

Once again, thanks to you all.

HAPPY NEW YEAR