LinuxMCE Forums
General => Users => Topic started by: tux-box1 on January 18, 2012, 12:47:30 am
-
Hello all.
I was hoping for some help on this, I'm currently experiencing poor network performance on my hybrid Kb a sec transfers.
I'm using cat6 cable and a gigabit switch, Gigabit nic's on both systems.
The lay out of the network..
Cable modem feeds-> router feeds-> Gigabit switch
Gigabit switch feeds ->
Lmce
PC
I'm getting Kb a sec here not near what I would expect getting.
The drives I'm copying to are 5400 rpm and 130.28Mb/sec using hdparm on average.
I have achieved 10Mib a sec transfers on that system in the past before Linuxmce.
Suggestions??
-
Why don't you try this: cablemodem to lmce to gigabitswitch to pcs you don't need a router this way, thus eliminating a potential troublemaker.
-
What's the layout of your internal network - do you have a Gbps switch there too?
You should be experiencing very good performance both from PC to Hybrid and from PC to Internet; lmce is not adding any overhead more than any normal Linux server. Did you check the logs for any misbehaviour on your NICs? Are they set up to run at Gigabit speed?
Try to isolate the problem - first investigate throughput speed from a PC connected to the outer network to Internet. Then from The Hybrid to Internet and last from a PC on the internal network to Internet. The results should be close to each other, if not you will see where the problem is introduced.
A second investigation would be to connect a PC to the outer network, disable the firewall in the Hybrid and test throughput towards the Hybrid; then connect the same PC to the internal network and redo the test. Any differences here?
/Joakim
-
First of all, try to understand (draw) your connection. What's your WAN interface, what's your LAN interface (fe in my case is LAN: eth0, WAN: eth1). This way you know where you're looking. A picture is saying a lot more then some words. ;)
I'll put some basic test here that can point you to the right place where to look...
Connect
Can you connect with ssh to your core? I'll assume you're using a linux client.
ssh username@192.168.80.1
Network interfaces
With ifconfig, you'll see a lot more about your network cards.
ifconfig
-> do you see any errors on the interfaces (collisions)? In fact, you should only see RX/TX packets here...
-> What the subnetmask of you interfaces (the same as for your routing table)?
fe
eth0 Link encap:Ethernet HWaddr 00:21:85:5d:d6:05
inet addr:192.168.80.1 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::221:85ff:fe5d:d605/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4338750 errors:0 dropped:0 overruns:0 frame:0
TX packets:2156345 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1817464938 (1.8 GB) TX bytes:161459655 (161.4 MB)
Interrupt:219 Base address:0xa000
eth1 Link encap:Ethernet HWaddr 00:04:76:97:7b:98
inet addr:192.168.0.184 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::204:76ff:fe97:7b98/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:49496 errors:0 dropped:0 overruns:0 frame:0
TX packets:32634 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:26781428 (26.7 MB) TX bytes:7483598 (7.4 MB)
Interrupt:21 Base address:0xac00
For more information, you can always do following:
sudo ethtool -S eth0
This should give you more information:
NIC statistics:
tx_packets: 2190314
rx_packets: 4367767
tx_errors: 0
rx_errors: 0
rx_missed: 0
align_errors: 1166
tx_single_collisions: 0
tx_multi_collisions: 0
unicast: 4350464
broadcast: 11762
multicast: 5541
tx_aborted: 0
tx_underrun: 0
Routing
netstat -rn
-> should give you a clear routing table
fe
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 <<<Your WAN network>>>
192.168.80.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 <<<Your LAN network>>>
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth1 <<<All the rest to your internetprovider>>>
Traceroute
With a traceroute you'll see the connection path. This is useful to see where your connections goes, and if any devices are taking a long time to respond.
This is best done from you client machine on (i don't have direct access to a client now, so i can't show you my result, sorry).
traceroute linuxmce.org
fe (this is done from another network, so it won't be the same for you). The most important thing in here are the first lines, with your private ip's, and the first public ip (your internetrouter):
traceroute to linuxmce.org (193.200.113.133), 30 hops max, 60 byte packets
1 192.168.80.1 (192.168.80.1) 3.575 ms 3.778 ms 4.355 ms <<< Your internal gateway >>>
2 192.168.0.1 (172.31.31.1) 7.338 ms 7.716 ms 7.932 ms <<< Your internet gateway >>>
3 3.123-123-109.adsl-dyn.isp.belgacom.be (109.123.123.3) 14.416 ms 16.522 ms 19.280 ms <<< Your public ip >>>
4 160.241-183-91.adsl-static.isp.belgacom.be (91.183.241.160) 21.939 ms 24.110 ms 26.778 ms
5 ae-15-1000.ibrstr1.isp.belgacom.be (91.183.246.107) 52.974 ms 31.339 ms 53.332 ms
6 bru-11-r6-t9-4.car.belbone.be (80.84.20.18) 36.676 ms bru-11-r6-t8-4.car.belbone.be (80.84.20.78) 13.732 ms bru-11-r6-t7-4.car.belbone.be (80.84.21.34) 14.288 ms
7 94.102.162.50 (94.102.162.50) 17.156 ms 19.748 ms 22.358 ms
8 neotelecoms.bnix.net (194.53.172.79) 31.315 ms 33.714 ms 36.135 ms
9 xe0-0-0.tcr1.gs.par.as8218.eu (83.167.55.20) 58.078 ms 58.461 ms 58.842 ms
10 83.167.56.155 (83.167.56.155) 62.023 ms 64.715 ms 67.321 ms
11 xe0-1-1.r06.uni.vie.at.nextlayer.net (212.69.191.150) 67.712 ms 69.814 ms 35.678 ms
12 xe4-1-1.r05.rdh.vie.at.nextlayer.net (92.60.3.144) 38.981 ms xe2-1-0.r05.rdh.vie.at.nextlayer.net (92.60.2.144) 39.179 ms xe4-1-1.r05.rdh.vie.at.nextlayer.net (92.60.3.144) 40.699 ms
13 vl21.r02.rdh.vie.at.nextlayer.net (81.16.147.192) 43.634 ms 45.582 ms 50.315 ms
14 abaton.rdh.vie.at.nextlayer.net (81.16.148.34) 51.541 ms 37.767 ms 39.880 ms
15 193.200.112.37 (193.200.112.37) 41.829 ms 43.772 ms 45.978 ms
16 94.247.144.120 (94.247.144.120) 48.424 ms 50.878 ms 53.792 ms
17 193.200.113.133 (193.200.113.133) 55.457 ms 35.938 ms 38.298 ms
Send us the results of this, from there on we can maybe start troubleshoot a bit more ...
Of course, it's best that you do all this tests from all the machines (or at least all the machines that are in the loop). This is the reason why i wrote that's always best to have a small drawing beside... This way, you can isolate the problem to a certain point/connection...
-
The network and it's components aren't always the root cause of poor network performance. Cat6 and gig nics only mean you have the potential for high speed data transfer but other factors can and usually are the bottle neck. To be completely thorough we'll also need to know:
The system specs of your core?
Are you using IDE drives for some reason?
Are you using raid and if so what type? Is your controller on board or a add-on card?
How much Ram do you have and what is your bus speeds? What's the processor speed?
Are you having speed issues from multiple clients on the network or just one?
Are they connecting over wireless?
What's your wireless AP speed? What are the client speeds? If you answer wireless N to the speed, are there any devices on your network that is not wireless N? How many?
What encryption are you using?
What are the client specs? Quick over view.
How many client are accessing the shares at any given time?
How long is the cabling between each of your network devices? Are you using pre-made cables or did you make them yourself? How old are the cables?
Most importantly:
What is the current transfer speed?
How are you getting these number?
What's the transfer size?
Are you testing with a large file or a bunch of small files that equal a larger number?
Any single one of these areas I asked about can diminish network performance. The last 4 goes to the simple fact that sometimes people are not realistic about the performance they're going to get. I'm not trying to insult you but it is what it is, just have to make sure.
-
Are you testing with a large file or a bunch of small files that equal a larger number?
4000 x 1MB files take longer to transfer then 1 x 4GB file .........
-
4000 x 1MB files take longer to transfer then 1 x 4GB file .........
You hit the nail on the head my friend. It's funny how many people don't know that.
We havn't even dove into the configurations of his router, switch and nics. Jombo frames, auto speed vs locking in a speed, QOS, etc. All the fun stuff!
-
Thank you all for your posts. I'm not in front of my network at the moment so the testing and tweeking will have to wait.
I thought that I had mention in my original post that I have had 10Mb second transfers on the exact setup in the past(3 days ago) only difference between those speeds and now is the install of linuxmce. So I believe its safe to say its not a hardware problem.
-
Thank you all for your posts. I'm not in front of my network at the moment so the testing and tweeking will have to wait.
I thought that I had mention in my original post that I have had 10Mb second transfers on the exact setup in the past(3 days ago) only difference between those speeds and now is the install of linuxmce. So I believe its safe to say its not a hardware problem.
I noticed the comment about 10mbs transfer speed but I ignored it.
Here's the deal 10 mbs is far from acceptable from what you have described thus far. For example, my old network setup was all gigabit, but I was using cat5e cabling and a dinky little $55 8 port unmanaged dlink switch. I would see 200-250mbs transfer speed from client to client, from Nas to client it was around 300-350mbs. It really didn't matter to much what I was coping but how I copied it made the world of difference. Now I have a $400 48 port power connect and I upgraded the cabling to cat6 (which doesn't actually offer much improve over cat5e but why not use it anyway?), I am maxing out my gigabit connections from client to client and nas to client. Again how I copy the files make the difference but either way I've never notice less than half the pipe filled. Except for rsync, which I'm sure is by design, but even rsync is using 200mbs to backup my nas. I'm not even using any advance features of the managed switch like lagg groups, jumbo frames, class of service, etc. So lets say you're using a cheaper consumer grade switch you still wasn't where you should have been before linuxmce. I'm not saying it's absolutely a hardware issue but I don't think you've provided sufficient proof to rule it out.
My questions and every other question and test suggested before mine needs to addressed before any useful, efficient troubleshooting can be done.
-
After re-reading the OP, is it possible you're talking about internet speeds? That kind of makes sense to me if you're will to accept 10mbs when you're using gigabit gear.
-
OK... I don't know what was going on but after a reinstall form scratch of LinuxMCE I'm getting much better speeds. I ran 5 quick tests of two types. Meaning I ran each test type 5 times.
First type, a folder of HD pictures ranging form 44 KB to 605KB ~ 500 pictures in all average speed 10Mb a sec.
Second type, A dvd ISO. ~4.15GB average 75MB a sec.
Again no hardware changes, only software changes on one system and that's Linuxmce.
I'll run some IPERF test's this weekend when I have more time.
Until then this is much better and workable.
-
Start first with some local tests. Check your collisions and so. This can help you to find out if you've got a cable issue, poor networkcard...
See my original post above...
Always work the layers up!!!
It's a very good and proven troubleshooting path...
(http://www.oniria.be/pri/images/stories/IT/9Layers.png)
-
All tests ran on fresh install, after I was getting the "better" speeds. The old install was getting KB second transfers on all files. The new install, well see my last post. I have tested connected to the eth0 port but got slower speeds on the eth1 port. So one thing at a time.
First the results of the test's you wanted. I'll do some hardware testing this weekend. I have a cable tester I can use, as for testing the nics them self's I'll use IPERF..
First of all, try to understand (draw) your connection. What's your WAN interface, what's your LAN interface (fe in my case is LAN: eth0, WAN: eth1). This way you know where you're looking. A picture is saying a lot more then some words. ;)
Been their done that, networking 101 I have a quick and dirty google doc if you can send me an email. I'm not a complete noob :) just no expert. I'll put some basic test here that can point you to the right place where to look...
Connect
Can you connect with ssh to your core? I'll assume you're using a linux client.
ssh username@192.168.80.1
Network interfaces
With ifconfig, you'll see a lot more about your network cards.
ifconfig
-> do you see any errors on the interfaces (collisions)? In fact, you should only see RX/TX packets here...
-> What the subnetmask of you interfaces (the same as for your routing table)?
$ ifconfig
eth0 Link encap:Ethernet HWaddr 20:cf:30:c0:ff:46
inet addr:192.168.3.80 Bcast:192.168.3.255 Mask:255.255.255.0
inet6 addr: fe80::22cf:30ff:fec0:ff46/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:643754728 errors:0 dropped:3991875718 overruns:0 frame:0
TX packets:156841564 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1054057917 (1.0 GB) TX bytes:1290091923 (1.2 GB)
Interrupt:221
I noticed the dropped packets this many raises a flag
fe
For more information, you can always do following:
sudo ethtool -S eth0
This should give you more information:
$ sudo ethtool -S eth0
NIC statistics:
tx_packets: 156841598
rx_packets: 643754748
tx_errors: 0
rx_errors: 0
rx_missed: 137
align_errors: 0
tx_single_collisions: 0
tx_multi_collisions: 0
unicast: 643740548
broadcast: 4967
multicast: 14200
tx_aborted: 0
tx_underrun: 0
Not perfect but nothing big here.
Routing
netstat -rn
-> should give you a clear routing table
$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.80.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth1
0.0.0.0 192.168.3.1 0.0.0.0 UG 0 0 0 eth0
fe
Traceroute
With a traceroute you'll see the connection path. This is useful to see where your connections goes, and if any devices are taking a long time to respond.
This is best done from you client machine on (i don't have direct access to a client now, so i can't show you my result, sorry).
traceroute linuxmce.org
fe (this is done from another network, so it won't be the same for you). The most important thing in here are the first lines, with your private ip's, and the first public ip (your internetrouter):
$ traceroute linuxmce.org
traceroute to linuxmce.org (193.200.113.133), 30 hops max, 40 byte packets
1 192.168.3.1 (192.168.3.1) 0.668 ms 1.804 ms 2.200 ms
2 X.X.X.X (X.X.X.X) 8.378 ms 12.214 ms 13.201 ms
3 XX.XX.XX.XXX.com (XX.XX.XX.XX) 13.566 ms 13.753 ms 13.921 ms
4 XXX.XX.XX.XX.com (XX.XX.XX.XX) 19.750 ms 20.698 ms 23.063 ms
I marked out the IPs with XXs, why your having me test the out side network is beyond me.
Send us the results of this, from there on we can maybe start troubleshoot a bit more ...
Of course, it's best that you do all this tests from all the machines (or at least all the machines that are in the loop). This is the reason why i wrote that's always best to have a small drawing beside... This way, you can isolate the problem to a certain point/connection...
-
eth0 Link encap:Ethernet HWaddr 20:cf:30:c0:ff:46
inet addr:192.168.3.80 Bcast:192.168.3.255 Mask:255.255.255.0
inet6 addr: fe80::22cf:30ff:fec0:ff46/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:643754728 errors:0 dropped:3991875718 overruns:0 frame:0
TX packets:156841564 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1054057917 (1.0 GB) TX bytes:1290091923 (1.2 GB)
Interrupt:221
I'm a bit worried about your dropped recieved packages. What is Eth0? Seen the amount of packages, i would say the LAN side?
This is mostly due a bad driver or lack of networkmemory... Bad cables mostly give collisions...
Maybe a quick test is to change the networkcard?
Try maybe to swap your LAN / WAN side?
-
I marked out the IPs with XXs, why your having me test the out side network is beyond me.
Just for your info:
I meant to do a traceroute from you client machine (behind your linuxmce box). This can bring forward who gives you latency (your network gateway, internet gateway...). Maybe not directly useful for you GB copy problem, since I guess it's something local (nice to have the drawing, remember).
These are always basic/first test i'll do to troubleshoot network issues.
Another very useful command in this line is the command MTR. Give it a try: 'sudo mtr www.linuxmce.org'
But i would focus on the dropped packets in your case... ;)
-
I'm not using my lmce box as a router at the moment. So my eth0 is both WAN and LAN for now. Ill do a reboot and do some more tests this weekend when I have the time.
As for the actual lay out of the etwork . My cable modem feeds my wireless router that has DD-wrt on it so I can do reserved dhcp leases to an VMware esxi server easier than static IPs. This is also so when I have friends over they. Have internet without access to my media. If once supports reserved dhcp ill be glad to make some changes.
-
I'm not using my lmce box as a router at the moment. So my eth0 is both WAN and LAN for now. Ill do a reboot and do some more tests this weekend when I have the time.
As for the actual lay out of the etwork . My cable modem feeds my wireless router that has DD-wrt on it so I can do reserved dhcp leases to an VMware esxi server easier than static IPs. This is also so when I have friends over they. Have internet without access to my media. If once supports reserved dhcp ill be glad to make some changes.
I'm pretty sure the lease time in LMCE is such that the IP addresses to clients don't change. I'm very certain I read that some where... maybe Thom said it.. don't quote me on who said it.
It would be great if you had a drawing of your network. I'd like to think you can set this up a little more straight forward and efficient than it is, at least based on what it sounds like you're doing. DD-WRT supports virtual ssids and vlans so right there you have a better network design that adheres to the supported LMCE network configuration.
-
Here's a dirty picture of the network lay out.
https://docs.google.com/drawings/d/16q2MupQs-oDVCizGwLtPblzh-kNESZ4_quzM27EaPrI/edit
I've recently changed it to
https://docs.google.com/drawings/d/1gNgYkh1P3Z7oS3F4_GzH5cGhrgQlPRtHePEbno-NHNk/edit
I moved to the second image to see if eth0 was the problem due to the amount of dropped packets.
My nest test I'm removing the wireless router completely. I don't think its the problem but.for through testing.
-
Here's a dirty picture of the network lay out.
https://docs.google.com/drawings/d/16q2MupQs-oDVCizGwLtPblzh-kNESZ4_quzM27EaPrI/edit
I've recently changed it to
https://docs.google.com/drawings/d/1gNgYkh1P3Z7oS3F4_GzH5cGhrgQlPRtHePEbno-NHNk/edit
I moved to the second image to see if eth0 was the problem due to the amount of dropped packets.
My nest test I'm removing the wireless router completely. I don't think its the problem but.for through testing.
Just curious, In the first diagram did you have the external and internal nics connected to the GB switch or was the internal nic for Lmce left unused or not even installed?
Does anyone know if the firewall in linuxmce will cause the dropped packets amount reported using ifconfig to go up as it drops undefined connection attempts? I'm guessing no but i've never used the lmce firewall. Either way you need to figure out why your dropped packets count is so high. Have you tried swapping the nics on the core? I think someone suggested that earlier. Are you conducting these test from the wireless or the PC? Have you tried using different ports on the switch? You never indicated the make, model or condition of your switch but i've seen a similair situation before and it turned out to be a bad port on the switch.
-
On of the reasons I'm reluctant to give out more information than what is required. No I'm not doing this over wireless, no I'm not plugging both NICs of the lmce box into the switch. I went so far as to completely disable the ip4 and 6 firewalls. Ran watch ipconfig and the dropped packets were still cranking.
-
I ran some IPERF tests using setup of network 2
[ 4] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51904
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 363 MBytes 304 Mbits/sec
[ 5] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51922
[ 5] 0.0-10.0 sec 374 MBytes 314 Mbits/sec
[ 4] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51932
[ 4] 0.0-10.0 sec 200 MBytes 167 Mbits/sec <---- This is because I was coping a file at the time. So ignore it
[ 5] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51941
[ 5] 0.0-10.0 sec 370 MBytes 310 Mbits/sec
[ 4] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51953
[ 4] 0.0-10.0 sec 370 MBytes 310 Mbits/sec
[ 5] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51954
[ 5] 0.0-10.0 sec 374 MBytes 314 Mbits/sec
[ 4] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51958
[ 4] 0.0-10.0 sec 360 MBytes 302 Mbits/sec
[ 5] local 192.168.80.1 port 5001 connected with 192.168.80.129 port 51959
[ 5] 0.0-10.0 sec 371 MBytes 311 Mbits/sec