Ripped DVD Playback Preformance Issues- Network maybe?

[Stutch]

Hello again.  First of all, please forgive me for the long post.  I am very passionate about getting this system working so that I can help others. 
I've been troubleshooting this particular problem for 3 months now and I am running out of ideas.  When attempting to watch movies (ripped to the core system) on my diskless media devices, playback stutters periodically.  Considering that my intention is to move my entire DVD Library to this core system, the performance is absolutely critical.  Being an IT Infrastructure professional, I took a pretty hard core approach to correcting this issue.  (I like buying big toys.)

First, I thought maybe it was a disk access speed issue, so I dedicated four 250 GB drives in a hardware raid 10 configuration for the Linux MCE OS.  I then created a separate (hardware) raid 5 LUN for my media.  This resulted in no improvement on the media directors during playback; however they do seem to boot up quicker. 

Next I replaced my Linksys Gigabit switch with a Cisco Catalyst 3550 (Layer 3) Gigabit Switch and a few cat 6 certified patch cables to plug the systems directly into the switch.  I did see notable improvement after this change.  Movie play back still stutters on occasion, but it is far less severe. 

I have a core set up and running like a scalded dog, the config is below:

Motherboard: MSI Platinum AM2
RAID controller: 3Ware PCI-Express x16 SATA II Controller Card RAID
Processor: AMD Phenom 9950 2.6 GHz Quad Core
Memory: Crucial Ballistix 4gig memory
Case: Norco RPC-4020 4U
Power Supply: Antec TruePower Quattro 1000W
Video Card: MSI Video Card Nvidia Chipset
NIC: Intel Gigabit Ethernet
Hard Drives:
System: Western Digital Caviar 250GB X 4 Raid 10 configuration
Data: Western Digital Caviar 1TB X 8 Raid 5 configuration

Here are my current Media Directors:
Media Directors:
-1 MSI Media Live x64
-3 Generic PC x86(Testing)

What I think may help is increasing the MTU size for the media network.  I can do that easily enough on the switch, however, what would I need to do on the core /media directors?
Would this even help me?
Is there something else that I may be missing that could cause my video playback preformance issues?
I love you guys.

[colinjones]

It seems very unlikely that bandwidth or settings like MTU are implicated given the infrastructure you have!

Have you tried plugging a media director directly into the internal NIC with a patch cable? Presumably they don't stutter when you play them on the core?

Can you temporarily plug a HDD into one of the MDs, transfer the files there and play locally?

Worst case, you could try sniffing the traffic and making sure that there are no missed ACKs or that neither side is generating RSTs that might interrupt the TCP flow...

[niz23]

Stutch.

Hello again.  First of all, please forgive me for the long post.  I am very passionate about getting this system working so that I can help others. 
I've been troubleshooting this particular problem for 3 months now and I am running out of ideas.  When attempting to watch movies (ripped to the core system) on my diskless media devices, playback stutters periodically.  Considering that my intention is to move my entire DVD Library to this core

Shortened....

I had this behavior earlier with some of my nForce based motherboards.
When you see your video problem do the screen freeze for short while?

Look in dmesg. Run "dmesg" from a shell prompt right after you have had the video problem.
What is in the rows at the end?
Do yo see loop/iteration text?

At least one of my nForce based MBs had the irq for graphics and ethernet hardwired to the same physical interrupt line.
There is also some known problems with nForce ethernet chips.

/niz23

[Stutch]

Oops, that is not the motherboard in use on my core.  I forgot that ended up replacing it due to wanting the 4 16x PCI express slots for future raid expandability...  I am actually using the K9A2 Platinum 790FX AM2+ ATX Motherboard http://global.msi.com.tw/index.php?func=proddesc&prod_no=1332

Colinjones - As far as playing DVDs on the core, I can't get the sound to work; it has the integrated Realtek ALC883 which doesn't seem to work at all.  I wasn't too concerned about this fact though since this system is my core.  I would have to get the sound working on it to do an appropriate test on movie playback quality.  In that case, is there an easy way to get my sound working?  I will also attempt a crossover cable directly to my media device and see if that works (God help me if it does.).  I have played DVD's locally on the media device internal DVD players without issue, but I haven't tried coping the DVD to the local system.  I can see how that test would help eliminate the network as an issue.  I'll give it a shot as well.  I'm still green to the Linux world, which is a problem for me.  I'll look at dmesg to see what is going on, and post it on here as I'll have no clue what I am looking at.

Niz23- I will have to see if the whole system freezes or just the playback.  I’m thinking it is just the playback though. The internal NIC on my core is in use on the public facing network.  I have a separate gigabit NIC in place for my internal 192.168.80.0/24 network.  Would that eliminate the issue with the shared IRQ for the Nic and video card?  Also the issue is on both the Intel and AMD based media directors, so I assume your concern is with the Core being an nForce based motherboards.

[colinjones]

You probably won't even need the cross over as most NICs do auto MDI-X these days.

Are you using SPDIF on the core? If so, try alsamixer at the command line and check that the IEC958 level isn't muted, that is a common cause of no sound when using optical or coax sound

[Stutch]

I'm just using good old alalog sound at the moment.  Or at least that is what Im trying to use.  I have a simple pair of pc speakers plugged into her at the moment.

[johanr]

plugged the speakers into her? *giggle* sorry..

hehe, I like the approach  cisco 3550 at home
have you modded it as well to be little less noisier?

What is occasionally? every 20 minutes or 5 minutes?
Always same place in the movie?

HW:
Well just a thought from my side, Are you doing anything else at the same time that needs processing at the core?
Since I am not a daily linux user I can't recommend a program/app to run but how about monitoring cpu and memory for a longer period and then review when this happens if it has to do with cpu/mem

Network:
Have you checked the network traffic with for example wireshark for dropped packages, package delay or similar?
Set it up so you do port mirroring on the interface towards the core and then review it later. Even better do a merge cap of two files like interface to the core and the interface to the media directors.

I also don't think increasing the package size will help. Off course, the bigger the better(in some cases)

edit.
You are using Raid5 for the media data?
 From performance perspective I belieive Raid 0 or 1 would be better. Can this maybe be the cause?

-johan

[hari]

nice switch. I assume we are talking about the C3550-48? Can you post the config of the switch and the duplex settings of your clients? (mii-tool or ethtool)

btw, the 1500 MTU should be fine on 100mbit. If you have the 12T variant, using jumbo frames on the gig ports could be an option. But you don't need that for DVD streaming :-)
Also check the port counters for errors.

The intel is a e100 or e1000? a "cat /proc/interrupts" would be interesting, too..

best regards,
Hari

[Stutch]

Didn't you know, when you start plugging $4000 into a server, it develops a gender?  And it has to be a woman, because it drives me F"ing crazy, and yet I keep coming back for more.  

Yes a 3550 is an extremely loud switch.  But we have a (almost full) 72" server rack in the basement that houses all of this stuff so that it can be sealed off from the rest of the house. Thus noise is not an issue.  I am a Cisco guy at heart, so it wasn't difficult to justify the purchase (Ebay ~ $600).

HW: Nope, the core is just that, a filthy huge machine dedicated to a single purpose in life.  It is running LMCE and that is it.  I could use help with monitoring resources though, as I am not well versed in Linux either.  I even turned off the (Autostart Media Director) so that it is only running as a core.

Network:  This is what I am going to try tonight, I'll set up port mirroring and see what is going on.  

Raid: I chose raid 5 to maximize my storage capacity, while still allowing for redundancy.  Raid 0 is out since it offers no protection in the event of a lost drive.  I could use raid 1 or 10, but then I go from having 7TB of storage to 4TB.  I have given this a bunch of thought, and if I don't see anything using the packet sniffer, I guess that is what I will try next.

[Stutch]

Hari- I have a WS-C3550-12G It is a 12 port gigabit switch capable of both fiber and RJ45 connectivity.  I even have my eye on a few fiber nics to try on the core...

http://cgi.ebay.com/Cisco-WS-C3550-12G-Gigabit-Switch-10-x-GBIC-2-Copper-GE_W0QQitemZ260303411174QQcmdZViewItem?hash=item260303411174&_trkparms=72%3A1205%7C39%3A1%7C66%3A2%7C65%3A12%7C240%3A1318&_trksid=p3286.c0.m14

 I'll have the rest of that infor for you as soon as I get home.  I agree that DVD streaming should not require jumbo frames, but at the same time, I can't ignore the preformance gain that I saw after installing it.

Johanr- The freezing up is randam, and is less frequent with the Cisco switch in place.

[johanr]

yep, I am soon about to start drewling.. Fiber..
 wish I had the same opportunity to go the same route
Good luck troubleshooting tonight. Hope you find something.
-johan

[niz23]

Stutch.

Oops, that is not the motherboard in use on my core.  I forgot that ended up replacing it due to wanting the 4 16x PCI express slots for future raid expandability...  I am actually using the K9A2 Platinum 790FX AM2+ ATX Motherboard http://global.msi.com.tw/index.php?func=proddesc&prod_no=1332


Niz23- I will have to see if the whole system freezes or just the playback.  I’m thinking it is just the playback though. The internal NIC on my core is in use on the public facing network.  I have a separate gigabit NIC in place for my internal 192.168.80.0/24 network.  Would that eliminate the issue with the shared IRQ for the Nic and video card?  Also the issue is on both the Intel and AMD based media directors, so I assume your concern is with the Core being an nForce based motherboards.

Ok. IRQ is probably not your problem then.
Still. Check dmesg when the issue occur.

/niz23

[Stutch]

Ok, is there a quick way for me to allow access via SSH and web to access the Core on its public side?  That will give me a quick way to grab the config for you.

[Stutch]

Below is the current config of my 3550 switch.  There are a few modifications that I made that I can point out.

test3550#show config
Using 1784 out of 393216 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname test3550
!
enable secret 5 $1$pJkJ$L5DQXi1e5t.7vXaG9IstH1
enable password cisco1
!
no aaa new-model
ip subnet-zero
!
!
!
crypto pki trustpoint TP-self-signed-598334592
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-598334592
 revocation-check none
 rsakeypair TP-self-signed-598334592
!
!
crypto pki certificate chain TP-self-signed-598334592
 certificate self-signed 01 nvram:IOS-Self-Sig#3201.cer
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
interface GigabitEthernet0/1
 switchport mode access
 bandwidth 10000000
 delay 1
 spanning-tree portfast
!
interface GigabitEthernet0/2
 switchport mode dynamic desirable
!
interface GigabitEthernet0/3
 switchport mode dynamic desirable
!
interface GigabitEthernet0/4
 switchport mode dynamic desirable
!
interface GigabitEthernet0/5
 switchport mode dynamic desirable
!
interface GigabitEthernet0/6
 switchport mode dynamic desirable
!
interface GigabitEthernet0/7
 switchport mode dynamic desirable
!
interface GigabitEthernet0/8
 switchport mode dynamic desirable
!
interface GigabitEthernet0/9
 switchport mode dynamic desirable
!
interface GigabitEthernet0/10
 switchport mode dynamic desirable
!
interface GigabitEthernet0/11
 switchport mode access
 duplex full
 spanning-tree portfast
!
interface GigabitEthernet0/12
 switchport mode access
 bandwidth 10000000
 speed 1000
 duplex full
!
interface Vlan1
 ip address 192.168.80.254 255.255.255.0
!
ip classless
ip http server
ip http secure-server
!
!
control-plane
!
!
line con 0
line vty 0 4
 password cisco1
 login
line vty 5 15
 password cisco1
 login
!
end[/b]

The server currently sits on interface 0/12. 

interface GigabitEthernet0/12
 switchport mode access (This tells the switch the a computer is attached to this port, vs another switch or router.)
 bandwidth 10000000 (This manually sets the port to allow the use of maximum bandwith.)
speed 1000 (You guessed it.  This is tells the switch to run at 1000MB/sec)
duplex full (Run at full duplex).

My test media extender lives on port 0/11
interface GigabitEthernet0/11
 switchport mode access
 duplex full
 spanning-tree portfast (this is essential for PXE clients as it disables the 45 second port diagnostic and spanning tree checks that each port typically goes through when a new device comes on line.)

no service password-encryption (in the terminal configuration of a cisco switch, (config t) you can specify whether or not to encrypt all passwords.)

[colinjones]

you have your bandwidth set to 10Mb...?