(Solved) I think. Media Files Sync will not refesh or update.

[rodercot]

 Hey All,

 I have a brand new (fresh install) today of LMCE710RC2. All is setup and running connected my Unraid server at 2pm today. 10 drives with movies and tv shows and music. Everything is now setup inside the lan under a LinuxMCE domain. I was surprised to find out that at now 5:19PM My Update media is still not done.

 I have some new movies that I installed while I was not running LMCE for a week or two all the rest of the files or most of them have .id3 tags in the folders already.

 Anyhow, I goto one of those new movies after doing a resync on that drive to force the files into the dB and then load the cover art from Amazon it finds it I choose it and then try to delete some of the attributes and set the media sub-type and then hit update it just runs forever I have to shut down Websdmin and then re-open and start a new session for every movie so twice per movie.

 I have the update daemon turned off as well as the firewall currently. When I open a new session the file is updated.

 any ideas!!!

 rgds,

 Dave

[tschak909]

* tschak909 *thwacks* rodercot HARD.

dude,

guess what?

if you turn off the update media daemon, YOU DON'T SEE ANY NEW MEDIA put into the database!

*shake-head*

-Thom

[rodercot]

 I did not have update media turned off all day. I turned off the update media when I a came home after four hours and only had one drive in the database and there are ten. then IF you read my post I tried to resync the drives at the files and media sync in Web admin. When I noticed it was taking forever to even update a file after changing the simple media sub-type.

 but as always, flame and flame but no questions answered as to how to fix the actual problem at hand.

 but thanks for the brilliant post.

 Dave

[colinjones]

Actually Dave, I must admit I read your initial post as you had had UpdateMedia turned off the whole time... it wasn't that clear...

Perhaps your database is corrupted? Do you know how to run SQL queries against it using mysql? Or perhaps if you have Excel you could install the MySQL ODBC connection provider and run queries from Excel - that way you could check how well it is responding.

Have you run "top" at the command line and checked your CPU usage, esp whilst you kick off a manual update? Also, in the output window that pops up when a manual scan is running, what is it logging? Is it freezing at any particular point, or just progressing continuously? Check what its doing when it freezes. I believe (purely anecdotally!) that the actual update to the database only occurs at the very end of the script. Whereas it seems that when it is running in the background, it seems it updates the database at the end of each directory - this obviously changes how things appear as it progresses, especially if you have a lot of media.

So when testing the scan manually, it is best to navigate into a small directory with no subdirectories and scan from there, so you get quick results.

[royw]

Another thing to verify is that the drives are accessible from /home/public/data/videos/*.  Actually cd to data/videos then ls each of the drives linked there.  Example (abridged):

linuxmce@dcerouter:/home/public/data/videos$ ls -l NFS\ Share\ \[39\]/
total 35463128
drwxr-sr-x 5 root public      20480 2008-06-28 07:40 Action
-rw-rw-r-- 1 root public 7686193152 2008-06-12 05:25 It Happened One Night - 1934.dvd
-rw-r--r-- 1 root public      79872 2008-06-22 23:41 It Happened One Night - 1934.dvd.id3

Next check ownership and permissions.  From the linuxmce side, you can see in the above example the files should be root:public 664

On checking the database:

Code: [Select]

linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'show databases;'
+--------------------------+
| Database                 |
+--------------------------+
| information_schema       |
| asterisk                 |
| asteriskcdrdb            |
| mysql                    |
| mythconverg              |
| pluto_main               |
| pluto_media              |
| pluto_myth               |
| pluto_security           |
| pluto_telecom            |
+--------------------------+
linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'use pluto_media;show tables;'
+-------------------------+
| Tables_in_pluto_media   |
+-------------------------+
| Attribute               |
| AttributeType           |
| Attribute_Settings      |
[snip]
linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'use pluto_media;select Filename from File where IsDirectory=0' | less


That should give you a warm fuzzy that your database is running.  It still could be corrupt but that's a can of worms to check...

HTH,
Roy

[rodercot]

Right! Sorry should have been more clear.  I posted a while back about a power outage that shut down both the nas and the hybrid and when I rebooted everything I had slow movie playback and the database was slow to update from within Media File Sync.

 I hope I am still making this clear enough to understand the problem.

 Webadmin - Media file Sync - /public/data/other - windows share disk 1 - 10.
 Disk 1-9 are setup - Movies1/movie_title/bup,ifo,vob etc.. also TV1/tv_show/season/episode s01e01 epname.avi as per proper show naming convention.

 Disk 10 is all HD/BD in a mkv for hd and ts for bd.  disk10 /movie_name/movie_name.mkv or .ts

 when I pick a movie from disk 1 - 9 shares from the left column is public/data/other/windows share disk 1/movies1. now choose a movie from the list in the right column ie terminator. it is seen as LinuxMCE DVD, I choose amazon and get the attributes it searches fine returns the results and I pick one, it update the terminator file with the cover art. Now I select the attributes in the radio boxes i do not want and choose delete, it pops up and asks me if I am sure I say OK and it starts to delete those and will then not refresh. so i have to shut down explorer or firefox and open a new session of the webadmin and get back to terminator and the attributes are in fact deleted now. It does the same thing for media sub-type etc...

 Now Disk10 was added and populated while the hybrid was not up over the last month or so with files as I stated above. It works just fine. I can add the cover art, delete the attributes and then set the media sub-types and all are ok in fact it refreshed and updates very quickly. Once completed with all the attr. for the movies on this disk I resync'd disk 10 and it took about 30 seconds to do so. So the disk 10 share seems to be OK.

 I did infact turn of the update media daemon after noticing the data had not been added to the db after 4-5 hours of building so I switched it off to try a couple files and see if there was any change in the performance of the db, which there was not so I turned it on again. I was only testing to see if that was causing any issues. I do not have the post or link handy, but I only did this in Lieu of a post or wiki issue I had read about.

 I am assuming as well it is a corruption somewhere in the db. as I stated this all happened after a power outage a while back. but I have since rebuilt the hybrid with a fresh install as of Sunday whereas I drew out a flowchart of the overall system, built and setup everything to that flowchart. I added nothing extra and did not try anything new this time as I have in past installs to progress my learning curve of the system.

 setup network wiring - setup all windos machines to LinuxMCE domain and dhcp (auto), reset the servers (Unraid) to LinuxMCE domain and shut them down from the Unraid admin window. Installed LMCE710RC2 - AV wizard issue - 640x480 instead of 1080p (1920x1080), installed Nvidia 173.14.09 driver and rebooted the hybrid - ALL GOOD. Setup 52d64u as per ales9r system sheet in the wiki, found and installed all windows worksations. booted first Unraid server found it and set it up - OK. Turned on Media Server, A little troublesome due to new nic in that machine had to change MAC address and reboot the server all found set up shares as per LMCE.

 Start using system. dvd playback OK, black screen on .ts and .mkv playback or still black screen and some sound then nothing just black screen.

 I hope this makes it more clear and not redundant. I guess I will do some searching for SQL issues. What am I looking for as far as an SQL issue or will I get a report.

 Roy - I have nothing in the videos dir, the movies and files are all in /public/data/other.

 One thing I did notice and not sure if this is an issue but when resync part way down the window it would have lines called "mapping file" and then continue with the resync. The resync on large queries do complete they just take forever.

 Thanks,

 Dave

[colinjones]

If you have rebuilt and are still experiencing the same problem, then I'm not sure what to suggest. That it was only happening on one share and not the others makes me think it was a DB corruption, but if you have rebuilt then that would seem to rule that out. You haven't indicated at all whether the symptoms are the same after the rebuild (esp. the drive 10 versus drives 1-9).

But do note that if you have a very large amount of media, it does take a long time to scan back in. Particularly some media types (like music and pictures). And the order that they are actually added to the DB (as mentioned in my previous post) will certainly effect the "apparent" speed.

I guess taking a "long time" is meaningless unless we know how much media is being scanned in. Perhaps look at how many pics/vids/etc are being scanned in per sec/min. If we are talking a minute per media file then yes there is definitely a performance issue somewhere. If it is grabbing one every few seconds then you are good.

[rodercot]

Roy,


 So I ran the second part of your post from the code window. I get all the tables and all the files in the db listed. I do notice some strange characters in some of my music son titles.

 Ie. Discoth<A1>qe.flac
 
 now as for the permissions.

 linuxmce@dcerouter:/home/public/data/other$ ls -l Windows\ Share-disk1\ \[49\]/
total 0
drwxrwxrwx 1 root root 0 2008-06-01 20:04 Movies1

 
 rgds,

 Dave

[tschak909]

apologies, I didn't realise.

[colinjones]

strange characters looks like it is trying to deal with unicode or something. that one is obviously an "e" with acute accent. can't see how that would effect performance, but there have been other cases where strange characters cause strange problems....

[rodercot]

If you have rebuilt and are still experiencing the same problem, then I'm not sure what to suggest. That it was only happening on one share and not the others makes me think it was a DB corruption, but if you have rebuilt then that would seem to rule that out. You haven't indicated at all whether the symptoms are the same after the rebuild (esp. the drive 10 versus drives 1-9).

But do note that if you have a very large amount of media, it does take a long time to scan back in. Particularly some media types (like music and pictures). And the order that they are actually added to the DB (as mentioned in my previous post) will certainly effect the "apparent" speed.

I guess taking a "long time" is meaningless unless we know how much media is being scanned in. Perhaps look at how many pics/vids/etc are being scanned in per sec/min. If we are talking a minute per media file then yes there is definitely a performance issue somewhere. If it is grabbing one every few seconds then you are good.

 Colin,

 I did rebuild yes, BUT my id3 files were still in the folders on the servers and that is what the db used to rebuild. I understand that it would take a while to rebuild the db from those id3 files as all the cover arts were no longer in the db from the old build so LMCE would have to go and get the cover art and populate the db again, I am assuming.

 symptoms before and after. Well up until the power outage all was fine, playback (normal dvd) was fine and updating a media file on disk 1 - 9 worked A-1. Disk 10 was not in the server at that time. As Soon as I rebooted after the power outage the dvd playback was slow (like in slow motion) lack of audio/video sync and then the db would not update as it doing now.

 With the clean install the dvd playback is fine but the files and media sync is as explained in the above posts. Disk 10 is performing as it should.

 Thom - no sweat chum. I know that it is difficult with all you guys are doing. I am sometimes not a very good communicator, at least that's what my better half tells me. I am not sure though " I wasn't really listening when she said that" LOL. I cannot remember what movie that is from. We have to make light of this stuff sometimes.

 Thanx,

 Dave

[Zaerc]

...
 Anyhow, I goto one of those new movies after doing a resync on that drive to force the files into the dB and then load the cover art from Amazon it finds it I choose it and then try to delete some of the attributes and set the media sub-type and then hit update it just runs forever I have to shut down Websdmin and then re-open and start a new session for every movie so twice per movie.
...

So you closed the window while the update (don't you mean resync?) was still running? 

[rodercot]

Zaerc,
 
 No! even though the resync took a very long time I let it runs it's course until complete and that was a per drive resync so I know there was a lot of info to scan.

 The trouble happens after the resync on all drives has been completed. where as I am just trying to Update a particular file on drive 1 - 9 and THEN it just runs forever and I have close and open a new session so really media files sync is just not refeshing when I do an update within a particular movie file.

 I have located the srange character files and fixed them all. They were namely all accents on the e's or i's in the song titles. I just renamed them an reran the second part of Royw's code suggestion from his post here and the strange characters are fixed.

 but in further checking that scan. I get this.

 In the middle of an album of flac files I also have movie names. see below.

Batman
1049 Last Fair Deal Gone Down.flac
Troy
Casino
32 New Orleans Is Sinking.flac
29 Ahead by a Century.flac
United 93
24 Grace, Too.flac
DanceswithWolves
27 Twist My Arm.flac
Dragon The Bruce Lee Story
25 Fully Completely.flac
folder.jpg
31 Courage.flac
Blue Collar Comedy Tour
AlbumArtSmall.jpg
The Aviator
Folder.jpg
37 The WhereWithal.flac
The Quick and the Dead
34 Scared.flac
26 Springtime in Vienna.flac
:

 I am not sure if this is a problem, but it could be also why I see movie cover art in my audio section of the orbiter.

 rgds,

 Dave
 

[rodercot]

Which mysql command line checks are we able to run on the db and can these be issued from SSH or local on the core.

 CHECK TABLE

 mysqlcheck

 I think these can both be run with the server up and running but not too sure what the actual command line would be for either.

 pluto_media File being the db and then the table.

 thx,

 Dave

[royw]

While I scratch my head on this one, I'm going to throw in an experience.  When first setting up NFS, performance was slow and unreliable.  Turned out to be a switch.  So the lesson learned is to try different network gear/config when things are flaky.  May or may not help, but you might try eliminating or replacing any switches and/or cables between your core and NAS.