LinuxMCE Forums
General => Users => Topic started by: tschak909 on July 06, 2008, 11:59:06 pm
-
/me *thwacks* rodercot HARD.
dude,
guess what?
if you turn off the update media daemon, YOU DON'T SEE ANY NEW MEDIA put into the database!
*shake-head*
-Thom
-
Actually Dave, I must admit I read your initial post as you had had UpdateMedia turned off the whole time... it wasn't that clear...
Perhaps your database is corrupted? Do you know how to run SQL queries against it using mysql? Or perhaps if you have Excel you could install the MySQL ODBC connection provider and run queries from Excel - that way you could check how well it is responding.
Have you run "top" at the command line and checked your CPU usage, esp whilst you kick off a manual update? Also, in the output window that pops up when a manual scan is running, what is it logging? Is it freezing at any particular point, or just progressing continuously? Check what its doing when it freezes. I believe (purely anecdotally!) that the actual update to the database only occurs at the very end of the script. Whereas it seems that when it is running in the background, it seems it updates the database at the end of each directory - this obviously changes how things appear as it progresses, especially if you have a lot of media.
So when testing the scan manually, it is best to navigate into a small directory with no subdirectories and scan from there, so you get quick results.
-
Another thing to verify is that the drives are accessible from /home/public/data/videos/*. Actually cd to data/videos then ls each of the drives linked there. Example (abridged):
linuxmce@dcerouter:/home/public/data/videos$ ls -l NFS\ Share\ \[39\]/
total 35463128
drwxr-sr-x 5 root public 20480 2008-06-28 07:40 Action
-rw-rw-r-- 1 root public 7686193152 2008-06-12 05:25 It Happened One Night - 1934.dvd
-rw-r--r-- 1 root public 79872 2008-06-22 23:41 It Happened One Night - 1934.dvd.id3
Next check ownership and permissions. From the linuxmce side, you can see in the above example the files should be root:public 664
On checking the database:
linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'show databases;'
+--------------------------+
| Database |
+--------------------------+
| information_schema |
| asterisk |
| asteriskcdrdb |
| mysql |
| mythconverg |
| pluto_main |
| pluto_media |
| pluto_myth |
| pluto_security |
| pluto_telecom |
+--------------------------+
linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'use pluto_media;show tables;'
+-------------------------+
| Tables_in_pluto_media |
+-------------------------+
| Attribute |
| AttributeType |
| Attribute_Settings |
[snip]
linuxmce@dcerouter:/home/public/data/videos$ mysql -u root -e 'use pluto_media;select Filename from File where IsDirectory=0' | less
That should give you a warm fuzzy that your database is running. It still could be corrupt but that's a can of worms to check...
HTH,
Roy
-
If you have rebuilt and are still experiencing the same problem, then I'm not sure what to suggest. That it was only happening on one share and not the others makes me think it was a DB corruption, but if you have rebuilt then that would seem to rule that out. You haven't indicated at all whether the symptoms are the same after the rebuild (esp. the drive 10 versus drives 1-9).
But do note that if you have a very large amount of media, it does take a long time to scan back in. Particularly some media types (like music and pictures). And the order that they are actually added to the DB (as mentioned in my previous post) will certainly effect the "apparent" speed.
I guess taking a "long time" is meaningless unless we know how much media is being scanned in. Perhaps look at how many pics/vids/etc are being scanned in per sec/min. If we are talking a minute per media file then yes there is definitely a performance issue somewhere. If it is grabbing one every few seconds then you are good.
-
apologies, I didn't realise.
-
strange characters looks like it is trying to deal with unicode or something. that one is obviously an "e" with acute accent. can't see how that would effect performance, but there have been other cases where strange characters cause strange problems....
-
...
Anyhow, I goto one of those new movies after doing a resync on that drive to force the files into the dB and then load the cover art from Amazon it finds it I choose it and then try to delete some of the attributes and set the media sub-type and then hit update it just runs forever I have to shut down Websdmin and then re-open and start a new session for every movie so twice per movie.
...
So you closed the window while the update (don't you mean resync?) was still running?
-
While I scratch my head on this one, I'm going to throw in an experience. When first setting up NFS, performance was slow and unreliable. Turned out to be a switch. So the lesson learned is to try different network gear/config when things are flaky. May or may not help, but you might try eliminating or replacing any switches and/or cables between your core and NAS.
-
The 250GB of music isn't really relevant, only the number of music files. If you are talking about a couple thousand music files, then 1.5 hours is reasonable.
Correct - in a music folder, UpdateMedia is only looking for audio files. There is no reason to assume it would recognise another media players files (eg cover art). If you don't need it anymore then just delete it. If you do, just ignore it - it doesn't make any difference to LMCE.
As for the web admin issue - can I suggest that you use the Firefox browser actually from KDE desktop to eliminate any strange compatibility or network issues.
-
hmmm... that does seem a little excessive! I tried it, it takes less than a second from when I hit Update. Maybe time to reinstall LMCE? Add one share at a time and see when it starts...
-
Quick thought. Can the core connect to the internet? Just guessing an excess of connection timeouts might be the culprit.
HTH,
Roy
-
I've been working on the Media Browser and Media Files Sync all week, so I am quite familiar with the code. Can you better explain the problem? Are you getting these hangs only if you try to change the Media Type from Media Files Sync? Please tell me all actions that are causing your system to hang.
A couple of other quick notes.. File paths are passed as UTF-8 encoding. Make sure your naming convention conforms to this.
Also, when ever you change something from Media Files Sync (such as the media type you mentioned), the binary file UpdateMedia (located in /usr/pluto/bin) is run, then the page refreshed. It sounds to me like the problem is in UpdateMedia.. Maybe you should ssh to the core and try running it from the commandline to see if you get some output that could be useful for debugging...
sudo /usr/pluto/bin/UpdateMedia -d "/path/to/your/media/directory"
-
for a path, try /home/public/data/videos/..... all the way to your destination device.. for example, if I were to do it for my software raid device, my path would be:
"/home/public/data/videos/Software Raid 5 [64]"
-
*laughs-a-bit*
um, a CHECK TABLE isn't going to find table corruption unless the ISAM itself is corrupt.
For those that don't know, the pluto_media database, like the other pluto_* databases, takes the HABTM (Has and Belongs To Many) database design pattern to whole new heights. There are multiple join tables, which depend on each other, creating a nested join situation, a mis-match in any of the keys in the join table, will create a rippling effect that causes subsequent entries into tables like Attributes, etc. to become more increasingly corrupt, which is why it is a good idea TO NOT FUCK WITH THE DATABASE MANUALLY.
While sqlCVS -v can fix some of the problems in the database, it only can fix problems that are easily identifiable such as null foreign keys, or references to foreign keys that no longer exist.
-Thom
-
Thom, you just beat me to pointing out that the db check only verifies the db infrastructure and not the contents. I've played a little with a brute force content checker but it currently runs really slow. Oh, and on the HABTM design pattern, I wish it were, but unfortunately the join tables are not pure (they have fields in excess of two foreign keys), so it's more like a HABTM implemented with the Has Many design pattern.
Going back a few messages, if I understand correctly, UpdateMedia ran fine from the command line, the problem is just when updating from FAMS. If so, then the place to look is /var/log/apache2/error.log.
HTH,
Roy
-
the one with spaces won't run because of the shell, not because of mediaUpdate.. Place the path in quotes and try it again. If it runs fine from here, it should be running fine from the web admin.
-
if you are comfortable modifying the web admin .php source code, the next step would be to comment out where updateMedia is called...
ssh into the core.. then change to the directory where the web admin page is stored..
cd /var/www/pluto-admin/operations/mediaBrowser
then, you need to edit editMediaFile.php (if you're doing it from the command line, you'll probably have to use Vim, also remember you must be root)
sudo vim editMediaFile.php
now, towards the bottom (about a few pages up from the bottom or so), you'll see
if(isset($_POST['update'])){
$type=(int)$_POST['type'];
$subtype=(int)$_POST['subtype'];
$subtype=($subtype==0)?NULL:$subtype;
$fileFormat=(int)$_POST['fileFormat'];
$fileFormat=($fileFormat==0)?NULL:$fileFormat;
if(file_exists($oldFilePath)){
if($path==$oldPath){
if($fileName!=$oldFilename){
$isRenamed=web_rename($oldFilePath,$newFilePath);
$error=($isRenamed==0)?'Rename failed':'';
}
}
else{
copy($oldFilePath,$newFilePath);
exec_batch_command('sudo -u root rm -f "'.bash_escape($oldFilePath).'"');
}
}
$mediadbADO->Execute('UPDATE File SET Filename=?, Path=?, EK_MediaType=?,FK_MediaSubType=?,FK_FileFormat=? WHERE PK_File=?',array($fileName,$path,$type,$subtype,$fileFormat,$fileID));
// update pics urls
$picsArray=explode(',',$_POST['picsArray']);
foreach ($picsArray AS $pic){
$picUrl=cleanString(@$_POST['url_'.$pic]);
$mediadbADO->Execute('UPDATE Picture SET URL=? WHERE PK_Picture=?',array($picUrl,$pic));
}
$cmd='sudo -u root /usr/pluto/bin/UpdateMedia -d "'.bash_escape($path).'"';
exec_batch_command($cmd);
header('Location: index.php?section=editMediaFile&fileID='.$fileID.'&msg='.$TEXT_MEDIA_FILE_UPDATED_CONST.'&err='.@$error);
exit();
}
the line... "exec_batch_command($cmd)" is what is calling UpdateMedia... simply comment it out (with //) so that you have:
if(isset($_POST['update'])){
$type=(int)$_POST['type'];
$subtype=(int)$_POST['subtype'];
$subtype=($subtype==0)?NULL:$subtype;
$fileFormat=(int)$_POST['fileFormat'];
$fileFormat=($fileFormat==0)?NULL:$fileFormat;
if(file_exists($oldFilePath)){
if($path==$oldPath){
if($fileName!=$oldFilename){
$isRenamed=web_rename($oldFilePath,$newFilePath);
$error=($isRenamed==0)?'Rename failed':'';
}
}
else{
copy($oldFilePath,$newFilePath);
exec_batch_command('sudo -u root rm -f "'.bash_escape($oldFilePath).'"');
}
}
$mediadbADO->Execute('UPDATE File SET Filename=?, Path=?, EK_MediaType=?,FK_MediaSubType=?,FK_FileFormat=? WHERE PK_File=?',array($fileName,$path,$type,$subtype,$fileFormat,$fileID));
// update pics urls
$picsArray=explode(',',$_POST['picsArray']);
foreach ($picsArray AS $pic){
$picUrl=cleanString(@$_POST['url_'.$pic]);
$mediadbADO->Execute('UPDATE Picture SET URL=? WHERE PK_Picture=?',array($picUrl,$pic));
}
$cmd='sudo -u root /usr/pluto/bin/UpdateMedia -d "'.bash_escape($path).'"';
//exec_batch_command($cmd);
header('Location: index.php?section=editMediaFile&fileID='.$fileID.'&msg='.$TEXT_MEDIA_FILE_UPDATED_CONST.'&err='.@$error);
exit();
}
and now it won't be automatically executed. Save the file, then try to reproduce the bug again by changing the media type from the web admin.. if it still hangs, then UpdateMedia and/or exec_batch_command is not resoponsible, and we can look elsewhere...
(A few notes on VIM - if you've never used it, it can be daunting.. Look up how to use it online if you have to. The basics are, use your arrow keys and find the line to change. Press "I" to go into insert (edit) mode. You can now make your changes. When finished, hit esc to get out of insert mode. type :w to save (colon folowed by a w), the :q to quit...)
-
ok, at least we narrowed it down that much.. No, you did not have to reboot...
What browser are you using? Have you tried the web admin from any other browsers or computers?
-
Anything in /var/log/apache2/error.log?
-
To stop all of the favicon.ico missing errors, you just need to install a favicon.ico. Two ways. First way is to copy the blank one from LinuxMCE wiki (if you are using firefox 3, goto wiki, right click on a page, View Page Info, Media, select favicon.ico and save) to dcerouter:/var/www/localhost/htdocs. The second is reportedly (I haven't tried it) to simply touch /var/www/localhost/htdocs.
To ignore these messages when viewing a log, do an invert match with grep like: grep -v favicon | less
HTH,
Roy
-
My bad, mixing up systems in my head. The favicon.ico would need to go in /var/www
Just FYI, if you look in /etc/apache2/sites-enabled then you can find the DocumentRoot in the file pluto. The favicon.ico needs to be in the DocumentRoot.
linuxmce@dcerouter:/etc/apache2/sites-enabled$ grep DocumentRoot *
pluto: DocumentRoot /var/www/
I think the reinstall was a good decision. Hopefully it will work this time.
Have fun,
Roy
-
I have it working now at least I can update a media file in less than 10 seconds weather I am changing media sub-types or actually deleting a bunch of attributes it is the same 10 seconds.
Whoooop! Congrats!
Also - Roy - with that favico.ico issue, I copied that file from the wiki to my desktop - then I opened up dolphin and the /var/www files as root to past the file into that directory. When I ext Dolphin now I get an error that dolphin could not save the bookmark - What am I doing wrong here. I am not sure how to do this from terminal or the prompt.
CLI would be:
linuxmce@dcerouter:~$ cd Desktop
linuxmce@dcerouter:~/Desktop$ ls -l *.ico
-rw-r--r-- 1 linuxmce linuxmce 209 2008-07-12 22:14 favicon.ico
linuxmce@dcerouter:~/Desktop$ sudo mv favicon.ico /var/www
[sudo] password for linuxmce:
linuxmce@dcerouter:~/Desktop$ sudo chown www-data.www-data /var/www/favicon.ico
linuxmce@dcerouter:~/Desktop$ ls -l /var/www/*.ico
-rw-r--r-- 1 www-data www-data 209 2008-07-12 22:14 /var/www/favicon.ico
To test, tail -f /var/log/apache2/error.log
Then open up a browser to pluto-admin, you shouldn't see any new favicon errors.
HTH,
Roy
Maybe I'm a dinosaur, but I find dolphin unusable compared to konqueror. ;D
-
On the 25' cable issue. You may want to try putting new connectors on both ends before replacing the wire.
HTH,
Roy