LinuxMCE Forums

General => Users => Topic started by: avajon on September 30, 2009, 02:03:51 pm

Title: Update media daemon
Post by: avajon on September 30, 2009, 02:03:51 pm
hi,

when i resynchronize a directory of videos i have sometimes problems with the metadatas. For example: i have two directories, in one there is the movie The bourne ultimatum. I insert the coverart and metadata from imdb and everthing seems ok. But when i synchronize the second video directory the movie scooby doo gets the same metadata from the first movie (bourne ultimatum). i tried to delete all data in media file sync and the id3 file, but when i recreate the metadata there is the same problem again...

first movie:
Code: [Select]
Reusing record PK_File 521 for /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi ? Md5 db , md5 file  <0xb6b976c0>
10 09/30/09 13:54:40.619 PlutoMediaFile::HandleFileNotInDatabase /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi N db-attr: 521 Inode: 67125258 size 1 mt 0/5, md5  <0xb6b976c0>
10 09/30/09 13:54:40.619 /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi not IN db-attr: 521 INode: 67125258 <0xb6b976c0>
10 09/30/09 13:54:40.619 Media Type is: 5, is folder 0 <0xb6b976c0>
10 09/30/09 13:54:40.620 Reusing file, updating media type to 5 <0xb6b976c0>
10 09/30/09 13:54:40.620 File /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi assigned to user 0 <0xb6b976c0>
10 09/30/09 13:54:40.621 File /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi on pluto device 0 <0xb6b976c0>
10 09/30/09 13:54:40.621 UpdateMedia::ReadDirectory PlutoMediaFile_.HandleFileNotInDatabase 521 <0xb6b976c0>
10 09/30/09 13:54:40.621 SetFileAttribute /home/public/data/videos/spielfilme1/DasBourneUltimatum.avi 521 <0xb6b976c0>

second movie:
Code: [Select]
Sync mode for /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg: modeBoth <0xb6c286c0>
05 09/30/09 13:54:12.462 # PlutoMediaFile STARTED: dir /home/public/data/videos/spielfilme2 file ScoobyDoo2.mpg <0xb6c286c0>
10 09/30/09 13:54:12.462 # ID3FileHandler::LoadAttributes: loading 0 attributes in the attribute file /home/public/data/videos/spielfilme2/ <0xb6c286c0>
10 09/30/09 13:54:12.462 # LoadPlutoAttributes: pluto attributes loaded (from id3 file - general object tag) 0 <0xb6c286c0>
10 09/30/09 13:54:12.462 # LoadPlutoAttributes: id3 attributes loaded (from id3 file - common tags) 0 <0xb6c286c0>
10 09/30/09 13:54:12.462 MergePictures: merging 0 pictures from our tag with 0 pictures from PIC tags <0xb6c286c0>
10 09/30/09 13:54:12.462 Processing path /home/public/data/videos/spielfilme2, file ScoobyDoo2.mpg. Found 0 attributes, 0 long attributes in file <0xb6c286c0>
10 09/30/09 13:54:12.463 HandleFileNotInDatabase media type 0 <0xb6c286c0>
05 09/30/09 13:54:12.463 Reusing record PK_File 521 for /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg ? Md5 db , md5 file  <0xb6c286c0>
10 09/30/09 13:54:12.466 PlutoMediaFile::HandleFileNotInDatabase /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg N db-attr: 521 Inode: 67125258 size 1 mt 0/5, md5  <0xb6c286c0>
10 09/30/09 13:54:12.466 /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg not IN db-attr: 521 INode: 67125258 <0xb6c286c0>
10 09/30/09 13:54:12.466 Media Type is: 5, is folder 0 <0xb6c286c0>
10 09/30/09 13:54:12.466 Reusing file, updating media type to 5 <0xb6c286c0>
10 09/30/09 13:54:12.467 File /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg assigned to user 0 <0xb6c286c0>
10 09/30/09 13:54:12.467 File /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg on pluto device 0 <0xb6c286c0>
10 09/30/09 13:54:12.468 UpdateMedia::ReadDirectory PlutoMediaFile_.HandleFileNotInDatabase 521 <0xb6c286c0>
10 09/30/09 13:54:12.468 SetFileAttribute /home/public/data/videos/spielfilme2/ScoobyDoo2.mpg 521 <0xb6c286c0>


you can see both of die videos have the same db-attr (521) and Inode (67125258). How can this happen? What can i do, that this will not happen again?

thanks
avajon
Title: Re: Update media daemon
Post by: merkur2k on September 30, 2009, 06:02:33 pm
that seems like it could only be caused by an underlying filesystem inconsistency. you may need to reboot to single user mode and run fsck on that drive.
Title: Re: Update media daemon
Post by: avajon on October 01, 2009, 03:30:10 pm
hi,

i think i know why this happens. in /home/public/data/videos i have 4 subdirectories where each directory has its own harddisc mounted. now, there are some movies in different directories/disks with the same inode-number.

can i manually change the inode of a file?
Title: Re: Update media daemon
Post by: tschak909 on October 01, 2009, 06:07:38 pm
DO NOT MESS WITH THE SYMLINKS!

updateMedia needs these to traverse different disks mounted as LMCE File Structure.

-Thom
Title: Re: Update media daemon
Post by: colinjones on October 02, 2009, 01:25:30 pm
thom - I don't necessarily think he means that he created his own symlinks for these 4 disks... I think he is just reflecting that he has 4 drives that have been pnp added to the system in the normal way... just that he has more than one media file that has the same inode number, by conincidence, because they are on separate disks. ... that said, it seems unlikely that would happen and be a problem for UM without it being a problem for many more people....
Title: Re: Update media daemon
Post by: avajon on October 02, 2009, 02:00:21 pm
correct - i didn't use symlinks. now i try to let linuxmce manage all my disks... i hope this will solve the problem with the same inode-numbers...

thanks
Title: Re: Update media daemon
Post by: Zaerc on October 02, 2009, 05:20:39 pm
Yet another good reason not to mount drives (and/or mess with the symlinks) manually...
Title: Re: Update media daemon
Post by: avajon on October 02, 2009, 06:20:18 pm
hi,

i don't think that has something todo with symlinks or mounts. Now i let lmce manage all my disks and i get the same issue as before. I have two movies on different disks with the same PK_File - Id in the database. I think the reason is, these two files have the same Inode.

inode:

Code: [Select]
dcerouter_1001968:/home# ls -lhi /home/public/data/videos/SAMSUNG_HD103UJ\ \(sdb1\)-CORE\ \[65\]/ScoobyDoo2.mpg
67125258 -rwxrwxr-x 1 bind public 2,4G 2008-12-08 20:09 /home/public/data/videos/SAMSUNG_HD103UJ (sdb1)-CORE [65]/ScoobyDoo2.mpg
dcerouter_1001968:/home# ls -lhi /home/public/data/videos/SAMSUNG_HD753LJ\ \(sda1\)-CORE\ \[64\]/DasBourneUltimatum.avi
67125258 -rwxrwxr-x 1 root public 2,5G 2008-06-03 16:36 /home/public/data/videos/SAMSUNG_HD753LJ (sda1)-CORE [64]/DasBourneUltimatum.avi

I saw this on more than this two files. So, after a lot of work the problem still exist...
Title: Re: Update media daemon
Post by: tschak909 on October 02, 2009, 06:28:55 pm
can someone look at updatemedia's source and verify this?

-Thom
Title: Re: Update media daemon
Post by: Zaerc on October 02, 2009, 07:10:47 pm
Somehow I doubt that the DB is going to spontaneously correct itself.  But since there are absolutely no details on what you "fixed" let alone how...
Title: Re: Update media daemon
Post by: colinjones on October 02, 2009, 11:25:46 pm
The function "int PlutoMediaFile::HandleFileNotInDatabase(int PK_MediaType)" in PlutoMediaFile.cpp (starts at line 166 in 0810) doesn't seem to make any distinction when checking the inode numbers between files on one file system and files on another.... ie it only checks the inode number, and doesn't check which directory (and therefore filesystem) the inode comes from. It also explicitly states in the comments that it only ever expects there to be a single file with a single inode number.

My question is, under normal circumstances, are inodes supposed to be system-wide unique or only unique within a particular filesystem. It seems to me that the latter is most likely, in which case inode duplicates could be possible with more than one filesystem, and so this code looks like it would be confused by that situation in the way the OP describes....

does that seem right?
Title: Re: Update media daemon
Post by: tschak909 on October 03, 2009, 12:08:10 am
that's definitely a bug.

inode #s should be unique only within a given filesystem. good find col!

-Thom
Title: Re: Update media daemon
Post by: colinjones on October 03, 2009, 12:25:42 am
cool... my first contribution! (although no way can a code a fix :) )

In the meantime avajon you can choose one of the files that has a duplicate inode number and move it from one filesystem to another, and then move it back again. This will force a new inode number to be allocated (hopefully unique!) Then force a resync in the web admin of the folders containing the two files that used to have the same inode number.

Now you can go into that folder in web admin with the "Show files who are missing from disk" option turned on and you should see the old version of the file which it now thinks it "missing" - delete that reference. And you should have resolved the problem.

Actually, come to think of it... it is probably better to do the deleting of the old reference between moving the file from its original location to a new filesystem/disk and returning it to its original location, so..

move file to a different disk
remove reference to missing file in web admin
move file back to original disk.
Title: Re: Update media daemon
Post by: avajon on October 03, 2009, 12:10:27 pm
thank you colin. i'll give it a try... ;)
Title: Re: Update media daemon
Post by: avajon on October 15, 2009, 03:29:08 pm
hey guys,

i tried what colin suggested, but this isn't a solution. When i move the file to a different disk, remove the reference to missing file in webadmin and move back to original disk i get the same inode number.
I also tried a lot of other things, but i always end in that a lot of files have the same inode. I moved the files from one disk to another (i have 4 hdds for the videos mounted under /home/public/data/videos)...

Is there a possibility to change the Inode manually and then resync the folders? Am i the only one who has this problem with more than one disk for the videos?

thanks.
Title: Re: Update media daemon
Post by: colinjones on October 15, 2009, 10:01:09 pm
No, I don't think there is an easy way to change the inode number manually. Seems strange that it maintains the inode number across multiple disks.... no idea how/why it does that. Here are a couple of other options.

It isn't a Single Instance Store even at the inode level, so if you make a copy of the file on the same disk, the copy should have a completely different inode number. If so, then you can delete the first file and let UpdateMedia rescan the copy. Failing that, if you have a Windows machine somewhere on your network, move the file there and then back again.... there is no way I can think of that Linux would be able to maintain the inode number in this scenario (unless it is somehow generating the inode number by hashing the filename or contents, but that is just ridiculous!)

Col.