LinuxMCE Forums

General => Developers => Topic started by: chriss on December 13, 2008, 03:10:40 pm

Title: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 13, 2008, 03:10:40 pm
Hi *,

I added an option to get my movies tagged with IMDB. It's basically working (see pictures attached) but still needs some fine tuning I think.

Cheers,
/chriss
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 13, 2008, 03:12:02 pm
BTW, it's quite slow ATM, see attached log output (50s for 20 IMDB items).
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Zaerc on December 13, 2008, 08:26:10 pm
Looks good, and 20 in 50s is still a lot faster then I can do manually. ;)
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: golgoj4 on December 13, 2008, 11:48:49 pm
how did you get around their tos? or did they open up a new api?
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 14, 2008, 12:50:58 am
how did you get around their tos? or did they open up a new api?

I just tried the first IMDB PHP library I found (http://sourceforge.net/projects/imdbphp/) and it works well.  ;D
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: hari on December 14, 2008, 12:54:47 am
cool!

br, Hari
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Zaerc on December 14, 2008, 02:08:34 am
If you open a ticket I'll patch it into the 0810 branch so it could be included in the next alpha update.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: totallymaxed on December 14, 2008, 09:46:12 am
how did you get around their tos? or did they open up a new api?

I just tried the first IMDB PHP library I found (http://sourceforge.net/projects/imdbphp/) and it works well.  ;D

Very nice! Now if we could also get IMDB used when a DVD is ID'd on insertion into a drive....

Great work.

All the best

Andrew
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: tschak909 on December 14, 2008, 11:24:13 am
Andrew, IMDB does not have this info, and unfortunately DVD fingerprints of the type that the EMI emits are very specific to itself and its endpoint.

This is a much tougher nut to crack than would first appear, especially since almost nobody has this information. If you'd like, I can explain the problem in more detail...

-Thom
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: totallymaxed on December 14, 2008, 11:45:21 am
Andrew, IMDB does not have this info, and unfortunately DVD fingerprints of the type that the EMI emits are very specific to itself and its endpoint.

This is a much tougher nut to crack than would first appear, especially since almost nobody has this information. If you'd like, I can explain the problem in more detail...

-Thom


Hi Tom,

I understand the DVD fingerprint problem. What I am interested in is the much richer data available from IMDB about each movie. It may be that a hybrid solution is possible where we allow the DVD to be ID'd using the EMI then we use that data to query the IMDB. This would be deliver a massive improvement in the attribute data per DVD.

As far as i can see this is technically feasible. What your view?

All the best

Andrew
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: tschak909 on December 14, 2008, 12:09:55 pm
Yes, this could be done, We might want to augment the media identifier stuff to be able to query more than one media identifier.. It seems to only work with a single media identifier device.

*hmm* too much to do.

-Thom
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: totallymaxed on December 14, 2008, 01:11:36 pm
Yes, this could be done, We might want to augment the media identifier stuff to be able to query more than one media identifier.. It seems to only work with a single media identifier device.

*hmm* too much to do.

-Thom


Yeah... just need to add it too the list ;-)

Your already doing more than your share Thom. Anyone else up for looking at this and then getting it into shape if it looks doable?

Andrew
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: freymann on December 14, 2008, 03:58:20 pm
Is there a link to the code that we can download and try?
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: tschak909 on December 14, 2008, 05:33:05 pm
Yes, the code is in SVN. You can do a checkout from this URL:

http://svn.linuxmce.org/svn/trunk/ for the current dev sources for 0810
http://svn.linuxmce.org/svn/branches/LinuxMCE-0710 for the 0710 sources.

you'll probably need some help setting it up as the self hosted build no longer works. So contact us in the channel.

-Thom
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 14, 2008, 05:42:47 pm
Is there a link to the code that we can download and try?

Freyman, if you are referring to the IMDB code please be patient. I'm still cleaning up a few things and will prepare a patch soon.

Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 14, 2008, 06:10:20 pm
Ok, I submitted the patch to Trac [1]. If you want to try it, you should also consider patching a small bug [2] (which appears on Amazon scans, too). It is working without any problems on my systems, however, use it at your own risk ;)

I'll appreciate any kind of feedback and suggestion.

Bye,
/chriss


[1] http://svn.linuxmce.org/trac.cgi/ticket/53
[2] http://svn.linuxmce.org/trac.cgi/ticket/52
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on December 14, 2008, 10:06:00 pm
Zaerc,

thanks for applying my patches!

/chriss
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Zaerc on December 14, 2008, 10:45:41 pm
No, thank you for your contributions.  Keep it up and you'll get your own SVN access after the worst dust has settled...

ANyway, I was just about to say that I've added them to the 0810 branch, but I guess you already noticed.  There will probably be another update tonight so it should be available to the alpha1 users tomorrow
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: nite_man on December 17, 2008, 11:56:05 am
Additionally we can create a simple Perl script, which will use IMDB::Film to try to retrieve a movie info by its file name in background. The module works fine and as I know many people use it.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 25, 2009, 12:15:27 am
Thanks for a great addition :)

I did have problems patching it...
I had to manually edit the following files:
coverArt.lang.php
editMediaFile.lang.php
editMediaFile.php

Would the .rej files help?

I also found 2 bugs, the first is just a a text that is not translated.
The option to include attributes is called "Include Amazon attributes" and should be IMDB.
The other is Genres. If a Genre does not exist, it gets added as a Title or a Album.
For example I have never used the Genre "Drama", all movies get a second Title called "Drama".
After adding the Genre "Drama" manually to one movie, it gets added correctly as a Genre.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 25, 2009, 08:44:44 am
I am now tagging my movies with IMDB attributes. I must say I love the extra info.

But not all is well... I have now come across 2 movies that produce a error.
I find the movie, but when I press "Get coverart" I get an error.

Step Brothers:
Code: [Select]
Fatal error: mysql error: [1062: Duplicate entry '25678-2131-0-0' for key 1] in EXECUTE("INSERT INTO File_Attribute (FK_File,FK_Attribute) VALUES ('25678','2131')") in /var/www/pluto-admin/include/adodb/adodb-errorhandler.inc.php on line 77
Transporter 2:
Code: [Select]
Fatal error: mysql error: [1062: Duplicate entry '26512-871-0-0' for key 1] in EXECUTE("INSERT INTO File_Attribute (FK_File,FK_Attribute) VALUES ('26512','871')") in /var/www/pluto-admin/include/adodb/adodb-errorhandler.inc.php on line 77
One other thing. It's slow (and worth it). Sometimes the search is fast, and whenever I see a difference in the speed the list is shorter. When It's slow, I get a duplicate list of selectable movies. When It's fast I get only the movie I'm after.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: tschak909 on January 25, 2009, 09:40:47 am
The speed has to do with the datagrid fetch.

As for the error you're getting, you'd only get that if you were manually fucking with the File table. (No, don't say otherwise, you were messing with the database manually. DON'T DO THAT!)

-Thom
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 25, 2009, 08:59:00 pm
Hehe... How do I fuck with the file table?!?

I'm sorry to say that I have not done anything to the database. But what I have done, is install my DVB-C card in the Core again hoping that it would be stable now... The problem is that every time I do that, the server starts hardlocking. Usually every 6-8 hours. Black screen, no response to keyboard/mouse, network is down, ssh not responding, no internet on other PC and the power button does not shutdown the server. I have to hold the power button til it shuts down.

The server did hardlock on me while I was tagging media, just after pressing the "Get Coverart" button and I had to power down and boot. That could explain the file table with errors.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 25, 2009, 10:19:08 pm
Just did a Clean 0710RC2 reinstall. Wiped the drive....
I patched with no media on the server.
As you can see, I get the same errors.

Code: [Select]
linuxmce@dcerouter:/var/www/pluto-admin$ sudo patch -p0 < imdb_cover_art.patch
(Stripping trailing CRs from patch.)
patching file coverArtExtractor_IMDB.php
(Stripping trailing CRs from patch.)
patching file include/imdb/browseremulator.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb_base.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb_config.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb_person.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb_person.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdb_request.class.php
(Stripping trailing CRs from patch.)
patching file include/imdb/imdbsearch.php
(Stripping trailing CRs from patch.)
patching file index.php
(Stripping trailing CRs from patch.)
patching file languages/en/coverArt.lang.php
Hunk #1 FAILED at 21.
1 out of 1 hunk FAILED -- saving rejects to file languages/en/coverArt.lang.php.                                                                                                                     rej
(Stripping trailing CRs from patch.)
patching file languages/en/editMediaFile.lang.php
Hunk #1 FAILED at 24.
1 out of 1 hunk FAILED -- saving rejects to file languages/en/editMediaFile.lang                                                                                                                     .php.rej
(Stripping trailing CRs from patch.)
patching file operations/mediaBrowser/checkIMDB.php
(Stripping trailing CRs from patch.)
patching file operations/mediaBrowser/editMediaFile.php
Hunk #1 FAILED at 97.
1 out of 1 hunk FAILED -- saving rejects to file operations/mediaBrowser/editMed                                                                                                                     iaFile.php.rej
linuxmce@dcerouter:/var/www/pluto-admin$

The .rej files is posted below:
coverArt.lang.php.rej
Code: [Select]
***************
*** 21,27 ****
  $TEXT_MATCHCOVERART_CONST='Match Cover Art';
  $TEXT_COVER_ARTS_UPDATED_CONST='Cover arts updated';
  $TEXT_GET_AMAZON_ATTRIBUTES_CONST='Get Amazon attributes & cover art';
  $TEXT_AMAZON_KEYWORD_CONST='Amazon keyword';
  $TEXT_FILENAME_CONST='Filename';
  $TEXT_WARNING_AUTOMATIC_GRAB_ACTIVE_CONST='Automatic script who retrieve coverarts from Amazon is working. You can stop it at any time or let it finish then match the results.';
  $TEXT_SEARCH_FOR_PATH_CONST='Search for path';
--- 21,29 ----
  $TEXT_MATCHCOVERART_CONST='Match Cover Art';
  $TEXT_COVER_ARTS_UPDATED_CONST='Cover arts updated';
  $TEXT_GET_AMAZON_ATTRIBUTES_CONST='Get Amazon attributes & cover art';
+ $TEXT_GET_IMDB_ATTRIBUTES_CONST='Get IMDB attributes & cover art';
  $TEXT_AMAZON_KEYWORD_CONST='Amazon keyword';
+ $TEXT_IMDB_KEYWORD_CONST='IMDB keyword';
  $TEXT_FILENAME_CONST='Filename';
  $TEXT_WARNING_AUTOMATIC_GRAB_ACTIVE_CONST='Automatic script who retrieve coverarts from Amazon is working. You can stop it at any time or let it finish then match the results.';
  $TEXT_SEARCH_FOR_PATH_CONST='Search for path';

editMediaFile.lang.php.rej
Code: [Select]
***************
*** 24,29 ****
  $TEXT_DATE_ADDED_CONST='Date added';
  $TEXT_CONFIRM_DELETE_PICTURE_CONST='Are you sure you want to delete this picture?';
  $TEXT_GET_INFO_ON_AMAZON_CONST='Get Amazon attributes';
  $TEXT_SEARCH_PHRASE_CONST='Search phrase';
  $TEXT_SEARCH_OPTIONS_CONST='Search options';
  $TEXT_SEARCH_AMAZON_CONST='Search AMAZON';
--- 24,30 ----
  $TEXT_DATE_ADDED_CONST='Date added';
  $TEXT_CONFIRM_DELETE_PICTURE_CONST='Are you sure you want to delete this picture?';
  $TEXT_GET_INFO_ON_AMAZON_CONST='Get Amazon attributes';
+ $TEXT_GET_INFO_ON_IMDB_CONST='Get IMDB attributes';
  $TEXT_SEARCH_PHRASE_CONST='Search phrase';
  $TEXT_SEARCH_OPTIONS_CONST='Search options';
  $TEXT_SEARCH_AMAZON_CONST='Search AMAZON';

editMediaFile.php.rej
Code: [Select]
***************
*** 97,102 ****
  $externalAttributesBtn='';
  if(in_array($rowFile['EK_MediaType'],array(3,4,5))){
  $externalAttributesBtn='<input type="button" class="button_fixed" value="Check Amazon" onClick="self.location=\'index.php?section=checkAmazon&fileID='.$fileID.'\'">';
  }
 
  $mediaSubTypes=getAssocArray('MediaSubType','PK_MediaSubType','Description',$mediadbADO,'ORDER BY Description ASC');
--- 97,103 ----
  $externalAttributesBtn='';
  if(in_array($rowFile['EK_MediaType'],array(3,4,5))){
  $externalAttributesBtn='<input type="button" class="button_fixed" value="Check Amazon" onClick="self.location=\'index.php?section=checkAmazon&fileID='.$fileID.'\'">';
+ $externalAttributesBtn .='<input type="button" class="button_fixed" value="Check IMDB" onClick="self.location=\'index.php?section=checkIMDB&fileID='.$fileID.'\'">';
  }
 
  $mediaSubTypes=getAssocArray('MediaSubType','PK_MediaSubType','Description',$mediadbADO,'ORDER BY Description ASC');
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 25, 2009, 10:39:16 pm
After manually patching the remaining files, Genres does get made correctly.

But I did get this error on the second media I tried adding after pressing the "Get Coverart" button:
Code: [Select]
Fatal error: mysql error: [1062: Duplicate entry '168-78-0-0' for key 1] in EXECUTE("INSERT INTO File_Attribute (FK_File,FK_Attribute) VALUES ('168','78')") in /var/www/pluto-admin/include/adodb/adodb-errorhandler.inc.php on line 77
The strange thing is, if I enter the same media again, now it has coverart and all the attributes.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on January 26, 2009, 07:14:55 am
cirion,

But I did get this error on the second media I tried adding after pressing the "Get Coverart" button:
Code: [Select]
Fatal error: mysql error: [1062: Duplicate entry '168-78-0-0' for key 1] in EXECUTE("INSERT INTO File_Attribute (FK_File,FK_Attribute) VALUES ('168','78')") in /var/www/pluto-admin/include/adodb/adodb-errorhandler.inc.php on line 77
The strange thing is, if I enter the same media again, now it has coverart and all the attributes.

I've seen this error happen to my system several times... no only with the IMDB patch but also with the older amazon attribute fetching. I haven't looked into this yet, but if you just do a reload the error does not occur again and the data is stored as expected.

I did have problems patching it...
I had to manually edit the following files:
coverArt.lang.php
editMediaFile.lang.php
editMediaFile.php

Hmm, would have to check the patch again. If I remember correctly, there was a problem when adding the stuff to SVN, too, but the problem was fixed. I think it was my fault because I somehow mixed up different branches (0710, 0810 and trunk).

I also found 2 bugs, the first is just a a text that is not translated.
The option to include attributes is called "Include Amazon attributes" and should be IMDB.

Ok, this is a simple one. I attached it to the original ticket and will fix it. (#53)

The other is Genres. If a Genre does not exist, it gets added as a Title or a Album.
For example I have never used the Genre "Drama", all movies get a second Title called "Drama".
After adding the Genre "Drama" manually to one movie, it gets added correctly as a Genre.

I think this is not related to the IMDB patch which only fetches the attributes. Under some circumstances you should see that behavior with amazon, too. Please try to apply the patch from ticket #52 (http://svn.linuxmce.org/trac.cgi/ticket/52) and report back if the problem stays.

One other thing. It's slow (and worth it). Sometimes the search is fast, and whenever I see a difference in the speed the list is shorter. When It's slow, I get a duplicate list of selectable movies. When It's fast I get only the movie I'm after.

Yeah, I know that it is slow. Unfortunately you have to do several queries to IMDB to fetch all the information and the more movies you find with the search, the slower it gets...

Anyway, thanks for the feedback!

br,
/chriss
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: nite_man on January 28, 2009, 10:18:11 pm
Is it possible to download IMDB patch as one file? It's too annoying to copy and paste many times :)
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: cirion on January 28, 2009, 11:15:38 pm
I used the link at the bottom of the page...
http://svn.linuxmce.org/trac.cgi/attachment/ticket/53/imdb_cover_art.patch?format=raw
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: chriss on January 29, 2009, 05:56:11 am
I used the link at the bottom of the page...
http://svn.linuxmce.org/trac.cgi/attachment/ticket/53/imdb_cover_art.patch?format=raw

The line numbers of the patches will be different to 0710 because there are several modifications already done for web admin in 0810. However, usually this should work without problems.

cirion,
did you apply the patch from ticket #52? Does it help your problem?

br,
/chriss
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Bozza on August 25, 2009, 02:15:56 pm
Attempted to use this script as amazon is no longer an option, after what I believe was a successful patch each attempt simply shows "no results"

Any clues what im doing wrong? or is this simply not compatiable with 710 anymore?

Any guideance is appreciated :)
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Bozza on August 25, 2009, 02:26:15 pm
Cancel that, i'm an idiot :P

I ran the patch more than once after it failed the first time so every file in the includes/imdb folder had the same code 3 times

silly me :P
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Itsik on September 09, 2009, 10:31:09 am
The imdb_cover_art.patch, contains multiple files, some of which are new and some are patches for existing once. How do I apply this patch ?

Thnx
Itsik
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: nite_man on September 09, 2009, 11:28:03 am
The imdb_cover_art.patch, contains multiple files, some of which are new and some are patches for existing once. How do I apply this patch ?

Thnx
Itsik

Just apply it and then resolve conflicts manually using *.orig and *.rej files.
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Itsik on September 09, 2009, 05:51:04 pm
I tried
Code: [Select]
patch -p0 <filename.patch> inside var/www/pluto-admin, but nothing happened
any chance you could write the exact phrase I need to use ?

thnx
Itsik
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: Zaerc on September 10, 2009, 01:27:50 am
I tried
Code: [Select]
patch -p0 <filename.patch> inside var/www/pluto-admin, but nothing happened
any chance you could write the exact phrase I need to use ?

thnx
Itsik
Here is the exact phrase you need to use:
Code: [Select]
man patch
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: igorguerra on January 05, 2011, 04:48:21 pm
I tried to use it, but I keep getting the following message, can anyone help me please?

Code: [Select]
(Stripping trailing CRs from patch.)
can't find file to patch at input line 2970
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: languages/en/coverArt.lang.php
|===================================================================
|--- languages/en/coverArt.lang.php     (revision 21561)
|+++ languages/en/coverArt.lang.php     (working copy)
--------------------------
File to patch:
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: kyfalcon on January 05, 2011, 10:00:49 pm
I tried
Code: [Select]
patch -p0 <filename.patch> inside var/www/pluto-admin, but nothing happened
any chance you could write the exact phrase I need to use ?

thnx
Itsik
Here is the exact phrase you need to use:
Code: [Select]
man patch


Classic!  ;D
Title: Re: Annoucement: Getting cover art and attributes from IMDB
Post by: apagg on January 05, 2011, 10:01:51 pm
Why are you doing this?

Regards
Andreas