LinuxMCE Forums

General => Users => Topic started by: royw on May 20, 2008, 08:41:38 am

Title: Anyone using DVDProfiler?
Post by: royw on May 20, 2008, 08:41:38 am
Howdy,

For years, I've used the shareware DVDProfiler (http://www.invelos.com/) to manage my DVD collection.  The best feature is that I just scan the barcode of new DVDs and I get good data about the dvd.  While the amazon lookup that LMCE uses is ok, it is neither as accurate nor as complete as I am use to.

So I've written a little hack to allow me to sync the LMCE database from my DVDProfiler database.  It is currently something only the programmer who wrote it could love (it's a rails rake task :) ).

What I would like to know is if there is any interest in the community for this application? 


Thank you,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: cirion on May 21, 2008, 06:36:51 am
Sounds like a great addition to the media manager :)
I would love to have DVDProfiler support!
Title: Re: Anyone using DVDProfiler?
Post by: royw on May 27, 2008, 07:41:30 am
Howdy,

OK, made some progress on the dvdprofiler sync so thought I'd share it.

First, the sync consists of three commands: 
* isbn_dvdprofiler (used to add ISBN attribute to pluto_media File objects based on fuzzy title match with DVDProfiler exported profiles),
* sync_dvdprofiler (used to replace pluto_media File object's attributes with values from DVDProfiler exported profiles),
* missing_isbn (used to show.pluto_media File object's that don't have an ISBN attribute).

isbn_dvdprofiler
DVDProfiler keys it's database off of the movie's ISBN while LMCE keys off of path + filename (ok, technically the pluto_media.file object).  Each database has a title for each DVD, so the first command attempts to match via a fuzzy match (lowercase all words, drop punctuation,  with and without "The", "A", or "An" prefixes, minimize whitespaces).  This is matching about 90% of my 543 movies.  Matching does not work with boxed sets.  Actually I'm not sure how to handle multiple movies to one ISBN, currently I'm just ignoring them (i.e., use LMCE data).

sync_dvdprofiler
The second part is the actual sync'ing of attributes and images.  When a pluto_media File object has an "ISBN" attribute, then the following attributes are replaced using the DVDProfiler data:  Genre, Release Date, Rated, Run Time, DVD Release Date, Synopsis, Title, Performer (limited to first 10 actors).

missing_isbn
The last part is a report of what dvd media does not have an ISBN attribute.  Basically helps you to manually (via pluto admin) add the ISBN attribute for the files that the auto ISBN matching script had troubles with.

Workflow
The workflow is:

1) Export profiles from DVDProfiler
2) isbn_dvdprofile
3) sync_dvdprofile
4) missing_isbn
5) manually add ISBN attributes using pluto admin
6) go to #3

When you add DVDs or update profiles in DVDProfiler, then you just run through the workflow from the start.  There are some optimizations to prevent resyncing unchanged objects.

Current Status
Initially the script was developed on my workstation using a copy of the databases.  I've added Capistrano deployment to my dcerouter.  I've ran and verified core functionality of the commands on my dcerouter.

To Do
I need to test the condition where I do a reinstall without keeping the database to see what happens with the new ISBN pluto_media.AttributeType.  I pessimistically expect that update media might not like it.  Also need to verify that my capistrano deployment recipe contains all of the dependencies.

Next I'm going to move the functionality into a rails controller so the commands will be accessible via RESTful routes.  I'll write some simple scripts to access the commands via the RESTful routes.

Then I need to decide how best to package everything.

I'm still undecided on whether a web UI would be useful or not.  I'll probably play with a basic scaffolded UI just to see.

Have fun,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: Pastor on May 27, 2008, 09:31:45 am
Hi Roy

What a great update for Linuxmce, I used DVD profiler for years on my windows system. I would love to use it to catalogue my dvd collection probably from scratch asd I cannot find my "backed up" files!

Thanks a million for the work (a web ui is always nice to have as  choice)

Thanks


Paul
Title: Re: Anyone using DVDProfiler?
Post by: royw on June 08, 2008, 12:06:29 pm
Howdy,

Getting real close now for a beta release.  I had to do a redesign because adding additional AttributeTypes ('ISBN' and 'DvdProfiler LastEdited') were screwing up how LMCE finds Genres for displaying in UI2.  I couldn't find the code causing the problem (I think it is in orbiter generation, but it might be in the orbiter itself) so punted and created a new table that contains the two fields and a reference to the FK_File.

So what I need for beta testers are people who:
* use DvdProfiler.
* don't mind installing additional software on their core.
* comfortable using CLI on their core.
* willing to give feedback.
* don't get too upset if their db gets totaled requiring a re-install. :)

A note on the last bullet.  While developing this, I only had to do one re-install when I tried to remove unused genres and mistakenly removed records that still had references.  So IMO, the risk is very low.

The first release will have explicit installation instructions instead of a script.  Primarily because I want details of any problems.

The first release will be command line only.  I've worked on a GUI, but am having a little trouble with the background tasks.

Any volunteers?

I'll package up the beta and write the install docs in the next few days.

Have fun,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on June 09, 2008, 04:49:38 am
Count me in  ;D
Title: Re: Anyone using DVDProfiler?
Post by: Pastor on June 11, 2008, 09:59:24 am
Hi Roy

I would be delighted to help out just need to start cataloging my collection. Please count me in on the beta.

Thanks


Paul
Title: Re: Anyone using DVDProfiler?
Post by: royw on June 17, 2008, 10:55:07 am
Howdy,

Well, finally!  Announcing the first beta.  :)

I named the project LMCE Manager as the intent is to add features beyond DVD Profiler synch'ing.

Please start with the README file.  It contains installation instructions and how to use everything.

  http://roy.wright.org/lmce_manager/README

And here's the tarball:

  http://roy.wright.org/lmce_manager/lmce_manager-beta-124.tgz

FYI, the packaging number (124) is my svn revision.

BTW, this is for LinuxMCE 0710 rc2 and DVD Profiler 3.1.1

Have fun,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: royw on June 20, 2008, 08:14:46 am
Howdy,

Here's a couple of sample cron job scripts from my system.  I have them in cron.daily, but they could easily be in cron.hourly:

Code: [Select]
linuxmce@dcerouter:/etc/cron.daily$ cat sync-dvdprofiler
#!/usr/bin/env ruby

# Synchronize the LinuxMCE database from the exported DVD Profiler /var/rails/lmce_manager/db/dvdprofiler/Collection.xml

# run the lmce_manager's rake task
def run_rake(task)
        # rake tasks need to be ran from the project's root directory
        puts `cd /var/rails/lmce_manager && rake #{task}`
end

run_rake "dvdprofiler:sync:isbns"
run_rake "dvdprofiler:sync:attributes"

exit 0

Code: [Select]
linuxmce@dcerouter:/etc/cron.daily$ cat set-movie-attributes
#!/usr/bin/env ruby

# set all media under /home/(public|user_.+)/data/videos to be:
# MediaType = "LinuxMCE DVD"
# MediaSubType = "Movies"
# FileFormat = "DVD"

# run the lmce_manager's rake task
def run_rake(task)
        # rake tasks need to be ran from the project's root directory
        puts `cd /var/rails/lmce_manager && rake #{task}`
end

# The Dir='path' option is actually a pattern match on pluto_media.file.Path
# '*' characters map to SQL % (match anything, basically a /.*/)
# '?' characters map to SQL _ (match one character, basically a /./)

run_rake "lmce:set_mediatype_for_dir MediaType='LinuxMCE DVD' Dir='*/data/videos/*'"
run_rake "lmce:set_mediasubtype_for_dir MediaSubType='Movies' Dir='*/data/videos/*'"
run_rake "lmce:set_fileformat_for_dir MediaType='DVD' Dir='*/data/videos/*'"

exit 0

Have fun,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on July 22, 2008, 07:35:20 pm
I end up with an error unkown database 'lmce_manager_developent'  when i start to rake the system
 :(


Just solved my problem with "rake db:reset" to create a new database  ;D
Title: Re: Anyone using DVDProfiler?
Post by: royw on July 22, 2008, 08:02:38 pm
Good catch.  Sorry about that.

BTW, I'm getting close on beta 2 which adds a web interface and a setup script.  I have the dvdprofiler pages done, but am working on a pluto_main.file editing page to facilitate manually setting the ISBN.  I'm using this page as a learning experience for doing a progressive enhancement UI over RESTful routes.

Thank you for giving the package a try.

Have fun,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on July 22, 2008, 08:05:41 pm
I commend you on your work, Bloody thing rocs  ;D
oh windows user for dvdprofiler is documents\dvd profiler\databases\default\images <--- Vista that is
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on July 26, 2008, 06:34:58 am
Royw:  Having acouple of issue with the Lmce_manager database when i try to rake after updating my collection and copying over a new xml etc....  i get errors in the rake,  i will post but im redoing my core cuz of some other issue but what i had to do as a work around was to blowout the data base with reset and build again and all was well just a note,  still waiting on version 2 cant wait.  Wish you could get chapter info some how, that would be great anywho keep up the good work man
Title: Re: Anyone using DVDProfiler?
Post by: royw on July 26, 2008, 07:14:37 am
I'm very interested in the error messages, if you can capture them that would be great.  I've done a couple of dozen pushes of Collection.xml without issue so it's probably some config issue I overlooked.

DVD Profiler does not included chapter info.  If you add the LinuxMCE meta data, either by using LinuxMCE's rip or pluto-admin FAMS, search Amazon, then the media file should have chapter info.  The dvdprofiler sync should not erase it.  It only replaces a few of the attribute types like genre, title, performer, studios that it has in common.

I did find a problem last night, the lmce:db:backup task doesn't work.  This was mostly a dev task for letting me grab the db from my linuxmce and copy it to my dev system.  Let me know if you need that task, otherwise I'll include the correction in the next release.

Thank you,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on July 29, 2008, 02:17:45 am
Royw: Question are you gonna handle box sets in version 2 and is there a workaround now in version 1 that can be applied ?
Title: Re: Anyone using DVDProfiler?
Post by: royw on July 29, 2008, 03:13:08 am
That is a good question and I don't have a good answer yet.  Here's the issues.

The first option is simply what I'm currently doing, saying boxed sets and multi-packs are not synced from DVD Profiler.

I've thought of allowing multiple DVDs to have the same ISBN then provide a shadow title that if defined would be synced to the LinuxMCE Title attribute.  I just don't see how to get the shadow title from DVD Profiler so would have to provide UI for manually setting.  Same for images.  This is a limitation of DVD Profiler, i.e., one ISBN, one title, one pair of cover images. 

I've thought of using DVD Profiler's Discs fields, but their usage just doesn't seem to fit.

Here's some examples:

The Adventures of Indiana Jones
* contains:
** Indiana Jones and the Raiders of the Lost Ark
** Indiana Jones and the Temple of Doom
** Indiana Jones and the Last Crusade
* Only the collection box has an ISBN, the individual disks do not
* The three titles are only listed in the Overview
* There are no entries in the Discs section (I know, correctable)

Back to the Future, Ma & Pa Kettle, all have the same style

In Broken Trail, the feature is on two discs and the Discs sections is:
# Description
1 Main Feature: Part 1
2 Main Feature - Part 2

A lot of single titles will have Discs sections like:
# Description
1 Main Feature
2 Bonus Materials

TV series usually will fill in the Discs section like:
# Description
1 Episodes 1-7
2 Episodes 8-15
...

The only other option I can think of is to replace the box sets entries in my DVD Profiler database with individual entries.  In other words force my DVD Profiler database to only contain one title per ISBN.  That still doesn't work on epic movies that take two DVDs.

While I'm leaning on the first option, I'm open to any ideas here.

Thank you,
Roy
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on July 29, 2008, 03:44:52 am
Could you use the Disk ID Field since each one has a uniqe ID number and disk title from dvd profiler ?
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on September 26, 2008, 04:22:05 am
Any word on the new version  ;D

waiting to Beta test it
Title: Re: Anyone using DVDProfiler?
Post by: royw on September 27, 2008, 09:53:35 am
Sorry.  I working on a non-LinuxMCE project at the moment.  It'll be a couple of weeks before I get back to this.
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on December 13, 2008, 10:06:03 pm
Any one have problems updating to the 3.5.0 build 1240 of dvd profiler and raking the system im getting errors with the collection.xml

error "undefined method `downcase!' for {}:Hash"
Title: Re: Anyone using DVDProfiler?
Post by: MANDINGO on December 14, 2008, 04:35:53 pm
 ;D Bump