LinuxMCE Forums

General => Users => Topic started by: colinjones on January 10, 2009, 02:31:39 am

Title: Auto tagging folder - just a thought
Post by: colinjones on January 10, 2009, 02:31:39 am
Was just thinking this over - for those of us who acquire lots of video media, such as TV shows via, say, torrents - tagging manually all the time can be a pain (esp compared with ripping DVDs and mp3 which usually get tagged automatically)

How about if you could mark certain folders under videos as auto-tagging folders. Such that you might have a "movies" folder, that automatically tags any media placed in there as of Movie type, same for Music Videos, etc. Even subfolders that might tag Genres, perhaps Years, or Show, etc. True, it wouldn't be practical to set Titles and so on in this way. But the advantage would be that many people already have a folder structure like this, and merely by directing their BT client to download to the correct folder, you could set 3 or 4 of the general attribute types automatically.

This would assist in increasing the usage of the various other Sort modes other than Filename. I know its almost heresy here, but I have to confess that I do use Filename mode very regularly for this reason! The system could be generic so that there isn't a defined folder structure, you can simply mark the folder you want to have the autotagging feature and then list the attributes you want to set, and their values (perhaps this could be extended at some point to include some level of macro/regular exp)

This could either be integrated into UpdateMedia or be a separate function completely that accesses the database directly to stamp the attributes. The only things I can think of that would need to be addressed would be whether it overrides existing tags, or whether you would have to manually go to the admin console and tell it to override; whether it would look at existing files in a folder just configured for autotagging or only new files....
Title: Re: Auto tagging folder - just a thought
Post by: samuelmukoti on January 10, 2009, 08:05:15 am
Hi,

I've also been looking at the same thing.  I've moved from XBMC to LMCE recently, and its cetainly one think I miss  TV Shows and Music Videos autotagging. 

Downloaded the source of XBMC and Entertainer (another MediaCenter) and am looking a stripping some of the tagging code and adapting it to LMCE.  Maybe I'll be able to come up with a script that we can hook into UpdateMedia so we can get auto tagging done. 

My experience with XBMC tagging is very positive, 90% it gets it right - and also passes .nfo files, which is also nice. We could go a step further and even support libmusicbrainz too for audio tagging - the sky is the limit. 

Lastly been able to manage this from the onscreen orbiter would also be a bonus, as my wife would not be interested  bringing the WebAdmin - as it is, it intimidates her, and she's not interest in going in there when ever she upload her media from her laptop to the NAS drive.

Will post my further progress on this thread.. I'm really excited about this.. and if someone has worked on this before i'd love to chat in IRC and put our heads together to get the best possible experience in LMCE


Was just thinking this over - for those of us who acquire lots of video media, such as TV shows via, say, torrents - tagging manually all the time can be a pain (esp compared with ripping DVDs and mp3 which usually get tagged automatically)

How about if you could mark certain folders under videos as auto-tagging folders. Such that you might have a "movies" folder, that automatically tags any media placed in there as of Movie type, same for Music Videos, etc. Even subfolders that might tag Genres, perhaps Years, or Show, etc. True, it wouldn't be practical to set Titles and so on in this way. But the advantage would be that many people already have a folder structure like this, and merely by directing their BT client to download to the correct folder, you could set 3 or 4 of the general attribute types automatically.

This would assist in increasing the usage of the various other Sort modes other than Filename. I know its almost heresy here, but I have to confess that I do use Filename mode very regularly for this reason! The system could be generic so that there isn't a defined folder structure, you can simply mark the folder you want to have the autotagging feature and then list the attributes you want to set, and their values (perhaps this could be extended at some point to include some level of macro/regular exp)

This could either be integrated into UpdateMedia or be a separate function completely that accesses the database directly to stamp the attributes. The only things I can think of that would need to be addressed would be whether it overrides existing tags, or whether you would have to manually go to the admin console and tell it to override; whether it would look at existing files in a folder just configured for autotagging or only new files....

best regards,

Sam
Title: Re: Auto tagging folder - just a thought
Post by: bulek on January 10, 2009, 08:31:45 am
Hi,

I'm also missing autotagging feature. I think this should be available as an option for each directory (or all directories).
I think what we need is:
- to get more info about current id3 format (it's binary)
- see if it can be written by scripting language
- write import script that will autofill id3 regarding various standard ways of tagging files (like folder.jpg being screenshot of folder, like same name.jpg means screenshot for the file name.xxx, etc...)

- it would be also nice feature (Freevo has this and it's quite functional) if you could change certain parameters for each directory (for instance reverse time order of display for surveillance videos, filter for displaying only certain files, etc...). I used Freevo some time ago and a lot of nice things comes out of that....

Regards,

Bulek.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on January 11, 2009, 12:38:26 am
Bulek - I don't think you need to go to the extent to writing id3 files - if you insert the metadata into the database, then UpdateMedia should just create the id3 file for you on the next path. If a trigger is needed then I'm sure that the code for this is in the web php code for the admin site when you manually tag a media file. This writes to the database and UpdateMedia on the next pass creates the id3 file, so the logic has to be there somewhere.
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on January 11, 2009, 12:46:40 am
Yes, an auto-tagging feature would be great!  I wrote a small app to help me with my tagging needs, and it's working pretty well so far, but it would be great to have something more integrated.  I'm probably not doing things the right way, but its a standalone app that reads from the media database with an optional filtering clause and lets you walk through and do some basic tagging.  It groups files together based on a common prefix (which would include but not be limited to directories), which helps with (for example) TV shows that might be in a root folder with other files, but share a common prefix.  Avoiding the web interface was also a goal of mine, so I tried to stick with only using big buttons easily hit quickly with the gyro remote.

I only bring this up because I think that I had similar motivations to what you guys are discussing.  Something rule-based that was more automated than what I'm currently using would be fantastic.  I think there probably would still be a place for something in the orbiter that let you walk through and easily tag on a more fine-grained basis, though.

If anyone is interested in this app in the meantime, I'd be happy to share the source.  It still needs some extensions, like offering the other media and file types. 
Title: Re: Auto tagging folder - just a thought
Post by: golgoj4 on January 15, 2009, 04:54:48 pm
lol, i should really get back to finishing what i started before. Seems like there are more people wanting this so is there a common language we can collaborate in?

Im ok with php/mysql
still learning the javascript
abandoned the adobe flash route.

i think we need to really push and get this done.

i will track down and strip out the code I have later and maybe see how to get this going once and for all. Where I left off was at the point of being somewhat overwhelmed by all of the things I wanted to add while at the same time realizing adobe flash was making the job way harder than it should have been (like smarter folks than me told me already  :D)

i figure i we combine all these attempts in to one master one, we can get something done that meets our goals.

-auto tagging would be nice
-extended metadata grabber via imdb lookup or other method. there are a few api's out there for this.
-drag and drop tagging for media

anyone else have any ideas?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on January 15, 2009, 05:10:44 pm
I've been using the auto-tagging part of it as an opportunity to get into development.  I have a 7.10 development environment set up and have been looking into adding the auto-tagging code to UpdateMedia.  My plan was to have rules attached to each folder, and update the UpdateMedia::ScanFiles method to load rules for each folder, and apply them to files that are detected as not yet in the database.  These rules could be recursive or not, and example rules would be:

Version 1 was to just have a few simple rule types (short and long attributes), loaded from a text file.  After that, a front end within the admin site (though I have zero php experience, so I wasn't looking forward to that part).  I was thinking nice to have features would be allowing custom rule types, and these rules should probably live in the database instead of the text file.

Does this sound in-line with what you were considering?  I'd definitely be interested in combining efforts ... the only reason I didn't mention that I was working on this was because getting the dev environment set up seemed like such a daunting task, and I didn't want to commit and then bail.
Title: Re: Auto tagging folder - just a thought
Post by: chriss on January 15, 2009, 10:59:56 pm
Hi guys,

I've been thinking about starting to redo the media management part in JavaEE, e.g., using the Seam framework for a web application. This would give us the opportunity to have a nicer GUI while sticking to the approach of being able to access your system via a simple web browser.

What do you think?

/chriss

ps I implemented a grabber for IMDB attributes for the current web-admin, check the svn...
Title: Re: Auto tagging folder - just a thought
Post by: golgoj4 on January 15, 2009, 11:15:24 pm
Hi guys,

I've been thinking about starting to redo the media management part in JavaEE, e.g., using the Seam framework for a web application. This would give us the opportunity to have a nicer GUI while sticking to the approach of being able to access your system via a simple web browser.

What do you think?

/chriss

ps I implemented a grabber for IMDB attributes for the current web-admin, check the svn...

im all for working together on this. I guess to get this seriously started we need:
-a wiki page laying out what we want to do
-volunteers to pick off a part and go to work on it
-....
-profit!!! :)

My main issue has been interpreting the ADODB that is used. I never tied anything into the webadmin because i could barely figure out what as going on. That should be a major focus so that we dont create problems down the line.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on January 15, 2009, 11:27:20 pm
jthodges - I definitely think it should be recursive at least as an option (default?) The thought was that you could potentially set up a folder structure like genre->series->episode or genre->artist->album, and then apply the folder attributes so that the genre flows all the way down, the artist only flows down the albums and tracks that it applies to and album slows to all is tracks. Should it have a "once off" vs "dynamic" radio button that tells UpdateMedia to restamp either only new media files or even media files that get moved from one location to another to aid management? If the latter or only a "new media" option then my other point of being able to track renamed/moved files better (perhaps by stamping the installation number and PK_File key) becomes more important as currently UpdateMedia considers a renamed or moved file to be a new one...

I think a simple "regular expression" concept would be good, but perhaps a true Regular Expression might be a bit complex (and unnecessarily comprehensive) for the average user?

golgoj - I thought your effort was focused on playlists? Would definitely like to see that expanded on jon's work (which is great). But using the same logic to do the manual tagging side of things would be great, too!
Title: Re: Auto tagging folder - just a thought
Post by: golgoj4 on January 16, 2009, 12:02:41 am
jthodges - I definitely think it should be recursive at least as an option (default?) The thought was that you could potentially set up a folder structure like genre->series->episode or genre->artist->album, and then apply the folder attributes so that the genre flows all the way down, the artist only flows down the albums and tracks that it applies to and album slows to all is tracks. Should it have a "once off" vs "dynamic" radio button that tells UpdateMedia to restamp either only new media files or even media files that get moved from one location to another to aid management? If the latter or only a "new media" option then my other point of being able to track renamed/moved files better (perhaps by stamping the installation number and PK_File key) becomes more important as currently UpdateMedia considers a renamed or moved file to be a new one...

I think a simple "regular expression" concept would be good, but perhaps a true Regular Expression might be a bit complex (and unnecessarily comprehensive) for the average user?

golgoj - I thought your effort was focused on playlists? Would definitely like to see that expanded on jon's work (which is great). But using the same logic to do the manual tagging side of things would be great, too!

ive probly done about 3 different versions (only showed the 1st) because I hated all of them :) I expanded it to media and playlists and gave myself a real headache. I dont have the largest media collection, but at a couple thousand video and media files managing the metadata in a way that was quick and intuitive became a road block for me.

anyways, back to teh drywall :)

http://langstonball.com/images/stories/LinuxMCE/howto/videopanellayout3.png
this is what the video attributes page last looked like. Attributes on the left with the file list in the middle bottom and the actual file data in the top middle. The empty spaces are where I intended the episode of movie description to land. Its a lot of info and I suppose its hard to decide whats important and whats not. And for some reason im to stupid to embed it properly atm.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on January 16, 2009, 12:19:01 am
funny, from what I remember it looked really cool! BTW, that link doesn't seem to work (404)
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on January 16, 2009, 12:25:23 am
jthodges - I definitely think it should be recursive at least as an option (default?) The thought was that you could potentially set up a folder structure like genre->series->episode or genre->artist->album, and then apply the folder attributes so that the genre flows all the way down, the artist only flows down the albums and tracks that it applies to and album slows to all is tracks.
Yeah, I agree recursive is possibly the only useful mode.  The management UI may want to hide the option to turn it off altogether to keep it simple.

Quote
Should it have a "once off" vs "dynamic" radio button that tells UpdateMedia to restamp either only new media files or even media files that get moved from one location to another to aid management? If the latter or only a "new media" option then my other point of being able to track renamed/moved files better (perhaps by stamping the installation number and PK_File key) becomes more important as currently UpdateMedia considers a renamed or moved file to be a new one...
Hmm, could you clarify this one?  I was under the impression that moved/renamed files were recognized and handled by UpdateMedia, and some (very minimal) testing showed the same.  I was considering the need for a "Run Now" for the rules to allow you to catch up your existing media, but it sounds like you are getting at something different.

Either way, moved files is definitely something to consider.  I think they would keep their tags (which I think would happen automatically), except as overwritten by any autotagging rules on the new directory. 

Quote
I think a simple "regular expression" concept would be good, but perhaps a true Regular Expression might be a bit complex (and unnecessarily comprehensive) for the average user?
I agree.  What I really want to accomplish is automatic titling based on standard file names (such as S01E06 type naming used for a lot of tv shows).  Perhaps the answer is to put some stock, well thought out regular expressions behind the scenes and just allow the user to write something like "24 - [Season].[Episode]".  Any other ideas?
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on January 16, 2009, 12:49:36 am
jt

I was told in the past, as well, that UM can handle moves/renames. It simply isn't true - or at least I have never seen an instance where it has worked. Theoretically, there are several triggers it could use as the db stores an MD5 hash (not sure how it gets this, whether it calculates it which seems unlikely due the the network load, or relies on the remote fs to supply it) and the inode number (presumably of the first block, but not sure how this works for non unix-like fs's but certainly all my media is on an NTFS share and these attributes are filled out fine!)

I was told it uses the inode. But in practice, if you rename a file or move it, UM finds it as a new file in the new location (you can see this in the UM log) and in the admin site, if you click the option to show missing files, you will see the old file in the old location. This not only prevents LMCE from treating a file as a continuous, discrete object in its logic, but also ensures that the database gets messy unless you scrupulously clean up or only use the admin site to do the moves/renames. That's a thought - you didn't think I mean via the admin site did you? Of course that works, it would be pointless otherwise. I'm talking about using any other method to move/rename files, LMCE terminal, Windows Explorer, NAS management tool. It is blinkered to say that file management should only ever be done within the admin console - there are many instances where this simply isn't practical or even possible

On the same subject, duplicating files is another difficult one. If you use only the MD5 hash to uniquely identify a file, then duplicating will screw up the logic as both files will match. This seems to indicate that a combination of MD5 hash and 'inode' need to be combined in the logic to complete it.

If a "new" file is encountered that has the same MD5 and inode as an "existing" file -> check does the "existing" file still exist? If no, then action was a move or rename -> update existing db record; else error?
If a "new" file is encountered that has the same MD5 but different inode as an "existing" file -> check does the "existing" file still exist? If yes, then action was a copy -> create new db record, and copy old db record's attributes over; else error? (although this could potentially still be a move/rename, just between different filesystems)

?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on January 16, 2009, 01:07:55 am
As I said, my test was simple and so didn't cover a network drive - that may be the source of the confusion.  I shutdown LMCE, moved my file and deleted the id3, and started LMCE back up.  The id3 reappeared, and the tags were still correct in the orbiter.  Maybe I'll try a network drive later on, but I can see what you mean about why it would possibly be different.

You are correct about the inode and MD5 use.  The relevant logic is in PlutoMediaFile::HandleFileNotInDatabase in PlutoMediaFile.cpp.  It looks for a File row with a matching inode, and then checks the MD5 of that row against the new file.  If they match, it is recovered.  Duplicate files should not be an assue, because the inode will not match.

Do you know if there's already a bug in the tracker for this?  My thoughts are that if this is not working correctly, that should be addressed directly in the existing UpdateMedia code rather than worked around with autotagging.  I'm not sure though, to be honest I'm still just getting my bearings.  It will be a little while before I get the basic functionality down, so we can kind of mull the options over in the meantime.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on January 16, 2009, 01:24:23 am
I did log a ticket in the old mantis system, and there was some discussion there - I believe that hari may still have access to that. But I never relogged it in trac. You are already far deeper in than I have ever got, so I will be of limited use to you except as a sounding board!

From your description of that method, it looks like the inode is the key issue here. If matching is done on that, then you would expect it to work on the local drives, and possibly/probably even on remote NFS shares. But on a remote NTFS/Windows share? We have to accept that a very large percentage of people will have their media on such a share. I hesitate to say perhaps even the majority?

So where the hell does it get these from on NTFS? I just looked at File's inode field - it clearly is not valid! And this is the reason it doesn't work. The inodes are different and probably unique, however they are allocated sequentially in the order that UM scanned them in with usually a gap of 2 or 3 (ie 1,3,5,7,9, etc or 1000,1003,1006,etc) Not only is that obviously not going to be the arrangement on disk, but also the gap between them is the same whether the file is a small jpg or an entire DVD. This could only work if they where indexes to pointers in a table. But my understanding was that inodes are absolute pointers to a physical block on disk, is that not the case??
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 22, 2009, 04:48:04 pm
I'm just getting back to the auto tagger after a long period of various distractions.  I have the basics in place, where it will automatically tag every new media file in a particular directory.  Short attributes, long attributes, and File columns (e.g., media sub-type, file format) are implemented.  The rules are stored in a configuration file for now, and there is not a friendly front-end for it.  I still think the rules will probably need to go in the database, but I'm waiting until the configuration is a little more well-defined to work that out. 

My next steps (in no particular order):

I'm hoping that others will find this useful as well, so please let me know if this direction makes sense or if you'd like to try it out.  I'm finding it's not incredibly useful yet without the pattern matching (I'm basically just using it to set media subtype on my prod environment right now), but once that and 'run rules now' are in place I think it will be pretty helpful.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on February 22, 2009, 10:32:21 pm
so cool! thanks for all your effort jt!! can we get it into 0810??
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 23, 2009, 05:13:43 am
so cool! thanks for all your effort jt!! can we get it into 0810??

That would be great, I'm just not really sure how I go about doing that (especially if the config data ends up in the database).  I will follow up with the devs when I wrap up the other items and see if that's a possibility.

In the meantime, I implemented the pattern-matching as a condition and 'run rules now' functionality.  I gave it some basic rules at /home/public/data/videos to tag anything with 720p / 1080p / dvd in the filename as the appropriate format, and let it run through my production system.  All looks good so far, and I think its in a much more useful state.  Now it just needs a front end and a better configuration store... that end is fairly ghetto at the moment :).


Title: Re: Auto tagging folder - just a thought
Post by: colinjones on February 23, 2009, 05:52:00 am
best to have a word with Thom, Hari or Zaerc about getting something checked in.... I will mention it to them now...
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 23, 2009, 10:16:00 pm
Thom is understandably a little nervous about including auto-tagging.  It sounds like the best direction is to just get people using it.  I also was toying with the idea of having the autotagger queue updates for approval rather than executing them directly.  This might help keep users from shooting themselves in the foot with a rule that is too broad, and there could be a hidden or heavily-disclaimered option to bypass the approval process.  I'm not sure if this kind of feature would help with the comfort level or would maybe just get in the way?

As far as getting others to use it, I think the main things in the way of that right now are lack of front-end and my ghetto config file.  Currently I'm using something like this:

Code: [Select]
[/home/public/data/videos/import/movies]
FileColumn, FK_MediaSubType, 2

[/home/public/data/videos/import/TV]
FileColumn, FK_MediaSubType, 1

[/home/public/data/videos]
FileColumn, FK_FileFormat, 4, 720p
FileColumn, FK_FileFormat, 5, 1080p
FileColumn, FK_FileFormat, 2, dvd
FileColumn, FK_FileFormat, 2, \.iso$

[/home/public/data/videos/wifesmovies]
ShortAttribute, 8, Chick Flick, true

To explain these examples:

Its basically a hacked up CSV file that I used as a temporary solution so I could get rolling on the functionality.  I'm not sure that I really see anything wrong with it right now, so maybe it can stay.  One possible issue is that it could get a little messy if any tag types allowed variable parameters.  Any thoughts?  Is there any advantage to having this stuff in the database?

As for the front-end, I'm getting ready to start looking at the admin website.  Fair warning though, I have very limited experience with php and UI is probably not my forte... if anyone wants to help with this portion it would be more than welcome!
Title: Re: Auto tagging folder - just a thought
Post by: Zaerc on February 23, 2009, 11:02:18 pm
Don't we have something like this in 0810 already?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 23, 2009, 11:08:00 pm
Don't we have something like this in 0810 already?

I'm not sure... the closest I know if is this: http://forum.linuxmce.org/index.php?topic=6571.0.  But I believe that is for applying tags recursively, rather than automatically tagging new media as it is added to the system. 

Please do let me know if this already exists, I'd rather not spend more time on it if it is already implemented...
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on February 24, 2009, 02:47:25 am
I think the recursive one is the only other thing there currently, and that is quiet a different beast.

1) How does your code handle pre-existing tags? ie if it already has a genre tag, does it add this as an additional tag or replace the existing one? Perhaps a flag that says replace|add|no-action when a tag already exists? Perhaps even an extra, new, attribute, boolean called Autotagged, that could be used to identify those entries that have been modded, which could allow for a rudimentary undo function...
2) Have you tested that once the db is updated, does updatemedia then correctly write those changes back out to both embedded id3 tags, and the .id3 tag files (depending on whether it is an mp3 or not)
3) Can you use your process to apply the '*' attribute that marks pictures as elligible for screen saver?
4) How does the process work? Is it a mod to UpdateMedia and happens as part of the scan, or a separate, new, process that runs at some other time?
5) How's your php? Is it possible to add a rough web page interface that provides a simple UI that writes out the config file?

sorry for all the q's!
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 24, 2009, 03:30:03 am
1) I'm working on the replace/add/no-action option right now, actually.  This didn't come up until I implemented the 'run rules now' feature, since new files wouldn't have existing tags.  And I like the idea of being able to undo, but I don't think I follow how the new attribute would allow it.  Can you expand on that?
2) When new files are tagged, yes (though I haven't specifically tested mp3s).  The db is updated just before the sync occurs, so id3 files are written out as UpdateMedia continues.  The 'run rules now' is a separate executable though, and the id3s aren't synced until the next UpdateMedia run.  I didn't think that was a big deal, but we could consider adding in a call to fire off UpdateMedia at the end if necessary.
3) I didn't know what that attribute was for until you asked the question, but yes the backend supports it.  It would be a rule like:
Code: [Select]
[/home/public/data/pictures/myscreensaverpics]
ShortAttribute, 30, *, true
4) Both.  First, an update to UpdateMedia for new files, which updates the new File row before the id3 is synced.  Second, the 'run rules now' is a standalone that uses the same library to load and execute rules against only files that are already in the db.  Note that when I say 'new file', I'm referring to new according to UpdateMedia. Files that are moved to the directory and recovered by UpdateMedia will have their paths updated in the database, but it will not trigger the auto-tagging.  The standalone would need to be used in this case.
5) Shaky, but yeah that's probably my next step.  I was thinking I would  extend the existing media sync screen, but it's starting to look like the rule definition will be different enough from the attribute creation that it will warrant a separate section or screen.

No problem, good questions and you are giving me some additional ideas for testing.  BTW, do you think the approval process is worthwhile?  I'm seeing it as something you use as you work through and test your auto-tagging rules, and eventually disable when things are running smoothly.  It's probably a significant extra bit of work though, so it's not worth it if it's just going to get in the way.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on February 24, 2009, 03:51:42 am
jt

1) Great. For undo, I haven't thought through the logic fully, but my idea was - if the new tag was added, when any tags are changed/added/deleted by your code, that would mean that with a simple SQL query you can select all the files that have been modified by your code. Then based on the folder location, you can determine what rules were applied to that file - so for a file that has an autotag attribute and an "add" rule, you can remove that tag added by the rule. A no-action rule only needs remove the tag if there is only exactly one instance of the tag and its value matches. Obviously, there isn't much you can do if the rule is a replace rule without getting tricky with history attributes (sure fire way of it not getting added to 0810!) Does that make sense?

One other point is, often "new" files will have tags. mp3s ripped from other sources will already have embedded tags, usually.

2) Probably should test the mp3s then as this is a different type of tag... I'm sure it will work if the .id3 ones do, though. I don't think it would be necessary to trigger UM, as the convergence time is never going to be much more than 2 mins.

3) Great!

4) Understood. In addition to moved files not being retagged, perhaps there should be some way of overriding the autotag, so that although you want a particular file to be filed with all the others in one of these folders, you could easily want a one off manual exception. Not sure that the file isn't "new" to UM is enough here, as manually moving a file, or even renaming it in situ will make UpdateMedia think it is new again... even manually editing attributes for mp3s I believe does this....

I'm not sure that the effort involved for the manual approval process is worth it. Its a great idea (perhaps use a db table as a queue like pnpqueue does), but perhaps the 'add' rule and even the 'undo' function provides enough security?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 24, 2009, 05:28:37 am
A no-action rule only needs remove the tag if there is only exactly one instance of the tag and its value matches. Obviously, there isn't much you can do if the rule is a replace rule without getting tricky with history attributes (sure fire way of it not getting added to 0810!) Does that make sense?
Yeah, I see what you are getting at.  I think that the reuse option for short attributes would screw up the logic for reverting a no-action as well, because it would not be clear if the tag was already associated with the file or not.  I think an approach that will adapt to more complex rules (as I expect these to get more complex) would probably be easier in the long run.  I could possibly extend the existing rules to log a description and sql necessary to revert the changes as they execute.  This could be used to provide a log of changes made as well as a method to revert them. 

Quote
One other point is, often "new" files will have tags. mp3s ripped from other sources will already have embedded tags, usually.
Good point.  I was thinking in terms of what what is in the database at the point the tags are applied, but that doesn't really matter.  I need to see how things are handled if existing tags conflict with a rule, and make sure the new replace/add/no-action rule handles it properly.

Quote
2) Probably should test the mp3s then as this is a different type of tag... I'm sure it will work if the .id3 ones do, though.
Yeah, I did look through that code before starting this and they are handled the same.  Still worth running some through and making sure.

Quote
4) Understood. In addition to moved files not being retagged, perhaps there should be some way of overriding the autotag, so that although you want a particular file to be filed with all the others in one of these folders, you could easily want a one off manual exception.  Not sure that the file isn't "new" to UM is enough here, as manually moving a file, or even renaming it in situ will make UpdateMedia think it is new again... even manually editing attributes for mp3s I believe does this....
I hadn't thought about exceptions ... I'll mull this one over.  It might make sense to add a new rule type for exceptions and allow it to be applied to a file or directory.  This would allow you to cut off another rule's recursion as well as handle the one-off files.

Quote
I'm not sure that the effort involved for the manual approval process is worth it. Its a great idea (perhaps use a db table as a queue like pnpqueue does), but perhaps the 'add' rule and even the 'undo' function provides enough security?
I agree, the undo is sounding like a better option.  It offers the security without getting in the way of the automation that the feature is supposed to offer. 
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on February 27, 2009, 04:59:22 pm
Update:


Title: Re: Auto tagging folder - just a thought
Post by: colinjones on February 27, 2009, 09:45:43 pm
jt - cool! Have a word with Thom in IRC and see if he is happier about putting this into 0810 now... looking good!
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on March 09, 2009, 11:06:33 pm
jt - how are you going with this?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on March 10, 2009, 02:55:13 am
The tests all went well, I just need to do the web admin.  Unfortunately, I've been swamped the last couple of weeks and unable to get a jump on it yet.  I'm out of town right now, but I will be back this weekend and am hoping to dig in at that point.
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on March 29, 2009, 04:54:17 pm
I sent a patch off to Thom for review.  If anyone wanted to test in the meantime, I would be grateful.  I've attached a patch for 0810, but if someone with 0710 wants to test, let me know and I can probably produce some binaries for you. 

Here are a couple screencaps of the front-end (hey, I said I was no php guy  :D):
http://i44.tinypic.com/96gjk5.jpg (http://i44.tinypic.com/96gjk5.jpg)
http://i44.tinypic.com/15x7ple.jpg (http://i44.tinypic.com/15x7ple.jpg)
http://i43.tinypic.com/eb53q1.jpg (http://i43.tinypic.com/eb53q1.jpg)

Here is some more information for anyone wanting to try the patch:

This code depends on libpcrecpp.  Install it with 'apt-get install
libpcrecpp0'

The basic structure of the changes is:
 - src/AutoTag - this produces libAutoTag.so and the AutoTag
standalone.  This handles managing rules and provides 'run rules now'
functionality.
 - src/UpdateMedia - I integrated libAutoTag.so here to automatically
tag newly-added files only.
 - web/lmce-admin - A couple new screens for managing rules and
restoring backups

After patching, you should just need to (1) 'make && make install' in
AutoTag, (2) 'make' in UpdateMedia, (3) deploy the new UpdateMedia to /usr/pluto/bin (backup the old one first), (4)
deploy the lmce-admin updates.
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on March 29, 2009, 11:06:15 pm
VERY cool!!!
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on June 04, 2009, 02:02:02 am
jt - any chance you could produce the 0710 binaries you mentioned? Unfortunately, I'm not in a position to upgrade to 0810 alpha yet as we are dependant on the Fiire remote which doesn't work in 0810. But I would still really like to test this functionality!!
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on June 09, 2009, 04:25:00 pm
Unfortunately, my 0710 development image seems to be out of commission, along with the MD that was the only machine I could run that image from.  I'm not sure how quickly I'll be able to resolve this (could be a while), but the patch for 0810 and additional info is at http://trac.linuxmce.org/trac.cgi/ticket/158 in the meantime.  If anybody has a 0710 dev environment and wants to take a stab at it, I'd be happy to provide any help I can!
Title: Re: Auto tagging folder - just a thought
Post by: colinjones on June 09, 2009, 11:01:52 pm
Thanks jt! Guess we'll probably just have to wait until the release!
Title: Re: Auto tagging folder - just a thought
Post by: pw44 on May 09, 2010, 08:27:38 pm
Any news about?
Title: Re: Auto tagging folder - just a thought
Post by: jthodges on May 12, 2010, 05:40:11 am
As I understand it, the developers have chosen to pass over it for this release.  Hopefully we will get it reviewed for the next release, but in the meantime the patch is still available on the trac ticket.  I haven't tested it with any recent releases and would welcome any testing as always ;).