applescript folksonomy (genre tagger)

 
  • Ahh that's great - It certainly works now as I've tried it on a few tracks. It's not the fastest thing in the world but it looks as though it's going to do what I want, thanks.

    I may well remove the grouping section, I'll have a play and see how goes but thanks again for that.


    • nautisch ha detto...
    • Utente
    • Feb 25 2009, 20:43
    well it definitely is not fast but i tried other scripts, it's just that the webpages load pretty slow (but they are essential ti the script..), I also added a 1 second delay (u are allowed to use the last.fm api only a i think once per second, btw, get an api_key!)

    tell me if you think any important tags are filtered that "should make it to the grouping" or if other things dont work at all or not to well.


    btw, a known bug is that if you mark too many files in itunes the script sometimes stops with some "time exceed" error, i do not know so far what causes that problem...

    ...jimi hendrix music aint no metal... although according to last.fm - itis..cant do anything if the tags are just plain dumb...

  • Yes, some of the genres are a bit strange, I'd not count "80's" as a genre, but some seem to come out like that. I'd also _generally_ expect all tracks in an album by a single artist to have the same genre, but it's not always the case.

    • nautisch ha detto...
    • Utente
    • Feb 25 2009, 21:26
    well the genre of each track is written based on the tag with the hightest count ( i excluded "rock" coz everything is rock according to last.fm..) so it is not the same for the whole album/artist (u can however use other scripts that just look up the genre for an album and apply that to all tracks)

    i never was interested too much in the genre function, i was/am more interested in the each tracks(!!) tags. It so far works quite good if you e.g. define a playlist that lists "sad" "metal" or "melancholic" "pop".

    btw, u can just erase "80s" etc from the valid_tags list and they'll be instant history...

  • Understood (and I've got an api btw, I grabbed it pretty quickly earlier). I shall tag up a few and experiment with smart playlists.

    • nautisch ha detto...
    • Utente
    • Feb 26 2009, 13:17
    ok cool.
    good luck ;-)

    if you have any more ideas, let me know.

    btw, you might want to experiment with the limit_absolut and _relative if you think too many/few tags are written to grouping/comment

  • Yet another version

    Hello, everybody,

    I stumbled over this pretty nifty script (BTW: I admire what you've done with it, nautisch; even though you're at the TU [just kidding]) two days ago and started making a few minor modifications and some code refactoring. All worked pretty solid and the fact that AppleScript isn't the fastest scripting processor was OK, as it ensured that requests to last.fm would follow in reasonable intervals. However, at one point I realized how awfully slow everything was, especially as I started moving away from barely known songs to pop songs yielding a whopping 100 tags. And then, the loops processing the XML gave me enough time (finding the tag with the highest count alone took 4 seconds) to think about tackling the problem with yet another solution.

    And then it hit me: Leopard comes bundled with PHP, all set, no hassles to configure it or whatever. So why not use that? It's fast and versatile, even though it lacks AppleScript's power to closely interact with the applications, such as iTunes. The solution: Use the best of both worlds. And that's what I did.
    Basically the AppleScript component acts as a highly functional wrapper bridging PHP power to iTunes. It fetches the tracks and executes the PHP script with the required information as argument. The PHP script in turn does all the tag fetching and filtering and so on and returns just the needed information to AppleScript which concludes by writing the information back into the iTunes tracks. The performance improvement is worth it: Fetching the XML from last.fm, filtering blacklist items, renaming synonyms, and selecting whitelisted and valid tags with a minimum amount of users --- all that is done in about 160ms. (Sure, this number depends on a lot of influencing factors. However, if you think about the 4 seconds it took on my machine to parse the XML once [sic!] in AppleScript, you'll catch my drift.)

    One thing that bugged me however, is the problem that nautisch already struggled with: The results from track.getTopTags didn't really reflect a numerical order. Just like the "metal-classic-rock-phenomenon" I was surprised to see that a pop song had more "electronic" tags than actual "pop" tags. Yet, the order of tags corresponded to the tag cloud on last.fm. Well, in fact, it's the global usage number of any given tag, that is, how often a tag is used in general on last.fm. This information, of course, is pretty useless to determine its importance for any given track. I'm eager to adopt v2.0 of the API but not at the cost of not having information that -to a certain extent- is provided by v1.0 of the API. In 1.0 tags are at least attributed a percentage which is calculated by dividing the tag usage by the usage of the tag with the highest usage (all in context of the given track).

    As I went along, I kinda got overzealous and wrote a lot of documentation and all that.

    So, here it goes:

    http://www.filefactory.com/file/af51dea/n/lastfm_tagger_v132_dmg

    Hope you like it and find it useful.

    Sylvère.

    • nautisch ha detto...
    • Utente
    • Mar 16 2009, 19:16
    nice, initially i intended to hack something in php coz i'm used to programming php, but then i found spoons script and ..well u know the story.
    i would have used xampp cozi did notknow that php is somehow included in leopard, but anyway, that sounds like a nice solution.

    i'mhaving some troubles downloading your file from filefactory but when that's done (sometimesoon i hope) i'll check it out. i guess from your post that you included synaonyms, blacklists etc as well which is good.

    and i really hope we'll find some way to solve that tag number cloud u know what i mean problem...

    btw, where are u from since your "mocking" munich ;-)


    edit:
    really nice.. and it's really damn fast. wondering how the tag selection works...


    BUG found (thought about solving it myself but u know the php better ;-)
    if a tag e.g. "german" is in the whitelist it appears in the tagcloud multiple times as the boolean is not 100% correct, u might want to change that.
    other than that it runs pretty well,thanx...

    just found out that there's also bugs, i.e.multiple same-tags inthe cloud for e.g. hip-hop (guess due to the synonyms... although u test twice if a tag is in the blacklist...?)

  • Fixing bugs, adding functionality

    Hi there!

    I'll go one by one through your post. Thanks for the feedback!

    "and i really hope we'll find some way to solve that tag number cloud u know what i mean problem... "

    As far as I can tell, it seems like this will never be resolved. The topic has been discussed a few times, but there never has been an official response to it (as far as I know).

    "btw, where are u from since your "mocking" munich ;-)"

    Um, from Munich. The "mocking" was referring to the TUM, as I work for the LMU. ;)

    "if a tag e.g. "german" is in the whitelist it appears in the tagcloud multiple times"

    Yes. I admit not having tested the script thoroughly (not on many files, anyway) and found this bug out just after my release. I actually wrote the script before seriously tagging my library and I have been optimising it ever since. Yet, my job keeps me quite busy so I didn't get really far.

    "just found out that there's also bugs, i.e.multiple same-tags inthe cloud for e.g. hip-hop (guess due to the synonyms..."

    I simply forgot the in_array($tagName, $tagList). Or I could have simply used array_unique before assembling the tag cloud. Anyway, that problem has been fixed.

    "although u test twice if a tag is in the blacklist...?)"

    This was primarily for performance reasons even though it is kind of ridicule. The first check is to prevent entering the foreach-loop checking for synonyms if we have a blacklisted tag which should absolutely not be included. The second check -as the comment says- tries to prevent blacklisted items to get into the tag cloud through a synonym. This bases on the assumption that a blacklisted tag should under no circumstances be included.

    Anyway, I'm posting the copy of my script that I actually work with right now. It's work-in-progress. As you can see, I have added a few synonyms and valid tags. This is to reflect my dumbed-down understanding of musical genres, as most of the genres confuse me and I just want some basic overview anyways.

    A new feature I added is the genre blacklist. With a couple of songs I had the genre set to tags that I think are viable information to be kept as tags but not useful as genre. So basically the genre blacklist excludes certain tags from being set as the genre.

    Then again, the main routine got quite more complex as I added some basic error checking and auto-correction. If no tag list is found (tag list found is true when 10 or more characters are found between opening and closing toptags XML tag), a little function tries to clean up common errors and retries getting a tag list. If that fails, it falls back to get the toptags of the given artist. Whether or not this is done can be defined via $fallbackToArtistTopTags and the minimum usage count (in percent) a tag must have in the artist tag cloud is set via $artistMinimumTagUsage, allowing for a higher cut-off (and thus including only tags that are very typical for the artist). If that fails as well, the script gives up. All this is logged to a HTML file (in the scripts directory) to ease manual correction and finding the errors.
    Currently I have been tagging a lot of very old samplers from the early 90s and the script yields approximately a 90% success rate in tagging. (On the other side: I'm not very picky and content if it finds any genre at all.) The 10 remaining percent being mostly one-hit-wonders that nobody today can remember (having maybe 100 followers on last.fm).

    Yet, there still is some room for improvements and as I tag along my library I will probably add more synonyms and such and maybe some more features that I see fit.

    Hoewever, here's a "beta" that I uploaded onto my server, hoping for it being less of a pain in serving the file:

    http://stoermann.info/lastfm/lastfm_tagger_v140b1.dmg

    Sylvère.

    • nautisch ha detto...
    • Utente
    • Mar 27 2009, 12:39
    hi there

    so far no time to really test the new version in detail but nevertheless i have another idea of making the skript EVEN cooler:
    what about writing a "tag" for each track if it is among the "best ofs" of any given artist? currently there are the "favourite" (and synonyms) tags which do not really capture what I mean (at least I think so). I'd quite like an option that uses the artist.toptracks xml to write a tag e.g. "toptrack" to a track if it's among the top X (static number or % of all tracks) of any given artist.

    that way one could easily make a "best of" playlist of any given artist. (nice if a library is kinda big...) Including that into the php would be feasible, but as you already know your code I guess it's far easier for you than for me ;-)

    btw thx fpr upping the script to your server instead of filefactory...

    • Bramw82 ha detto...
    • Utente
    • Apr 3 2009, 8:46

    No tags

    Magnificant script, love it, but ... some tracks are skipped in the tagging proces, altough the last.fm player for mac can find tags for them. any solutions?

    • Bramw82 ha detto...
    • Utente
    • Apr 3 2009, 8:52

    no popular tags =? no tags

    I discovered that when there are no popular tags, the script doesn't write tags at all to the grouping. i'm not a scripter,is there anyone else?

    • nautisch ha detto...
    • Utente
    • Apr 3 2009, 10:27
    not sure if i understand you correctly but you might want to check out the option in line 42 of the lastfm_tagger_helper and set that to a lower setting.

    also make sure the tags you want to use are included in the validTags (line 147 of the same script)

    hope that helps...

    • Bramw82 ha detto...
    • Utente
    • Apr 3 2009, 21:09

    Tags and TOPtags??

    so .. after some searching and #$%#$% i found out that there must be a difference between tags and toptags. when you have the last.fm player installed, you should try and plat some songs, and you will always see some tag... but when you click the tag button, you have 2 colums, the left one with your own tags for the track, the right one with "popular tags". somethimes this column is empty, altough the player gives you some tags. and when this happens, there are no tags written to the songs in itunes... hope someone understands my messy text?

    • Bramw82 ha detto...
    • Utente
    • Apr 3 2009, 21:22

    to be 100% clear

    forget my previous post, this is what i mean:

    somethimes the track.gettoptags results empty .. example

    http://ws.audioscrobbler.com/2.0/?method=track.gettoptags&artist=ozark%20henry&track=rainbow&api_key=b25b959554ed76058ac220b7b2e0a026

    any solutions?

    • nautisch ha detto...
    • Utente
    • Apr 7 2009, 19:12
    read sylveres post from march 21st where he describes
    "If that fails, it falls back to get the toptags of the given artist. Whether or not this is done can be defined via $fallbackToArtistTopTags and the minimum usage count"

    give that a try...

    • grintoul ha detto...
    • Utente
    • Apr 13 2009, 12:45

    Great script

    Looks useful... my iTunes genres are in a total mess so it'll save me hours, thanks! Now all I have to do is decipher the thread above and work out which is the latest version of the script and how to use it... :)

    • grintoul ha detto...
    • Utente
    • Apr 13 2009, 16:20

    Love it

    Just ran it on all my songs and they now have genres :) Great, thanks!

    • nautisch ha detto...
    • Utente
    • Apr 14 2009, 10:16
    if you only use it to write genres to music then you only use a fraction of it's functionality ;-)

    anyway glad you liked it...

    • grintoul ha detto...
    • Utente
    • Apr 17 2009, 11:19
    Well, genres is the first step :) Going to set up some smart playlists based on it as well, based on whether a tag is anywhere in the cloud it writes.

    • [Utente eliminato] ha detto...
    • Utente
    • Mag 1 2009, 2:38
    You are a fabulous, amazing person. Thank you *so much* for this!

    (Is there a way to disable genre changes, and keep the grouping changes?)

    • nautisch ha detto...
    • Utente
    • Mag 6 2009, 15:12
    i guess u are using sylveres version (php based?)

    • nautisch ha detto...
    • Utente
    • Mag 8 2009, 15:22
    whatever, just

    open http://stoermann.info/lastfm/lastfm_tagger_v140b1.dmg (or if you have already installed the script go to home library itunes scripts..)

    open (i.e. double click, should open in the script editor) "Tag selected tracks with last.fm" ,

    search for "theGenre" (exists in two lines only) and comment these by adding "--" in front of these to lines.

    should work...

    • nautisch ha detto...
    • Utente
    • Mag 8 2009, 17:37

    Artist TopTracks Update

    Hey there,

    so i finally hacked some more... the updated version (based on sylvère's php code) does the following:

    if a title is among the artists toptracks (based on http://ws.audioscrobbler.com/1.0/artist/{$artist}/toptracks.xml), the tag "toptrack" is added to the list of tags.

    You can define the number of Tracks that is given that tag, with descending popularity. e.g. when set to 2 only "with or without you" and "beautiful day" would get the tag, "one" would not. (see http://ws.audioscrobbler.com/1.0/artist/u2/toptracks.xml)

    shut that feature off by setting the variable $ArtistTopTracks in line 41 of lastfm_tagger_helper to -1


    download it here:
    http://www.michaelseemann.com/projects/folksonomy/ArtistTopTrack_Update_01.zip


    thought that is a good option to use smart playlists to generate "best of" lists...

    hope you like that addon, shoot me feedback..

Gli utenti anonimi non possono inviare messaggi. Per inserire messaggi nei forum, accedi o crea il tuo account.