Take your top 32 artists and pit them against each other in a knock-out style tournament. For each "match", randomly shuffle the artist's songs in iTunes or Spotify and play the first song. that comes up After you play both artist's songs, the artist with the song you preferred goes forward, and so on and so forth.
Some other rules about songs:
1. No song can be played twice
2. No live versions of a song to be played if there are album versions
Here's how my tournament went (artist play rank in brackets):
First Round:
(1) LCD Soundsystem - Time to Get Away Beat (32) The Cardigans - Burning Down The House
(17) Fall Out Boy -
(25) Pendulum - The Island (Part 1) Beat The Hoosiers - Who Said Anything (About Falling In Love)?
(9) Passion Pit -
(4) The All-American Rejects -
…
Everything Will Be Alright