Dr. Dobb's | Music Search Engine: Identifying Musically Meaningful Words

Music Search Engine: Identifying Musically Meaningful Words

Listen Game is an online, multiplayer game prototype that gets players to label songs with words

September 26, 2007
URL:http://www.drdobbs.com/web-development/music-search-engine-identifying-musicall/202101879

Electrical engineers and computer scientists at the University of California, San Diego are working together on a computerized system that will make it easy for people who are not music experts to find the kind of music they want to listen to -" without knowing the names of artists or songs.

In A Game-Based Approach for Collecting Semantic Annotations of Music, Doug Turnbull, Luke Barrington, and Gert Lanckriet demonstrate that the online music game they created provides crucial data for building the back-end of a music search engine that allows users to type in words to find songs.

"When my mom gets up in the morning and is like, 'I need some energy to go jogging,' she has no clue what title or artist is going to help her with that," said Gert Lanckriet, the electrical engineering professor overseeing the project.

What Lanckriet's mom needs is a "Google for music" -" a search engine for music that lets you type in regular words like "high energy instrumental with piano, " "funky guitar solos" or "upbeat music with female vocals, " and get songs in return.

One option for creating this kind of natural-language music search engine is to manually annotate as many songs as possible -" but this is expensive and limits the depth and breadth of the search engine's reach. Another option is to train computers to do the song annotations.

The researchers have, in fact, built such a system over the last two years. They call it a "computer audition system." You feed it songs and it annotates them, thanks to a series of algorithms they created. Once a song is annotated, you can retrieve it using a text-based search engine. But before the system can start annotating songs, it has to be trained "- via a process of machine learning. Getting enough data to properly train the system to label a wide range of music accurately is difficult.

According to Lanckriet, an online music game they created, called Listen Game, is capable of capturing the crucial word-song combinations that are needed to train their system to label large numbers of songs automatically.

Listen Game is an online, multiplayer game prototype that gets players to label songs with words. Like the popular image labeling games, such as ESP game, Peekaboom and Phetch, Listen Game relies on people surfing the Web to generate valuable data while playing the game. For Listen Game, the "human computation" occurs when users listen to clips of songs and determine which words are most and least relevant to the song. Users earn points when they pick the same words as others who are playing at the same time and listening to the same song clips.

Researchers use these "semantic annotations of music" to train their home-grown computer system to label songs it has not previously encountered.

"We've shown "- in academic terms -" that our game works. We're close to the performance we get with comparable survey data from the music undergrads we paid to fill out music surveys," said Doug Turnbull, a computer science Ph.D. student who is working on the project.