Music recognition apps continue to capture the fascination of music lovers around the globe with millions of searches per day. The major players in this space are companies such as Shazam, SoundHound and musiXmatch. Shazam is one company that’s taking a significant step forward by using its ever growing music database to predict next year’s likely movers-and-shakers.
Predictive Analytics and Music Recognition
With the Shazam app users can upload segments of songs they are listening to – on the television, on the radio or even those heard when in a shopping center – and find out the name and artist of the song. Let’s say you’re in a bar, and you hear a song that you like but you don’t know its name. Shazam can help you find out the name of that song. It lets you record up to 15 seconds of the song and then it will tell you everything you want to know about that song: the artist, the name of the song, the album, offer you links on YouTube and iTunes, etc.
Users make 15 million song identifications daily and Shazam is using this data to predict artists that will receive mainstream attention next year. Shazam has good results when it comes to predicting artists who are likely to gain major traction; the music service anticipated that Lana del Ray would cause a stir in 2012 and their list of “Artists to Watch” in 2013 included the likes of French Montana, with both achieving major success in their respective years.
So how exactly does it work? Big data analytics is the key. Shazam aggregates reviews by critics alongside the number of people that have used Shazam to find a song to understand which artists are generating the most industry buzz. This means that instead of only relying on what the criticism (positive or negative) of the music has been, Shazam is able to use consumer behavior to better judge the artists that have already started to pique the interests of listeners and are starting to gain traction. Shazam’s technology is an excellent example of how predictive analytics is shaping industries far and wide.
Underlying the Search Algorithm
There is some fascinating technology underlying the Shazam app. Here is a research paper written by one of the developers of Shazam technology, Avery Li-Chun Wang, Chief Scientist and Co-founder of Shazam – An Industrial-Strength Audio Search Algorithm (PDF). The algorithm uses a combinatorially hashed time-frequency constellation analysis of the audio, yielding unusual properties such as transparency, in which multiple tracks mixed together may each be identified.
This may seem like magic to some, but as a data scientist I’ve been working with large, messy data sets for years, and statistical methods can be wonderful at extracting signal from noise. Still, think what this application accomplishes: Shazam can recognize a short audio sample of music that has been broadcast, mixed with heavy ambient noise, captured by a low-quality cellphone microphone and subjected to voice codec compression, amongst other abuses. Not only that, but the actual song is identified from an arbitrary 10-second sample within a database of over 2 million songs in just a few seconds. It’s impressive to consider how such fuzzy matching could be done, with such high reliability, in such a short time.
For a detailed analysis of how the Shazam alorithm works, here are a few articles that dig quite deep into the technology: “That Tune, Named” appearing in Slate, “How Shazam Works,” and “How does Shazam work to recognize a song?“. The key is in compressing the database of 2M tunes using spectrographic methods. Each song is processed to create a frequency-intensity chart over time from which (and this is the key) only the peaks are extracted. Each song is therefore converted into a “constellation map” (see sample below) of, essentially, the frequencies of the loudest notes over time:
The Future of Music Credits
As powerful as the music recognition software discussed above has become, the industry still needs a central, open repository of music credits. A new organization is starting up to provide just that – ProMusicDB is a fiscally sponsored project of the Pasadena Arts Council, a non-profit organization. Its mission statement is to become the “Smithsonian of Music Credits” by preserving the legacy and contribution of professional musicians and music creators through the development of a standardized, authenticated online music reference resource. Stay tuned for more from ProMusicDB.
