Robot Rock: How deep learning is central to Spotify’s strategy

Robot Rock: How deep learning is central to Spotify's strategy

Credit: Engadget

Spotify’s Discover Weekly is extremely popular with users for finding new music. Many however don’t realise that deep learning is at the core of the feature.

Many feared that the internet would kill albums. Cherry-picking highlights from an LP is now common and AI is central to the next chapter in music’s evolution.

Nearly 60 years after the Beatles revolutionised the way that we think about pop music, we’ve arrived at the playlist. The playlist is now more dominant than the album: perhaps a natural progression from the days of recording the songs you heard on the radio onto a cassette tape.

Playlists offer customisation. They can allow listeners to pick their favourite songs from their favourite artists and place them into a potentially endless stream, based on mood, genre or even an event. Playlists are more than just compilation devices, though. For many, they have replaced radio DJs.

Music curation – like genre, styles and trends themselves – has been through its fair share of change. We began with individuals who compiled playlists that they thought others would like before they were replaced by Pandora’s tagging system. Tagging enabled users to search for a label they liked and gave them recommendations. However, tagging is time-consuming. Music is subjective. When it comes to something as personal and divisive as music, there is always going to be difference.

Spotify’s Discover playlists are some of the most accurate ways to get pinpoint music recommendations. So when you’re looking for new artists, how does the Swedish service manage to so accurately hit the nail on the head, without tagging and without DJs?

How does Spotify use deep learning?

Spotify has been working with machine learning for over a decade. What started out as one man working on AI in the company as a kind of side-project has blossomed into a much bigger aspect of the company. Rather like how machine learning itself is typically comprised of multiple algorithms challenging each other, Spotify has a number of ways that they generate music recommendations.

One of the primary ways that they do this is through Natural Language Processing (NLP). NLP analyses huge swathes of natural language data and this can include speech recognition. In the case of music recommendations, Spotify crawls the internet for written text about music to find out what words are used when talking about particular songs, finding out which other songs are mentioned in relation to them. So if, for example, there is a lot written on the internet about Beyoncé in relation to Rihanna, the NLP algorithms will take this into account and recommend Rihanna songs to Beyoncé fans.

Deep learning keeps pushing itself to learn more. It’s perfect for a streaming service that wants us to stay plugged in and listening.

Spotify also employs Convolutional Neural Networks (CNN) to assess the waveform data of a song. This is a form of deep learning that looks at a minimum of three layers of nodes: an input layer, a hidden layer and an output layer. CNN is most often used in image recognition, often for faces.

Spotify uses CNNs slightly differently though, as they apply them to sound rather than vision. Audio frames are analysed for attributes such as tempo, loudness and key so that Spotify can group together songs with similar characteristics.

Spotify also uses a Python library to run a matrix formula. It compares each user for the songs they listen to against other users and the songs that they listen to, so from here, you can compare a song’s data with other songs and find out which are similar.

Why is deep learning so important to entertainment?

Much to our parents’ disgust, music is ever-changing. The tastes that we had when we were 20 change by the time we’re 30. The songs that we can’t get enough of grow old very quickly and that’s a challenge for Spotify.

Deep learning keeps pushing itself to learn more, assess more data and come out with better results. It’s perfect for a streaming service that wants us to stay plugged in and listening. Deep learning across the entertainment industry, however, is offering deeper personalisation than marketing has ever managed before.

An image of deep learning, AI, Robot Rock: How deep learning is central to Spotify's strategy
Spotify’s Discover playlists are easy to use and have received widespread praise for their ability to tap into users’ musical tastes. / Credit: The Verge

Deep learning identifies patterns in how people consume content. It assesses what you may call “micro-preferences”, from when a consumer is interested in a product to how they may react next. A feedback loop is a crucial part of AI learning, as the machine determines whether the decisions that it made were the right ones or not. Ultimately, the more data that a company has on you, the better a deep learning algorithm can “get to know you” and offer recommendations.

According to Netflix, 75% of what consumers watch on their service comes from their recommendation algorithms. Consider too that Netflix is well aboard the hyper-personalisation train. The service will generate a random thumbnail for a TV show based on the content that you watch. So if you’re particularly into horror movies, expect to see a darker movie poster than someone who is into romantic comedies: they might see a leading couple as their thumbnail, regardless of whether the series focuses around that couple.

We’re not that far away from movie trailers being cut and spliced based on what you’re more likely to respond to. If YouTube, for example, could harness the power of deep learning to learn what movies you enjoy most, it could easily randomly generate a trailer for a new movie based on the characteristics you’re likely to respond to.

Or, perhaps even generate a movie based on what you’d like.

Deep learning is integral to the future of music

Incredibly, Spotify CEO Daniel Ek didn’t originally like Discover Weekly.

“I would have killed that if it was just me, 100%,” said Ek of the feature. “I thought, ‘Oh, this is going to be a disaster.’ And then obviously it turned out to be something really successful. It’s one of the most loved product features that we have.”

It’s perhaps poetic that Ek didn’t rate deep learning, in the same way that many critics disliked the Beatles in the 1960s. Both AI and the Fab Four are revolutionary to popular culture: not just music.

Music is ever-changing but the wheel revolves. Trends spike into fashion before fading from view. The way that we consume music will continue to develop but with deep learning integrated into the world’s biggest music consumption platforms, recommendations will no doubt develop with the sounds we listen to.

An image of deep learning, AI, Robot Rock: How deep learning is central to Spotify's strategy

Luke Conrad

Technology & Marketing Enthusiast

Switching from monolithic web apps to microservices

Jon Hewines • 08th June 2023

Unlocking value in existing applications is highly desirable for CTOs, and one of the most effective way of doing so is to make the switch from monolithic architecture to microservices. It’s not a straightforward task, but the benefits can be huge.

BT launches 5G immersive spaces

Alex Foster • 26th May 2023

New connected spaces provide fully immersive experiences by combining interactive, 360° video content – complete with lights, sounds and smells – compatible with Augmented Reality (AR), Virtual Reality (VR) and Extended Reality (XR)

What are tomorrow’s cloud leaders thinking about today?

David Devine • 25th May 2023

Organisations today – from end-users to channel companies and industry groups – are all working to optimize their use of cloud technologies. Although there are increasing numbers of businesses seeing cloud’s benefits, and diminishing minorities operating on-premises, there are nonetheless hurdles that are common to all. OVHcloud’s annual conference at the London Stadium on June...

Job creation and learning from China

Arnold Ma • 22nd May 2023

With the barrage of negative headlines, however, it’s often easy to overlook how the introduction of and advances in new technologies can boost the economy by opening up job opportunities. And with reports of unemployment on the rise, as the UK’s economy struggles for growth momentum, this is more welcome news.