Today, I wanted to take a look into the popularity of different genres over time. We all know that rap has experienced a huge surge in popularity over the past few years, but what about pop? How does country music compare to rock these days? When was rock music most popular?
To answer these questions, the first step is to compile an appropriate data set. I started with the Billboard Hot 100, which records the 100 most popular singles in the United States each week going back to 1965. However, Billboard only records the artist name, without any of the artist’s information. To get the artist’s genres, I cross-referenced the Spotify API for each artist and recorded their music genres. Finally, I tallied up the genres to arrive at a count for the number of songs of each genre on the Billboard Hot 100 each week.
The next step was to condense the genres of the final data set. Spotify’s genre classification includes over 700 genres, including some extremely specific and obscure genres (ambient psychill, cantopop, and vintage french electronic, to name a few). To reduce the data set to a manageable number of genres, I manually clustered the genres to arrive at a final list of 7 genres - pop, rap, rock, electronic, r&b, soul, and country - which were able to classify 81% of all artists. One important note to make is that the “soul” genre is a combination of soul, funk, disco, and motown music. The combination seemed appropriate, as there was a significant overlap between these genres in the data set. (Further discussion about the clustering and condensing process is discussed at the end of this post.)
Once the genres had been condensed, a final dataset was produced of the number of songs of each of the final 7 genres on the Billboard Hot 100 each week. This data set was used to produce the following visualizations. The first visualization provides for a general overview of genre representation over time, while the second visualization makes it easy for a more granular comparison of genres.
Hover over the first graph (or tap on mobile) to see a monthly breakdown of genre composition. On the second graph, select the genres to display a genre comparison.
From analyzing the graphs, a few genre trends can be identified:
Overall, this initial exploration into the Billboard charts provided an insight into the rise and fall of genres over the past 50 years. From the data, it’s evident that we’ve lived through an extremely eventful era for music - since 2000, we’ve seen the decline of rock and soul and the simultaneous growth of rap, and we’re currently seeing the rise of electronic music into the mainstream. In 2018, I’m predicting the continuation of rap’s dominance as well as further electronic presence in the Hot 100, probably at the expense of current pop music. However, with the increasing influence of electronic music in pop, the distinction between the genres may become harder to make over time. Regardless of what happens, I’m looking forward to music in the future.
As a footnote, I’d like to expand on how the final dataset was compiled. The 7 final genres were determined through a manual clustering of Spotify’s data as follows:
Although I wasn’t familiar with all of the genres, I did my best to classify them correctly. Please let me know if you believe any of them would belong better in other genres!
Spotify also classifies artists with multiple genres, so I had to create a way to cluster an artist into their most representative genre. To do so, for each artist I selected the genre which contained the highest number of the artist’s subgenres, treating each subgenre as a single vote for the final genre.
Lastly, I do acknowledge that the dataset isn't perfect, as it tried to match Billboard historical data with the Spotify API. Some artist matchings are incorrect, as evidenced by the slivers of rap representation prior to rap's inception. In this case, some of these points are a result of Spotify mistaking "Sonny", the 1960s pop/rock singer, with "Sonny Digital", a current hiphop producer. These inconsistencies occurred less and less the more current the artists were, as my guess is that the Spotify API sorts results by popularity.