2 min read AI & ML

Musical Key Recognition using a Hidden Markov Model

Every musical piece has a distinct key. In this project, a Hidden Markov Model is trained to try and predict such a key.
Musical Key Recognition using a Hidden Markov Model
Photo by Franck V. / Unsplash

Underlying every musical piece is its key. It determines the group of sound pitches that are used, and tells really a lot about its sound. Among others, it's useful for DJ's, who can benefit from selecting tracks that have similar key, so the sound matches nicely. Now, in this project, we took on the challenge of trying to train a model to predict such a key. Let's see how we did this.

Intuition

First of all, we must know there are 24 keys, 12 in major and 12 in minor. They can be illustrated like so:

Musical keys illustrated in a circle diagram. Courtesy of Tad Dreis.

Now, we chose Hidden Markov Models (HMMs) as our main instrument to try to predict the correct musical key, given a set of Chroma vectors. Such vectors represent the relative strength of each pitch in different segments of the track, which data we have for about 10,000 tracks. We obtain such Chroma vectors using Spotify's web API, which additionally gives us a ground-truth indication of the actual music key, such that we can use supervised learning to train our HMM.

The key insight is to train an HMM for both major and minor, separately. Then, we create copies of those models and retrain them with a circular shift in its key. In this way, we train a model for every key: such, we can predict the probability of a song belonging to a certain key, by searching for the model assigning the highest probability to its output. The process can be illustrated as follows:

Illustration of training an HMM using Chroma vectors. Courtesy of Peeters G (2006).

Execution

The actual implementation of the code and project can be found on GitHub:

music-key-classification

Published by:

Jeroen Overschie

You might also like...

24
Jul
Our internal AI agent for expenses: Bonbonnetje 🍬

Our internal AI agent for expenses: Bonbonnetje 🍬

Bonbonnetje is an AI Agent running in production on Agent Platform helping Xebians submit expense claims. Bonbonnetje is in active use by multiple internal departments and successfully completes hundreds of submissions monthly, helping save both employees and back-office valuable time.
5 min read
03
Oct
Fine-tuning LLMs yourself

Fine-tuning LLMs yourself

Fine-tuning can be a powerful way to specialised Large Language Models (LLMs). With open source models readily available, it is now also possible to fine-tune yourself. Read along to find out how.
7 min read
09
Apr
The GenAI automation potential of data extraction

The GenAI automation potential of data extraction

GenAI has been around for a little while and its capabilities are expanding quickly. But still, many companies fail to generate actual value with GenAI. Why is that? In this blogpost, we explore the GenAI automation potential that exists today for data extraction.
10 min read
27
Sep
The Levels of RAG

The Levels of RAG

LLM’s can be supercharged using a technique called RAG, allowing us to overcome dealbreaker problems like hallucinations or no access to internal data. But what can we expect from RAG? What use cases work well and which are more challenging? Let’s find out together!
15 min read
11
Jul
PyData Eindhoven 2024

PyData Eindhoven 2024

Eindhoven, The Netherlands — Retrieval Augmented Generation (RAG) is powerful. Let's go from Basic (chunking, vector embeddings, nearest-neighbor search) to Hybrid Search (TF-IDF, BM-25 and Reciprocal Rank Fusion) to Advanced (PDF parsing, tables, graphs) and finally, the cutting edge: Multimodal.