This open source AI tool quickly isolates the vocals in every song

Splitting a song into separate vocals and instruments has always been a headache for producers, DJs and anyone who wants to play with isolated audio. There are many ways to do it, but the process can be time consuming and the results are often imperfect. A new open source AI tool makes this tricky task faster and easier.


The software is called Spleeter and has been developed by music streaming service Deezer for research purposes. Yesterday the company released it as one open source package, applying the code on Github for everyone to download and use. Simply enter an audio file and split it slit splits it into two, four or five separate audio tracks called stems. The results are not perfect, but they are excellent for use and Spleeter is very fast. When using a special GPU, it can split audio files 100 times faster into four stems than real-time.

Below you can listen to an example of the software that works on the "Changes" by David Bowie. There are a few audio artifacts in both the voice-only and band-only strains, but the overall results are fantastic. And if Bowie is not your thing, here it is another Spleeter example for that timeless ballad of love and loss: "Scatman (Ski-Ba-Bop-Ba-Dop-Bop)."

Technologist Andy Baio wrote an excellent one blog post about Spleeter with many personal examples. Baio says the isolated vocals produced by the software "sometimes get a robotic autotuned feeling, but the amount of bleeding is shockingly low compared to other solutions." You can listen to an example generated by Baio with Spleeter running on Marvin Gaye's "I Heard It Through the Grapevine." (But be sure to click through to his original post if you want more isolated vocal songs from Lil Nas X , Lizzo, Led Zeppelin and others.)

Marvin Gaye – "I heard it through the vine"

Baio points out that Spleeter will also be very useful for anyone who wants to make mashups, because he demonstrates himself with an unholy union of the friends theme melody ("I & # 39; ll Be There for You" by the Rembrandts) with the lyrics of Billy Joel & # 39; s "We Didn & # 39; t Start the Fire."

This tool seems extremely capable, but be warned: you need some technical expertise to use it. Unless you regularly play with software such as Python or Google & # 39; s TensorFlow AI Toolkit (which was used to train Spleeter), you need to download a few programs to get Spleeter working. And You should feel comfortable using a command line entry (albeit a very simple one) instead of a more accessible visual interface.

Deezer notes that this is not the first time people have used machine learning to automate this specific task, and that the company has built on much earlier research. Speak against The edge via e-mail, Deezer's data and research officer, Aurelien Herault, says the company has trained its software on 20,000 musical tracks with pre-isolated vocals in different genres. Based on this information, the software has learned how to isolate the tracks themselves.


In general, Spleeter is another fantastic example of how AI tools can make fiddly pieces of creative work easier. Machine learning is currently used to automate a series of time-consuming tasks, from removing backgrounds on images to scaling up textures in old video games. And increasingly these tools are included in consumer software, from Adobe Photoshop to new contenders such as Runway ML.

Deezer says it has no plans to turn Spleeter into a consumer tool, but others can do their job and save a simple interface. The obvious applications are for DJs and producers who want to integrate isolated vocals into mixes, or for people who want to create self-made karaoke backing tracks. (Such activities may not be in compliance with copyright laws, depending on how the end product is distributed.)

Deezer uses Spleeter itself for a series of research applications that help improve its streaming service. "Internally we use it as a pre-processing tool for complex research tasks such as music category, transcription and language detection," says Herault.

Or you can of course simply use it to get to know the Scatman better. Ski bi dibby dib yo da dub dub.