@abadesi Hey! As a guitar player, I always felt that I'm not playing good enough to jam with others. During my time at NYU, I was extremely inspired by recent developments in engineering fields, such as digital signal processing, audio analysis, and neural networks. This led me to build Soundscape, a platform that uses machine learning models to allow musicians find other people, who play similar music, with a single click of a button. This music discovery journey is based on the music characteristics alone, no matter how weird this music is :) There were quite a few obstacles along the way 😬. The first one was to find a simple and efficient way to analyze audio recordings, that could contain very different materials, from clean beats and melodies to noise, speech and ambient sounds. After experimenting with many different techniques and algorithms, the best result for the beat detection came from a combination of a recurrent neural network that was trained on annotated music samples, and a few music information retrieval algorithms. Then, I needed to find a quick way to mix different audio samples, based on the analysis I described in the previous paragraph. This took quite a long time to figure out. The last obstacle I'll mention is the deployment of this complex service to the cloud and making sure that users won't have any trouble using it. I'm pretty sure that not many people (or companies) dealt with deploying a machine learning based audio analysis service to the cloud and with serving users in real-time. One of my goals for this project was to build tools, that will allow other creative developers to build algorithmic-based music experiments. Last week I released these two Python packages: 🦉 AudioOwl -- Fast and simple music and audio analysis package https://github.com/dodiku/AudioOwl 🐻 MixingBear -- Automatic beat-mixing package https://github.com/dodiku/Mixing... These packages give developers the core functionality that I used for Soundscape in just a single line of code 💪. I'm happy that I had a chance to deal with these technical challenges, but to me, the biggest and most interesting challenge was how to hide all these technical aspects of the project from the users, and how to let them feel connected through music. The interaction itself will always be the thing I'm interested in mostly 😻.

Abadesi

Hustle Crew Academy

I'd love to hear more from you about how you got the idea for Soundscape @drorayalon, and were there any obstacles you had to overcome to get it off the ground?

7yr ago

Dror Ayalon

Soundscape

Maker