Clarifai Featured Hack: Use Clarifai for Audio Recognition with Adki

Clarifai is well-known in the artificial intelligence world for being the best image recognition technology out there. But, did you know that Clarifai can be used to recognize audio as well? ADKI is an app that extends Clarifai’s API to work with audio – here’s how!

Sound can be expressed in many ways – as beautiful music, cacophonous yelling, soothing white noise, or even as a colorful mosaic image.

ADKI is an app that extends Clarifai to work with audio by converting sounds into images using space-filling curves. With Clarifai’s Custom Training product, the ADKI team was able to teach the Clarifai API new audio-image concepts and train it to understand audio. Click on the image to the right to hear what you’re seeing!

WHY WE ❤ IT

We love when hackers push the envelope of Clarifai’s visual recognition technology beyond its obvious use. Applying Clarifai’s visual recognition to music is pretty novel – read more about ADKI on Devpost or check out the GitHub repo!

HOW YOU DO IT

We caught up with one of the creators of ADKI, Kristin Moser (NYU student, designer, developer, and full-time dog enthusiast), to talk about her team’s inspiration for the ADKI app.

Clarifai: What inspired your idea for ADKI?

Kristin: Music is really important to everyone on our team so we felt like we really wanted to work with audio; however, we also really love Clarifai so we thought it would be great to combine the two.

How did you build the app?

We used Python with Clarifai’s API and an S-curve API we found on Github. One challenge was not having enough data. Because the images created were very abstract to Clarifai, we really had to train it with a lot of data for it to recognize patterns.

What was the best part about working with the Clarifai API?

It was so fun to use. It made machine learning more understandable and made all of us really want to continue this in the future.

Thanks for sharing, Kristin!

To learn more, check out our documentation and sign-up for a free Clarifai account to start using our API – all it takes is three lines of code to get up and running! We’re super excited to share all the cool things built by our developer community, so don’t forget to tweet @Clarifai to show us your apps.

And give Kristin some props in the comments below. Until next time!