A great musical score can make or break a film, and it’s no different for your home videos, either. Musify is an app that matches the mood of your video to appropriate background music, making your vid as equally enjoyable to listen to as it is to watch!
Musify is an app that makes it easy for beginners and content creators to find suitable background music that matches the mood of their videos. Using Clarifai’s Custom Training technology, Musify identifies broad categories like action or calm in a video and uses JukeDeck’s API to find an appropriate soundtrack to match.
WHY WE ❤ IT
Musify is one of the first developer hacks we’ve come across that combines Clarifai’s video recognition API with Clarifai’s Custom Training product. We’re really excited about the notion of a visual recognition model that can understand movie themes and film genres. Try it for yourself @ GitHub or read Yash’s excellent tutorial and code examples on his blog!
HOW YOU DO IT
We caught up with Yash Agrawal, a junior in Computer Engineering at the University of Illinois at Urbana-Champaign, to talk about the inspiration for Musify.
Clarifai: What inspired your idea for Musify?
Yash: We were thinking about something to make with Clarifai but couldn’t figure anything out so I was just browsing YouTube for some videos. I found a particular time lapse video with a really awkward background score and I realised that it was a real problem coming up with appropriate music. We wanted to make it easy for beginners and content creators to find suitable background music for their videos.
How did you build the app?
We used a bunch of Python libraries as we went along and a lot of caffeine. Musify takes in a video, uses Clarifai’s API on a custom trained model to find the mood and then uses JukeDeck’s API to find appropriate music. We also use a bunch of Python libraries to cut the original video into frames and overlay the sound.
What was the best part about working with the Clarifai API?
Clarifai’s API is so intelligent! The training of the entire set just took about 250 images and the results were near perfect so that was mind blowing!
Thanks for sharing, Yash!
To learn more, check out our documentation and sign-up for a free Clarifai account to start using our API – all it takes is three lines of code to get up and running! We’re super excited to share all the cool things built by our developer community, so don’t forget to tweet @Clarifai to show us your apps.
And give Yash some props in the comments below. Until next time!
