Iris is an application that helps people who can’t see experience vision in a new way, using gesture and image recognition to help blind people explore the world around them.
The first instinct many people have when they see our image recognition technology is, “Hey, this could be really helpful to blind people!” We’ve encountered a lot of apps that are meant to help the visually impaired “see” the world around them, but a lot of them fail to take into account the visually impaired user experience. For instance, we’ve seen apps require users to find and press a button on their (flat!) smartphone screens!
Iris is a new app that delivers a new way to see the world and really takes the user experience into account. With a simple gesture, Iris captures an image and gathers data about the image. Then, this information is spoken directly to the user without any physical interaction with the smartphone device.
WHY WE ❤ IT
We always try to deliver a smooth user experience that makes sense intuitively to our customers. That’s why we love the Iris team’s approach – from the beginning, they really thought about what would feel most natural and comfortable for a blind person to use. Read more about Iris on Devpost or check out the GitHub repo!
HOW YOU DO IT
We asked Chaitya Shah, a computer science student at Georgia Tech, to explain his team’s inspiration for Iris and how they built the app.
Clarifai: What inspired your idea for Iris?
Chaitya: My teammates Nick, Will and I came across the idea of Iris while exploring the unique ability of the Clarifai API to analyze images and parse them for descriptive tags. We broke Clarifai down into an equation: Picture + Clarifai = Tags. So why not flip that equation on its head and use tags to “create” a picture? Through the use of descriptive tags, Iris aurally illustrates a scene in the minds of the blind. After all, a picture is worth a thousand words.
What did you use to build your app?
Iris is an Android application coded entirely in Java. We also used the Clarifai API, Myo SDK, and Android’s TextToSpeech Engine. A lot of the challenges we faced were because of the many asynchronous threads required for Iris to function. We got around this by reorganizing our project in a certain way so that the correct functions were executed in the correct order.
What did you like about building an app with the Clarifai API?
In my opinion, the best part about creating Iris was showing people how it could really make a difference in the world. Demoing our app really made me feel proud of what we had done and proud of where my team’s knowledge has brought us.
Thanks for sharing, Iris team!
To learn more, check out our documentation and sign-up for a free Clarifai account to start using our API – all it takes is three lines of code to get up and running! We’re super excited to share all the cool things built by our developer community, so don’t forget to tweet @Clarifai to show us your apps.
And give Chaitya, Nicholas Petosa, and William Ma of the Iris team some props in the comments below. Until next time!
