Hack Days at Clarifai are all about innovation and teamwork. During our recent Hack Day in March, one project stood out as an impressive collaboration: ClariCam. Six individuals from three different teams came together to create a smart camera prototype in the span of a day!
The ClariCam device runs Clarifai’s upcoming IoT SDK on a Raspberry Pi with an attached camera, housed within a custom designed hand-held structure. In operation, the ClariCam processes images captured by the camera through a Clarifai computer vision model and displays the results (a set of concepts) to an output screen.
The project is a proof of concept for IoT devices running Clarifai. The prototype demonstrated two usage patterns:
1. A smart camera running inference locally and sending output to a server
2. A server handling the inference for several smart cameras
Dalmo Cirne, Clarifai’s Director of Edge Computing: “In our first demo we showed how a small device, with very limited computing power, can be a smart camera and run live inference on the images it captures. In our second demo, we showed a camera delegating the inference part to a central computer and receiving the result back to be displayed on a screen.”
Running the inference (i.e. processing results) locally on an IoT device is useful because you can get results with or without network connectivity, and faster. Also, if the IoT device doesn't need to connect to an external inference engine, data can stay on the device.
Example use-cases would be a flying drone looking for a lost dog or a coffee machine detecting when pods have been used.
For use cases where the server does the inference, a retailer with a few cameras (traditional IoT devices) throughout the store connected to a computer could utilize their system to get information on foot traffic in the store, interest in particular items, and even the size of the checkout line.
Dalmo: “The idea behind the Hack Day project was to create use cases that are not too specific, and at the same time allow us to extrapolate other use cases from it, facilitating others into applying our technology to their specific situations.”
The ClariCam team during the March Hack Day consisted of Dalmo Cirne, Trevor Masters, John Sloan, Sujit Poudel (Mobile/Edge Software Engineering), Amanda Coyle (People/HR), and Karen Herder (Data Strategy). Each team member contributed something essential to the finished product.
Trevor worked on finding the necessary components for the Raspberry Pi, along with Dalmo. He created a mobile application that interfaces with the IoT SDK (both server and client parts.) He also connected the Raspberry Pi's camera to the application, and 'glued' all the different software components so they functioned together. In addition, Trevor worked on hollowing out the metal ClariCam chassis and drilled and filed the openings.
Trevor: “I think the most unexpected thing I learned how to do during Hack Day was how to drill through and then file down metal to create the access ports we wanted. During the final push to get everything to work together, we also managed to overcome no less than 4 separate segmentation faults, so that was 'fun'.”
Sujit: “ There were about a million failed attempts during the build process. I guess you could say I learned what does not work more than what does work!”
Karen worked with Amanda on the design of a new camera body to hold the Raspberry Pi and engineering components. Her background in Photography informed her design process: 
Karen: “In a very tiny nutshell, my brain went back to when I was in college. I studied graphic design/art and whenever we created mockups for a project, we would use black foam core because it's lightweight and easy to cut with a precision knife.
I knew that the basics of a camera body would never change because it's a classic design solution: a box-like shape to contain your electronics. We just needed to measure the size of the components that would extend beyond the box: the shutter button, on/off button, burst mode button, LED screen at the back, and the lens in the front.
Amanda and I were able to cut out the pieces of the camera body from black foam core, one side at a time. We cut out little notches into each side so they would fit together like a jigsaw puzzle. Once we were satisfied with all of the pieces fitting together, we glued them and then inserted the electronics.
In the photo below, I think we were each taking turns to hold the camera body box “thingy” and envisioning where the shutter button and on/off button would fit best in relation to human hands holding the box.”
Amanda CoyleAmanda worked with Karen to create the camera body and worked with John to create output patterns for the results returned from the computer vision model.
Amanda: “On Hackday, I ended up using my high school woodshop skills to design and build the outer casing of the camera. Karen stopped over to see how it was going and offer support. We collaborated on the overall design and figured out what parts should remain fixed as opposed to open for the team to fit any hardware inside. It was really fun working with Karen and the rest of the team. As a non-tech employee, I was excited to be put to work at the engineers' desks and get to know their work better. Everyone was very collaborative and open to explain what they were working on, ask me questions, advise on how to solve challenges, and breakdown complex issues, so I would walk away with a better understanding of our product in general.”
Hack Days drive innovation at Clarifai, but just as important, Hack Days can be a fun and rewarding team-building experience for everyone involved.
Amanda: “When you can take your head out of your daily tasks and work on something totally different with a totally different team, it gives you a renewed excitement about your own work.”
Ready to do some hacking of your own? Build & deploy your own Clarifai app and visual models with our developer guide, and use us for free!
