In recent years, image recognition algorithms have made headlines around the world. The field of deep learning as a whole took off in 2012 when researchers from the University of Toronto demonstrated for the first time that deep neural networks could be used to classify images with very high accuracy. However, it’s easy to look at terms like “deep learning” and “convolutional neural network” and have no idea what they mean or how to use them in your own projects.
Fortunately, Clarifai provides an API that enables just this. In this tutorial, you'll learn how to create a model that recognizes different hackathon stickers in the span of a few minutes, so that you’ll be prepared to use Clarifai at your next hackathon!
Don't have a Clarifai account yet? Sign up for free API access!
Set up your development environment
In this tutorial, we’ll be creating a web app with Node.js. To get started, create a new folder and initialize your project. If you don’t have Node.js installed, make sure to download it here. It will install the npm command as well:
mkdir clarifai-example
cd clarifai-example
npm init
The npm init command will prompt you to answer a few questions; you may hit enter on all of them to accept the default values. Now that our project is set up, we’re going to install a couple of packages:
npm install clarifai express multer pug
This command installs four packages, all of which we’re going to use in this tutorial:
- Clarifai is Clarifai’s JavaScript client library.
- Express is a popular web framework for Node.js.
- Multer handles file uploads for Express web servers.
- Pug is a template engine for Node.js.
Create your frontend app
To demonstrate Clarifai’s image recognition API, we’re going to create an app where you can upload photos and have them classified on the spot. Begin by creating a HTML file, index.html, and paste the code written below.
[0.html]
Once this is done, create a folder called public. This is where we can keep static files, such as CSS and some JavaScript files. Inside public, create another folder called css. Inside css, create a file called style.css. Your file structure should now look like mine does in the picture to the right. After you finish doing this, paste the CSS code written here into style.css:
[1.css]
Now, we’re going to start building the meat and bones of our web app. Create a file called index.js in the root directory of the project (the same one as index.html), and paste the code written below.
[2.js]
Now, try typing node index.js into your terminal. If you open http://localhost:3000 in your browser, you should see the page we just created with the HTML and CSS files above. It doesn’t do anything yet, but we’ll get to that very soon!
To stop running your web server, you can press CTRL-C on your keyboard. You’ll need to do this every time you update your JavaScript code. If you want your web server to automatically refresh when you update your code, you may want to use something like nodemon. Now that we’re done making the frontend app, let’s get started with Clarifai!
Build your model with Clarifai
To get started with Clarifai, you’ll need to create an account and verify your email address. Once you’re done setting up your account, you should see a screen where you can manage your applications. For this project, we’re going to create a new application. I called mine “hackathon-stickers,” and kept the rest of the default settings.
Once you’re done with this, click “Create App,” and then click on the app that you just created. This is where you can configure the details of your application. Click on “Explorer” to go to the model explorer, where we can start training the model itself.
Then, create the model. I called mine “stickers,” but if you change this, you’ll need to remember it for later.
Now, I’m going to upload the photos I’ve taken of a few of the stickers I’ve kept from MIT’s hackathon and career fair. Feel free to try this with photos you take of your own stickers, but if you’d like to use mine, you can download them here.
Once all of your photos have finished uploading, assign “concepts” to all of your photos. Concepts are essentially labels; Clarifai’s algorithms use them to understand what is actually present in each picture. Below, on the top right, you can see an example of one I’ve labeled as “yelp.”
Once you’re done labeling all of your pictures, click on your model on the top left of the browser window.
Then, train your model by clicking the “Train Model” button shown in the picture below.
It will take a second, but once it’s done, you should see the status on the right side of your screen change to “Model trained successfully.” Now, we’re ready to finish up our web app.
Integrate Clarifai into your app
Start by going back to the index.js file. We’re going to add the code for the other dependencies that we installed earlier. Update your index.js file so it looks like the one below.
[3.js]
In the code above, you’ll notice that I left a placeholder for your API key. In order to find this, click on your name at the top right of your browser window, and click on “Settings.” Click on “API Keys” on the left side of your window, and copy the key that is displayed under “hackathon-stickers-all-scopes.” This should be pasted in place of that placeholder.
On the line with clarifai.models.predict, you’ll also notice that the name of the model is specified. Earlier, we created a model named “stickers,” but if you named your model something else, this is where you should put your model’s name.
The code above also references a results.pug template file, so you should create this file in the same directory as index.js. Your file structure should now look like mine does on the right. The template describes the webpage that our results will be shown on, and is later translated into HTML. If you’d like to learn more about how the template file works, check out Pug here.
[4.pug]
Finally, add some CSS to the bottom of style.css so that our results page looks alright.
[5.css]
Test your model
We now have a working model, and a web app that allows us to try it out. Type node index.js in your terminal, hit enter, and open http://localhost:3000 in your browser. Upload an image, click submit, and you’ll see what your custom model thinks it’s looking at.
You’ll notice that the words on the left are the concepts you created earlier, and each one has a number assigned to it. These are probabilities (from 0 to 1), and they correspond with Clarifai’s confidence of each model being present in the picture. Values close to 1 indicate high confidence, and values close to 0 indicate that the concept is almost definitely not present in the picture.
Next steps
Now of course, you probably won’t be identifying pictures of stickers at your next hackathon. The uses of image recognition are far-reaching and allow for endless possibilities.
Another concept to keep in mind is that your model is only as strong as its training examples. If you look closely at the photos I used to train this model, you’ll notice that all of my training examples were taken at my desk. As a result, my model will be very good at classifying pictures of stickers on wooden backgrounds. However, if I were to take a picture of one of these stickers on a different background, at an angle, or with a different orientation, the model may still be able to classify it correctly, but its confidence will be significantly lower. Try this out for yourself!
Congratulations, you now have a working custom image recognition model! The code we’ve written will work the same way with other concepts and models, so feel free to try this out with other types of images. If you had any issues putting together the code written in this tutorial, you can download the resulting codebase here.
Good luck at your next hackathon!
