DIY With Clarifai: Building Your Own “Smart” User Generated Content Solution à la Yelp

Yelp is a site that crowdsources local business reviews. They rely on users to contribute not only first-hand (and sometimes brutally scathing) reviews, but also millions of images.

But, you know what they say – more content, more problems. The good news for sites and brands that rake in user generated content is they’re probably getting some great customer engagement. The bad news is figuring out what to do with all those images and videos.

There are two major challenges with user generated content:

1. HIGH VOLUMES:
How do you know what content you’re getting?
Yelp receives tens of thousands of user uploads every day – some of the images they receive have captions, but most come with no information or unreliable metadata attached. At such volumes, it’s nearly impossible to go manually through each image and categorize it appropriately.

2. VALUE EXTRACTION:
Once you know what the content is, what do you do with it?
Knowing is only half the battle (thanks, G.I. Joe!) – once you have an understanding of your content, you need to put your knowledge into action. For Yelp, that meant finding a way to organize and curate relevant and diverse photos to their users.

Yelp solved its UGC problem with artificial intelligence:

Yelp had the resources to build its own in-house deep learning classification system to address its UGC challenges. That’s a great option when you have a billion dollars and an army of data scientists – if you don’t, you can use the Clarifai API to do the same thing Yelp did for way, way less time and money.

The first thing Yelp had to do was solve for its lack of data around user uploaded images. They built a photo classifier that allowed them to “see” what was in each image and sort the images into buckets. To save money and lighten the load on their servers, they opt to do this by batch at the end of each day.

Once Yelp had their photo classification service, they were able to tackle the challenge of surfacing the right images at the right time. The first feature they improved was their business page “cover photos”. A business page on Yelp shows a set of cover photos that are recommended by Yelp’s user feedback. These cover photos typically lacked diversity – for example, all the cover photos would usually be related to a single class (e.g. food). After implementing deep learning, Yelp was able to ensure a diverse set of cover photos to help users get a more holistic view of the business.

The second feature Yelp added was tabbed browsing on business pages to make it easier for users to jump to the information they wanted to find. Before, users could only see an unsorted grid of all user uploaded photos. Now, users are able to browse different tabs representing different classes of images, making it super easy for users to find the content they are looking for.

DIY with Clarifai
Now that you’ve been inspired by Yelp’s “smart” user generated content application, it’s time to build your own. Clarifai’s core model includes tags for over 11,000 concepts you can apply to your business. All it takes is three simple lines of code - sign up for a developer API account to get started for free!

Technical Instructions for Developers:

Once you’ve signed up for a developer account, head over to Applications and make a new one.  Make sure you nab that Client ID and Client Secret:

Now, head over to https://github.com/clarifai/clarifai-nodejs. There, you’ll find our Node.js client, which makes this process even easier. To set up your environment, download the clarifai node.js file and stick it in your project.
Boo yah. You’re set up. Now head over to your Node project and just require the Clarifai client:

var Clarifai = require('./YOUR_PATH_HERE/clarifai_node.js');

Remember that Client ID and Client Secret you nabbed earlier? We’re gonna use those now. You can either paste them in this function directly, or save them in an environment variable.

Clarifai.initAPI('YOUR_CLIENT_ID', 'YOUR_CLIENT_SECRET');

Now for the fun part. You can easily tag an image with just 3 lines of code:

var imageURL = 'MY_IMAGE_URL';

var ourId = 'my great image'; // any string that identifies the image to your system

Clarifai.tagURL(imageURL, ourId, handler); // “handler” is your basic error handler function

You’re all set! Now you can easily make like Yelp and tag and sort your images to your heart’s desire. If you’d like to see a more in-depth example, check out clarifai_sample.js in the GitHub repo.