With millions of reviews and photos uploaded a month, OpenTable needed to moderate the media content they posted, to not only ensure nothing explicit was posted on their site but also to accurately categorize that content and provide a consistent user experience. Below, we discuss how computer vision helped the restaurant booking platform become 15X more efficient in this operational effort.
Background
Looking for somewhere to dine tonight? Chances are you visited OpenTable. Founded in the late 1990s, the restaurant booking platform has become the go-to reservation site in the restaurant industry. Despite a few new competitors, OpenTable remains the most dominant player in that space, seating over 20 million diners a month at over 40,000 restaurants worldwide. OpenTable doesn’t just enable eaters to book a table though. It also helps them to find the perfect restaurant, letting them filter by factors such as the date, time, or price. More importantly, however, it allows real-life diners to review their experience and add their own images, therefore helping future patrons to make their restaurant decisions.
Problem
Every month, over 1 million diners leave reviews on OpenTable, using the company’s site to upload pictures of their meal and the restaurant they booked. Unfortunately, like many platforms, OpenTable struggled to moderate this user-generated content (UGC,) so nothing inappropriate or irrelevant was uploaded. With only one employee to review every single user-generated image, only 300 photos were moderated and curated per day. With the rise of social media, the company needed to incorporate visual content from sites like Instagram onto their website, to ensure their users got the most authentic reviews possible. Already struggling to handle the photos posted directly to their site, however, the company knew hiring more moderators would not be enough to handle this new influx of data. So what did they do? They turned to computer vision.
Solution
While Clarifai’s moderation model, and platform, has allowed businesses to filter out any explicit content from the millions of images posted to their site every day, OpenTable had a unique use-case. In addition to keeping the content appropriate and relevant, they also needed to categorize these images into a few major categories, like "pictures of the restaurant itself," "pictures of the food," "pictures of the wait-staff," "pictures of the guests," etc., so they could provide their users with a consistent browsing experience no matter which restaurants they were looking into. They started with our food model, a niche model that can recognize thousands of foods. That said, as robust as this model is at identifying different food items, like steak or chicken, in visual content, OpenTable needed to curate photos that encompassed the entire restaurant experience. The great thing about computer vision is it can be trained to recognize almost anything you want it to see. Custom training is valuable where you have a use-case that doesn’t neatly fall into the purview of any of pre-trained model built provided AI companies like Clarifai. For OpenTable, they needed a model to be trained with the diverse UGC they had so it would accurately recognize features relevant to restaurants as a whole.
One of the key points in the build vs. buy debate is what the responsibility of the client is on their own versus with an AI provider. In OpenTable’s case, all they needed to do was give us the data. Clarifai’s data strategy (DST) team took on training the entire model, so allowing the company to simply plug our application programming interface (API) endpoint into a user interface their small moderation team could use easily review the model’s predictions. After only 2-3 weeks of work on our end, the company had their very own model that could accurately recognize over 20 concepts relevant to the types of pictures a diner might take while at a restaurant.
Results
Thanks to computer vision API, OpenTable was able to increase their operational efficiency by 15X, going from reviewing 300 photos a day to 5,000. Their custom model ensures that any image uploaded to the site is actually relevant to the company’s goals and brand, to ensure their users can have both the UGC they rely on to make dining decisions and the high-quality customer experience they want.
