5,500 Years of Image Search

One might begin a history of information technology with pictures. Pictographs - in the form of little drawings and tally marks in clay - appear to be the earliest system of information storage and retrieval on record. It seems that 5,500 years ago, if you wanted to search for information related to a boot or a hand, you went looking for a picture of these things on your “mobile” clay tablets.

Even later forms of writing, ones that developed handy abstractions like letters, spelling and grammar, continued to be pictographic in nature. The Egyptian hieroglyphs are a famous example of this image/text mashup, and the modern logograms of Chinese and Japanese exhibit a strong connection to their original pictographic roots.

Children learn to recognize images of objects long before they learn to read, and people are more likely to remember pictures and images than words.

We have a strong affinity for image-based data storage and retrieval, but it has not been until the modern era of computer vision AI that software has been able to serve this fundamental aspect of human nature in a direct way - with search by image, or “reverse image search”.

(Possibly the earliest known writing. Surely the creator self-consciously pointed out that he or she was a “terrible artist”.)

“It’s like Shazam for stuff”

Companies like Shazam and SoundHound have used audio to solve important problems for the music industry.

It is difficult to describe music in non-musical terms. By enabling users to search for a song based on the actual sound of the song in question, these companies have been able to solve this difficult problem in a way that is intuitive for the end user. Before such services, the best we could do would be to “hum the melody” to someone who knew the song, or search by lyrics.

In a similar way, search by image enables customers to draw a direct line from the source of an idea (an image or video) to the information that they need - no explanation required.

This ability to “connect the dots” for customers is driving the adoption of image-based search technologies across a broad range of new products and services. These new technologies personalize customer experiences and reduce barriers to decision making. Enterprises can move beyond simply meeting customer needs, by delivering product offerings that are relevant and discoverable.

Considerations when integrating with enterprise. Most data is visual data.

Video content alone is projected to account for 82% of internet traffic by 2021. Most people carry a camera in their pocket at all times, and many modern phones have three or more dedicated camera lenses. The importance of visual data simply cannot be understated. Search by image can deliver ease and accessibility in the face of an overwhelming volume of unstructured visual information. 

A user who has gotten used to search by image, will find that it can be much more convenient and produce better results than are possible with conventional search. Digital asset management (DAM), stock photography and e-commerce are just a few examples of use cases that have already driven widespread adoption of these techniques.

But this search modality will be new to many, and thoughtful user interaction and interface design should be a priority when designing search integration with your products and services. Design for consistency to eliminate confusion and get your customers up-and-running in the shortest time possible.

Clarifai has made it easy for your development team to integrate image search into your products and services through the Clarifai API. You can find out more in our API documentation, or even follow this step-by-step QuickStart to get up and running in no time.

There are many cases where you need access to visual data without direct integration. Clarifai makes it simple for everyone to search visual information through the Clarifai Portal. Image data can be uploaded, managed and searched directly in Portal with no code required.