Vintage Cloud is the fastest way to preserve, digitize and monetize your film assets. Based on Steenbeck’s legendary film transport, Vintage Cloud software’s “Smart Indexing” uses artificial intelligence and machine learning to dramatically increase the speed and precision with which metadata can be included within the film asset.
Introduction
The history of film goes back over 100 years – to the 1890s, in fact. Since that time, it’s estimated that more than half a million movies have been created. It wasn’t until 1925 that John Logie Baird gave his first demonstration of television – and it’s probably conservative to say that over 40 million hours of content have been created in the 90+ years since then.
But movies and TV aren’t the only sources of shows and films that were committed to film. Think of all the commercials, all the documentaries, all the in-house productions, all the public information footage that’s been created – and that’s just scratching the surface.
And here’s the thing – much of that material has been lost. But, huge amounts of it remain, in a wide variety of film archives around the world – both commercial and non-commercial. Those huge amounts of content have significant social and historical value. More than that: that content also has real monetary value.
Challenge
How can you understand millions of hours of video content without having to watch it all?
There are, however, difficulties in working with this archive material. The first challenge is that film degrades over time. It deteriorates if it hasn’t been stored in ideal conditions; it gets scratched through mishandling and sprocket holes start to fray through repeated playing. It’s valuable – but incredibly fragile. For many organizations, that means that capturing it in digital form has become a matter of urgency – before its physical condition means that it becomes inaccessible.
Rising to that first challenge is relatively straightforward. The second challenge is much more difficult: understanding exactly what content is included within the film reel. A documentary producer might be looking for footage of a steam train in the 1930s. How would they locate samples of the appropriate footage?
Vintage Cloud is a company that was created to address these challenges – and the opportunities that solving those challenges presents. It specializes in transferring old film archives into digital files. In itself, that’s a fairly well-understood process – so long as appropriate care is taken of the fragile source material.
What’s much harder than simply converting the content into computer-storable, computer-retrievable files is to make sense of what those files contain – so that their value can be preserved and monetized. What’s needed is ‘metadata’ – information about the content, in as much detail as possible. Unfortunately, with much archive material, that metadata just doesn’t exist. Where it does exist, it’s often in rudimentary, unstructured, written form that requires substantial editing.
“During our film digitization process, which is in real-time in 4K resolution, we extract keyframes every third second. We send these keyframes to Clarifai for indexing so that metadata is completed in parallel with digitization. This maximizes the efficiency of our workflow while enhancing it with extremely detailed AI results.” – Peter Englesson, CEO & Co-Founder Vintage Cloud
Solution
With Clarifai’s visual recognition API, Vintage Cloud was able to understand the full content of hours upon hours of video in a matter of minutes.
Vintage Cloud quickly realized that the secret to the successful transfer of archive material to the digital world lay not just in capturing the source – but making the extraction and creation of metadata from the source as simple, straightforward, and fast as the physical transfer. The company quickly discovered that the manual method of watching and documenting film was not scalable and would take entirely too much time and manpower.
A period of investigating the possible alternative approaches led Vintage Cloud to Clarifai and its artificially intelligent visual recognition system – which, in effect, can take the place of a human being viewing all the footage in its entirety while frantically taking notes.
“We found the tagged results amazing,” says Peter Englesson, CEO of Vintage Cloud. “The Clarifai computer vision recognition found objects that human eyes would never have found, especially not in real time. The results were stunning – even though much of our footage consists of “vintage” images, those items that you can’t find in today’s world. Take something like helmets, for example. Helmet design has changed radically over the last hundred years – but Clarifai AI is able to identify helmets from 1920 up to today. That’s astonishing.”
“Clarifai’s visual recognition AI can understand concepts with better-than-human accuracy in many cases. For example, the accuracy for “helmet” is fantastic even though the design of helmets has changed radically over time from the early 20th century until today.”
That ability to automatically detect objects that appear in the archived material – not only with the highest accuracy but also far faster than a human eye could ever achieve the feat – means that the creation of easily searchable metadata is transformed, as is the value of the archive asset.
Implementation
Quick and Easy Implementation
With a team of two talented developers, Vintage Cloud built a platform that automatically creates a keyframe every third second, while digitizing old film content, and sends the image to the Clarifai API – returning it, fully indexed, in real time.
“Thanks to my background as a film editor, I recognise the power of keyframes,” says Englesson. “It allows you to overview a massive amount of film footage without the need to view hours of video.”
Clarifai AI produces an enormous amount of tags while indexing, which gives Vintage Cloud the flexibility to narrow down results into ones that are relevant to a particular subject. The Vintage Cloud “Smart Indexing” platform has built-in efficient “search” functionality to filter out, for the time being, tags that are not needed. For example, one could search only for race cars in films and exclude all other vehicles like buses and trucks.
“This function makes it very easy for our clients to search throughout their whole catalogue for items or tools,” continues Englesson. “For example: if the client searches for ‘vintage car’, they’ll find all the instances within the archive where cars built between 1920 and, say, 1970 can be found.”
“The Clarifai API was very easy to use – our developers were able to implement Clarifai’s tools very quickly. On top of that, Clarifai’s support has been excellent – they worked with us to improve the quality of our visual recognition results for our unique use case.”
Vintage Cloud’s Smart Indexing capability is an integral part of its Steenbeck Digitizer – but it also works as a standalone application for material that has already been digitized. The fact that Vintage Cloud is a leader in helping owners of film archives derive the true value of their assets is due in no small part to the remarkable technology that it leverages from Clarifai.
DIY with Clarifai
Now that you’ve been inspired by Vintage Cloud’s automated video metadata solution, it’s time to build your own. Clarifai’s core model includes tags for over 11,000 concepts you can apply to your business. Or, you can use Clarifai’s Custom Training solution to teach our AI new concepts. If you want more personalized insight into how Clarifai’s technology can optimize your unique business, let us know at sales@clarifai.com and work directly with our Data Strategy Team and machine learning experts!
