New
Introducing Compute Orchestration
December 30, 2021

Get to know the technology behind edge AI

Table of Contents:

Why Edge AI?

If you've ever attempted to automate business processes, reduce risk by enforcing regulatory compliance, or ensure physical safety and security in the workplace, you may have run into repetitive tasks that are expensive to scale by using a human workforce. However, when you looked into using AI, you discovered that you'd need a fast internet connection to make the system work.

What if you could automatically monitor video streams in real-time? What if you could do this while keeping your video data private? What if you could do all this without even having an internet connection?

A variety of new AI use cases are now feasible with the internet-independent, low-latency performance of edge AI technology, and it is worth looking at the ways that edge AI is ready for prime time. With AI on edge devices, companies can detect and alert workers to hazards, identify suspicious behavior, and monitor access to remote locations. They can provide more innovative, more secure in-store experiences. They can ensure that employees follow established business processes, departmental standards, and industry regulations.

Multiple technologies are converging

Edge AI technology is already in high demand, but widespread adoption across the enterprise and public sector has been slow until now. Critical pieces of the technology stack are maturing simultaneously and will accelerate edge AI adoption in the coming years. One of the most exciting developments is the creation of dedicated AI platforms, which makes it easier for non-experts to create and use models.

AI Development Platforms: 

Deploy edge AI without significant in-house technical expertise

AI-powered software - sometimes referred to as Software 2.0 - requires special development tools and support infrastructure. Platforms that can prepare training datasets, train models, and make them easy to deploy into production help you move fast and develop solutions effectively. 

 

Build accurate models with data-centric development: It is not an overstatement to say that Software 2.0 is “made out of data.” For AI models to perform well, they need to learn from data similar to the data they will encounter in the real world. AI development platforms allow users to deploy data-centric software that actively learns from real production data. Development platforms make it possible to deploy, improve and iterate on AI models over time.

dandelion-flower-web

Efficiently design AI solutions with autoML: Automating the process of algorithm selection, hyperparameter tuning, iterative modeling, and model evaluation greatly reduce the complexity of developing AI models and deploying them into production. These services, along with helpful tools for data visualization, model intelligibility, and model deployment are offered by modern AI development platforms. 

 

Achieve high performance with state-of-the-art algorithms: AI models can demand large amounts of memory and be computationally intensive. Model quantization, layer pruning, and other compression techniques make it possible to shrink these models down to fit onto smaller devices with limited memory. New algorithms like MobileNet, YOLO, and NanoNet produce highly accurate models with minimal memory and processor speed demands. AI development platforms make it possible for everyone to shop for the best viable model for their use case. 

 

AI Computational Hardware

 

Push AI computation to low-power devices: AI systems are compute-intensive. The AI lifecycle requires long-running training jobs, hyperparameter searches, inference jobs, and other costly computations. They also require massive amounts of data transfer for unstructured data sources like video. Specialized hardware is required to operate effectively, especially large-scale AI systems.

computer-chip-internal-mother-board

Processors: Machine learning and deep learning algorithms require extensive linear algebra, matrix, and vector data operations. Traditional processor architectures are not optimized for such workloads. The growing adoption of Edge AI has led to the development of specialized devices and chip architectures capable of performing AI inferencing efficiently.

 

Edge data centers: Edge data centers are small data centers located at the edge of the network, where they are closer to users and devices. Having compute power at the edge enables faster performance and lower latency because organizations don’t have to move the data to far-flung data centers to process it.  For example, by keeping data and compute power at the edge, a factory that manufactures car seats can use visual analytics algorithms to detect defects in real-time.

red bright lights in a tunnel

Wireless technologies: There is a limit to how much information radio waves can carry. 5G adds more capacity, more ”space” to use, which means that there's more capacity for wireless data transfer. 5G provides more capacity for social media, video streaming, and other bandwidth-intensive technologies. 

exhaust fan with propellers

Cameras

 

Capture unstructured data for actionable intelligence: Cameras of all kinds can serve as a computer’s “eyes” and “ears.” Video, still images, and audio can be a rich sources of information about the world. AI technologies can turn an unstructured stream of images, sound, and video into solutions that have never been possible before. 

 

Smartphones: Smartphones are everywhere and they function as cameras and minicomputers. Smartphone operating systems come with advanced computing capabilities, run applications, and enable devices to perform a variety of advanced functionality. Smartphones are a natural platform for edge AI computing, or simply as a camera for capturing data on the edge. 

cell phone on a table next to ipad

Specialty cameras: High-quality, inexpensive cameras are everywhere and can be augmented with AI capabilities. Cameras are available with a wide range of form factors and technical specifications that can meet specific business needs.  A variety of spectral capabilities are available through specialty cameras such as EO/IR, SAR, and WAMI. These cameras see in ways that people can’t.

video camera on building

CCTV: Edge AI can be integrated directly into camera hardware or deployed to a closed network that manages multiple cameras. Users can process video in real-time without having to send streams to a remote server for processing. The CCTV market is very active, with projected growth of over 100% over the next five years.

drones over a beach

Drones: Drones offer an incredibly flexible solution for gathering video data. They can be deployed quickly and can get to all kinds of hard-to-reach locations. Drones are lightweight,  battery-powered, and have limited-to-no internet capabilities. Edge AI will allow drones to process their video feeds and deliver powerful insights from the field.