MMOOZZGG.com

Glossary

What is edge AI

In short

Edge AI means neural networks that run directly on a device or close to it, such as a camera, a machine or a phone, instead of in a remote cloud.

Infrastructure · edge AI

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine a robot vacuum cleaner that notices a sock on the floor and drives around it. It has no time to call someone and ask what to do: while it waits for an answer, it will already run over the sock. So a small brain sits right inside it. Smart cameras and toys work the same way: they think for themselves and do not ask a big computer somewhere far away for help.

In plain words

Usually a neural network lives in a data center: a device sends a photo or a sound there and waits for an answer. With the edge approach, the model runs where the data appears: on a camera at a warehouse gate, in a machine controller, in a smartphone. The answer comes in a fraction of a second, no internet is needed, and photos and recordings go nowhere. The price is that a device is weaker than a server, so the models are small and compressed. For example, a camera on a conveyor tells a defective bottle from a good one by itself and gives a signal to remove it from the belt at once.

For business

Companies choose this approach when reaction speed matters, when the system must work without a connection, or when there is too much data to send to the cloud: video from dozens of cameras, machine telemetry, voice in cars. The money goes into devices and their maintenance, not into a bill for tokens, while traffic and cloud costs fall. The risks: you have to update the fleet of devices, track model versions and protect every device that people can physically reach. A small model is less accurate, so hard cases are often passed to a server. Before you deploy, check whether the hardware is powerful enough for your model and how you will update it across hundreds of sites; a pilot on one line answers this.

For a specialist

Edge AI, or edge intelligence, moves training and, above all, inference of models from the center of the network to its edge, closer to the data sources. Zhou and co-authors (2019), in a survey for the Proceedings of the IEEE, describe it as the meeting point of AI and edge computing, which moves computing tasks and services from the network core to its periphery, and review architectures for training and inference of deep models at the edge. Typical designs are fully local inference on the device, splitting a model between the device and the nearest node (model partitioning), and a cascade in which a light model answers by itself and sends uncertain cases to a server. Models are prepared with quantization, pruning and distillation, and run through runtimes such as LiteRT, built on TensorFlow Lite, with acceleration on CPU, GPU and NPU. The limits are the device’s memory, power use and heat, and the safe delivery of updates to the whole fleet.

Example

A chain of gas stations installs cameras near the checkouts that count the queue and spot unattended items by themselves. The video does not go to the cloud: only events and counters are sent to the server, so the network link is enough even at remote stations.

Related terms

Questions and answers

What does the term edge AI mean? Edge AI means that neural networks run on the device itself or close to it, not in the cloud. Smart cameras, voice features in phones and quality control systems in factories work this way.
How is edge AI different from cloud AI? A cloud model is more powerful, but it needs a connection and sends data to someone else’s server. A model on the device answers without network delay and keeps the data in place, but weak hardware limits it, so the two approaches are often combined.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us