(Source: sdecoret/Shutterstock.com)
In a previous blog post, we explored strong arguments for moving artificial intelligence (AI) to the network’s edge. In this installment, we discuss which AI applications benefit this approach. As a starting point, reviewing reasons to implement AI at the network edge gives some strong hints. Check whether any of the following apply to the project:
Given these factors, what specific AI projects might be made easier by running the machine-learning (ML) models at the edge? Here, we will examine the benefits of moving AI and ML models such as virtual assistants, facial recognition, and real-time monitoring applications closer to the edge.
As it has so often, Apple set a trend with Siri’s launch in 2010. This paved the way for many other virtual assistants, most famously Amazon’s Alexa and the Google Assistant. Virtual assistants make science-fiction-style voice control into a reality and work as follows:
It is easy to see how moving the ML models to the edge enhances the experience: The voice assistant would be more responsive, wouldn’t need an internet connection, and voice control could be embedded. The application being called for might itself require a network connection, for example, music streaming services.
Facial recognition is one of AI’s fastest-growing applications. The technology is still evolving, and with a few issues along the way. In 2016, Amazon’s Rekognition was mired in controversy and accusations of racism. The system incorrectly identified 28 ethnic minority US Congressional members as known criminals after being trained on a set of 25,000 images.
In 2019, an early trial of facial recognition technology by the Metropolitan Police, the largest police force in the UK, showed the technology to be inaccurate 81 percent of the time. However, the latest facial-recognition systems are becoming far more accurate. Earlier this year, the Met announced it was adopting the technology to scan for known troublemakers at large events.
Many use cases calling for facial recognition need the technology to work in near real-time. As a result, applications rely on moving ML models to the edge of the network. The system adopted by the Met is based on NEC NeoFace® Watch, which is entirely standalone and works in real-time. NEC targets its technology at several other markets, including retail, corporate events, festivals and other mega-events, and transportation.
Heavy industry and mining rely on extremely large and expensive machinery. Companies can potentially lose millions if this machinery suffers an unplanned breakdown. For instance, many mining operations rely on huge high-power pumps that keep the workings free from water and pump the mined slurry to the processing plant. The whole operation comes to a halt if one of these pumps suffers a catastrophic failure. As a result, mining companies invest significant resources into AI systems designed to predict potential failures before they happen.
Currently, these systems are often based on transmitting data from Internet of Things (IoT) sensors attached to the equipment. This data is then processed at a central location, and any warning necessary is sent back to the appropriate operator. However, mines and construction sites can be tens of kilometers across, often in hostile terrain, so integrating the ML model directly into the edge device would simplify the whole process.
Moving AI to the network edge requires three things:
Let’s look at each of these requirements.
As already discussed, ML models often rely on large numbers of parallel operations. Bluntly, they need raw computing power. However, there is always a trade-off between computing power and the actual power drawn by the device. For ML models to move to the edge, devices that draw as little power as possible are required. This is even more true when the device needs to be embedded. Fortunately, a wide range of high-performance, low-power MCUs is now available.
The next thing needed is a suitable toolchain for running ML models on microcontrollers. The overwhelming majority of ML frameworks are designed to run on 64-bit Intel family CPUs or graphic processing units (GPUs). By contrast, all the suitable microcontrollers have a 32-bit reduced instruction-set architecture, such as the ARM® Cortex® series MCUs. However, ML frameworks such as TensorFlow Lite enable ML to run on such MCUs.
The final piece of the puzzle is a different paradigm for creating and running ML models. This can be summed up with the phrase “Model once, run anywhere.” Essentially, this means exactly what it suggests: Create your model, typically using a high-power ML-optimized machine, then use your toolchain to convert it into code that can run on any microcontroller. Unfortunately, this eliminates the ability to benefit from continual learning or reinforcement learning.
The following table captures some of the trade-offs made when ML models run at the edge (Table1). Hopefully, it offers some pointers that will help decide whether to move your next AI project to the edge.
Table 1: Trade-offs of Running ML Models at the Edge
Feature
In the data center
At the edge
Real-time
No
Yes
Continual learning
Embeddable
Network needed?
Reinforcement learning
Full range of models?
Moving ML models to the edge enable new use cases for AI, which promises to bring about an embeddable AI revolution. Developments in both MCU hardware and the tools required to run ML models on those MCU’s provide the basis for the expansion of such technology. To learn more about AI-capable MCUs, check out the Mouser TensorFlow Lite article.
Part of Mouser's EMEA team in Europe, Mark joined Mouser Electronics in July 2014 having previously held senior marketing roles at RS Components. Prior to RS, Mark spent 8 years at Texas Instruments in Applications Support and Technical Sales roles and holds a first class Honours Degree in Electronic Engineering from Coventry University.