Bench Talk

Edge Impulse Fundamentals #1: Getting Data from the Real World to the Virtual World

(Source: putilov_denis - stock.adobe.com)

The workflow for developing solutions that machine learning (ML) models can be complex. The workflow can be broken into three major phases:

• data processing
• model training
• inferencing

At the heart of each phase is data. Data is the raw material that drives the development of ML models. That data can come from a wide variety of places, from enterprise systems to sensors that drive the Internet of Things (IoT), which will be the focus of this article. Without a large and accessible dataset to draw upon, it is impossible to train an ML model reliably. 

That data must first be collected, cleansed, organized, and cataloged before it can be used to train ML models. While it is possible to use pre-trained models, their drawback is that they are likely generic and may need to be refined to work well for your particular use case or operating environment. For example, an audio detection model may have difficulty picking out desired sounds if it is not trained with data collected from the actual location it will be fielded. This is because differences in ambient background noises could affect the model.

Compounding all this is that IoT endpoint devices such as microcontrollers and FPGAs are often highly constrained in terms of memory and processing horsepower. Being able to generate a model that can run on such constrained architectures is an additional challenge. Much progress has been made recently in getting neural networks to run low-power devices with the advent of TensorFlow Lite for Microcontrollers CMSIS-NN from Arm®. Still, collecting new datasets and training new models has not been for the faint of heart. Fortunately, various services are emerging to simplify the data processing, model training, and deployment process as simply as possible. One such service is Edge Impulse, a company based in San Jose, California.

Edge Impulse is a cloud-based service that, in a nutshell, allows developers to connect a variety of embedded platforms to get sensor data from the cloud, use that data to train a TinyML model, and send the model back to the IoT device for inferencing (Figure 1). It does so intuitively using a handful of well-designed tools and workflows.

Figure 1: Edge Impulse is a cloud-based tool that significantly reduces the workflow complexity for embedded systems developers to add machine learning (ML) technology to their products. (Source: Edge Impulse)

Over the next year, we will explore the various software components of Edge Impulse and how they can be leveraged to simplify the development of machine learning algorithms for use in embedded systems. This blog will look at developing a rudimentary model that leverages the accelerometers aboard an Arduino Nano 33 BLE Sense development board.

Getting data from a sensor to the cloud has become a relatively trivial matter thanks to the proliferation of wireless internet connectivity and simplified programming tools such as Application Programmer Interfaces (API). Edge Impulse leverages these advancements to make data ingestion a snap (Figure 2).

Figure 2: Edge Impulse provides multiple ways to bring training and test data into their environment, including .csv files of raw data. (Source: Edge Impulse)

Edge Impulse provides three tools to assist in this first stage of the ML pipeline.

• The first tool, the Serial Daemon, is used to onboard new devices, configure upload settings, and even serve as a proxy for devices that cannot connect to the internet.
   
• The second tool, the Data Forwarder, is used in conjunction with the serial daemon (Figure 3). Working in concert, they allow a sensor device that lacks built-in wireless connectivity to connect to Edge Impulse through an internet-connected desktop computer via a wired serial connection. Devices simply write their sensor data in a comma-separated or tab-delimited format to a serial port. The data is then collected, cryptographically signed, and transmitted to Edge Impulse’s ingestion service. With the proliferation of inexpensive Wi-Fi^®-enabled single-board computers, these tools make getting started with embedded machine learning not only easy but also cost-effective. The serial daemon and the data forwarder are accessible from a computer’s command-line interface (CLI). One drawback is that the data forwarder is rate-limited compared to programming an embedded device to use the API directly.

Figure 3: For development boards that lack native Internet connectivity, data can be sent to the Internet-connected host computer via USB and the host computer can send the data to Edge Impulse. (Source: Edge Impulse)

• The final tool is Uploader. This tool is available both via a CLI tool and the browser. It accepts .wav audio files and .jpg image files, and .cbor and .json structured data file formats that are useful for most other sensor data. This is a convenient way to allow developers to collect data however they see fit before ingesting the data set with a simple click of the mouse. Edge Impulse recommends this method as a way to migrate data between different Edge Impulse instances. A data acquisition format is prescribed for the .cbor and .json files can be found here.

Lastly, for a handful of embedded development boards with onboard sensors, Edge Impulse offers pre-built firmware that sends all sensor data to their ingestion service with minimal setup. The firmware leverages the tools mentioned above. Then in the browser, you can pick out which data you want to use to build your model. We will explore this further in a future edition of Mouser’s Edge Impulse Fundamentals series.