Bench Talk

Edge Impulse Fundamentals 3: Processing Blocks

(Source: putilov_denis - stock.adobe.com)

Welcome back to our ongoing series on how developers can leverage the services of Edge Impulse to bring machine learning technology to embedded systems. As a reminder, Edge Impulse is a platform for building, deploying, and managing edge-device machine learning models. One of the key features of Edge Impulse is its impulse design functionality, which enables users to create custom machine-learning models for their specific use cases. In this third chapter, we will explore in detail one of the most crucial steps of the Edge Impulse workflow: impulse design.

But first, let’s define what exactly Edge Impulse means by the term “impulse.” In Edge Impulse, an impulse is a machine-learning model created and optimized for deployment on edge devices. The term "impulse" conveys the idea of a small, self-contained unit of intelligence that can be deployed on a device and run independently. An impulse in Edge Impulse consists of a pre-processing pipeline, a machine-learning model, and deployment code. The pre-processing pipeline includes a series of processing blocks that transform the raw input data into a format the machine learning model can use. This will be the focus of this entry.

In the previous chapter, we demonstrated how users could upload their own data sets from the real world and onto the Edge Impulse platform. Once raw data has been uploaded to Edge Impulse, refined, and tagged; a developer can begin the process of “impulse design” or defining their own custom machine learning pipeline. The first step is to tell Edge Impulse what type of data has been uploaded. The workflow gives users three preconfigured options, including accelerometer data, sounds, or images. It also provides a fourth generic option titled “Something Else” to handle other types of sensors, potentially anything from temperature to light sensors and anything in between.

In general, if the data is not an image, then data will likely be some time series data (Figure 1). In other words, one or more sensor values will change over time in response to environmental stimuli. It is essential to determine if the changes will occur fast (i.e., automotive impact detection) or slow (i.e., the temperature in certain manufacturing processes) over time, as this will be an important consideration later.

Figure 1: The first two tools of Impulse Design are 1) the Input Block and 2) the Processing Block. (Source: Green Shoe Garage)

Once the data type is selected, the developer will be presented with the ML pipeline and options to tweak the input block if required (Figure 2). The content of the input block will vary depending on the data type. For our purposes of exploring embedded systems, we will demonstrate accelerometer data. Per Edge Impulse’s user guide, the following options are available to be tweaked for time series data:

• Window Size: The window size is the size of the raw features used for the training.
• Window Increase: The window increase is used to artificially create more features and feed the learning block with more information.
• Frequency: The frequency is automatically calculated based on your training samples. You can modify this value, but you currently cannot use values lower than 0.000016 (less than 1 sample every 60s).
• Zero-Pad: A yes/no check box that adds zero values when raw features are missing.

Figure 2: How the settings of the Input Block can impact extracting features from the raw data. (Source: Edge Impulse)

The next decision of the workflow is the application of various “processing blocks.” The job of the processing block is to pre-process and extract features from the raw data that will be used to teach the model. Edge Impulse conveniently provides a library of predefined digital signal processing blocks for different applications. The most common of these include:

• Spectral Analysis: Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time.
• Flatten: Flatten an axis into a single value, useful for slow-moving averages like temperature data, in combination with other blocks.
• Image: Pre-process and normalize image data and reduce color depth optionally.           
• Audio (MFCC): Extracts features from audio signals using Mel Frequency Cepstral Coefficients, which is excellent for the human voice.
• Audio (MFE): Extracts a spectrogram from audio signals using Mel-filterbank energy features, great for non-voice audio.
• Spectrogram: Extracts a spectrogram from audio or sensor data, great for non-voice audio or data with continuous frequencies.
• Raw Data: Use data without pre-processing. Useful if you want to use deep learning to learn features.

Of course, you may have highly unique data or you may have custom digital signal processing algorithms that you wish to employ. In that case, Edge Impulse provides a mechanism by which end users can define custom processing blocks and use them for model training within the Edge Impulse studio workflow. If you want to learn more about employing custom-made processing blocks, please check this article (Figure 3).

Figure 3: Edge Impulse provides many types of Processing Blocks for different types of data such as images, sound, and accelerometers. (Source: Green Shoe Garage)

It should also be noted that processing blocks can be combined together in a pipeline to create a custom data preprocessing workflow. The output of the processing blocks is then fed into “learning blocks.” A learning block is a neural network trained to learn from the processed data outputted from the processing block workflow.

Learning blocks will be the topic of our next entry in this blog series.