Bench Talk

Manufacturing and industrial processes are undergoing immense change. Initiatives such as the Industrial Internet of Things (IIoT) promise to deliver significant operational efficiency improvements. As a result, sensors that enable IIoT applications to provide sense and insight into what is happening in the real world of production processes, industrial automation, and quality control are being deployed at an unprecedented rate. While there are many metrics that environmental, fluid flow, and pressure sensors can provide, one of the most important human senses that IIoT systems need to fully manage as a process is sight. Provisioning computer sight, more commonly termed computer or machine vision, relies on fast image and video processing techniques coupled with artificial neural network platforms.

By introducing the capability of sight into any industrial manufacturing or assembly process, the potential applications for machine vision become almost limitless. Consequently, machine-vision systems can be found across the industrial landscape, catering to an extensive range of requirements. For example, machine vision can detect if a bottle of shower gel hasn’t been adequately filled, or perhaps the label hasn’t been applied straight or in the correct position. It can also instigate an actuator pushing a bottle into a reject bin if the top has not been applied correctly, or if the bottle appears cracked, broken, or deformed. Another example might be the complex automated assembly of mechanical parts by an industrial robot. The machine-vision tasks might include confirming the parts are correctly aligned for assembly, and then checking that the parts have been securely connected before the next step in the process takes place.

When it comes to implementing a machine-vision application, there are several factors to consider. First, the development team must determine whether their system requirements can be addressed using straightforward image processing techniques or whether it is a more complex task to which a deep-learning neural network is better suited.

• Simple image processing techniques can include edge-detection algorithms, thresholding techniques, and the use of low, band, or high-pass filters on an image captured by a camera. This approach has the benefit of requiring only low-to-moderate compute resources, meaning that production throughput is not compromised. The listed techniques are useful in many manufacturing and process automation scenarios. For example, consider the task of checking whether an industrial robot attached a cap onto a bottle. Machine vision can perform this task using an edge-detection algorithm with a high-pass filter that will display dark pixels if a bottle cap is missing. Thresholding separates colors from a background, so that, for example, pills within blister packaging can be identified and counted. Also, a machine-vision system could use a similar method to determine whether each pill is the correct size during manufacturing.

• If the machine-vision task is more complex, for example, reading the part number of a product, developers could implement an artificial neural network to infer the text characters and numbers. The design effort then becomes more involved, necessitating training a neural network model to rapidly, reliably, and correctly identify letters and numbers.

Perhaps the most significant consideration is the image-processing speed and compute-task latency dictated by the production-line process speed. To ensure flexibility of design and implementation, a machine-vision platform should also accommodate different image and video protocols as well as frame rates in order to make the platform adaptable and scalable for a wide variety of applications.

As the application examples above illustrated, the machine-vision compute workload can vary considerably. Most high-end microprocessors are perfect for computationally intensive tasks; however, field-programmable gate arrays (FPGAs) are particularly well-suited to implementing high-data rate, deterministic parallel processing techniques of image and video streams. Likewise, they are ideal for use with neural-network algorithms such as convolutional neural networks, which serve to emulate the human brain in inferring the result of an image with a high degree of accuracy.

For the development team tasked with implementing a machine-vision system for a wide range of industrial use cases—whether using image processing or neural-network techniques—the availability of a flexible prototyping platform on which to base their design is key.

The Microsemi/Microchip PolarFire FPGA Video and Imaging Kit is a comprehensive, high-performance evaluation platform on which to prototype and test machine-vision applications. With dual camera sensors, an extensive display interface, and peripheral I/O options, the kit is capable of 4k image processing and supports protocols such as HDMI 2.0, DSI, MIPI CSI-2 TX, MIPI CSI-2 RX, and HD/3G SDI. The PolarFire FPGA has 300k logic elements, 4GB DDR memory, and 1GB flash memory for buffering (Figure 1).

Figure 1: Image of the Microsemi/Microchip PolarFire FPGA Video and Imaging Kit showing the plug-in camera board. (Source Microchip)

The kit includes a reference design demonstration application that showcases the use of picture-in-picture capability, video stitching, and image-panning features. Disparity maps provide image-depth estimation. The kit’s software features an edge-detection algorithm IP. This edge-detection algorithm IP is based on a Sobel filter that allows the extraction of object edges to detect features in an image.

Machine vision is a key component of any industrial automation process. Using an FPGA-based development platform specifically designed for machine-vision applications, such as the Microsemi/Microchip PolarFire FPGA Video and Imaging Kit, helps speed development and shorten deployment time.