Traditional methods of defect detection faced a number of challenges that reduced the quality of the process. Applying deep learning algorithms to captured video information increases the speed and accuracy of identifying objects that do not meet a predefined standard. Though deep learning is a relatively new solution for defect detection, it can expand the scope of the solution from simple detection of a defect to classification of the type of defect. Training deep learning networks to identify types of defects makes it possible to automatically route objects based upon their severity—such as the size of the flaw. In this example of the Intel® OpenVINO™ toolkit, we will look at a simple example of how video images can be used to determine whether an object is defective based upon its surface area.
In prior blog posts, we’ve seen examples of face and vehicle detection using images captured by a video camera. In this application, we’ll look at a different type of detection using deep learning to identify an object on a conveyor belt, measure its surface area, and check for defects.
Figure 1 shows the Object Size Detection pipeline. Let’s explore this pipeline and the activities that occur.
Figure 1: The Object Size Detection Pipeline diagram illustrates how this application of the OpenVINO™ toolkit processes an image to determine whether an object has a defect based on its surface area. (Source: Author)
This image processing application uses images captured by a video camera mounted above a conveyor belt. A Convolutional Neural Network (CNN)—a type of image processing deep neural network—processes the captured images to determine if an object is present. First, the CNN identifies whether an object is in the capture frame. If an object is present, the CNN draws a bounding box and calculates the area that object occupies. Then, this area is checked against the predefined acceptable constraint. If the object is larger or smaller than expected, then a defect indication is communicated.
Figure 2 shows the output of the Object Size Detection application of the OpenVINO™ toolkit. Note that in this example, the CNN found the object and bounded it in order to calculate its area.
Figure 2: The Object Size Detector output screen shows an example of the calculated area of a detected object. (Source: Intel)
The sample application also illustrates the use of the Message Queue Telemetry Transport (MQTT) protocol, which communicates the zone information to an industrial data analytics system.
Defect inspection is a monotonous task and prone to error based upon the inspector. Using deep learning to inspect parts frees up people to do more useful and creative work while increasing the efficiency of defect classification. In this simple example, the area of the part is used to determine if a defect exists, but deep learning can be applied in more advanced models to inspect for various types of defects and classify them accordingly—for example, rework or salvage. When paired with capable hardware such as one based upon the 6th generation Intel® Core™ processor or Intel’s Neural Compute Stick 2 powered by the Intel Movidius™ X VPU, impressive inference speeds can be attained that enable real-time analytics.
Calculating the area of an object on a conveyor belt can be useful in a variety of environments. Take for an example the process of sorting fruits and vegetables. Traditional methods of sorting fruits and vegetables can lead to bruising. Therefore great care needs to be employed when handling these items. By adapting this example to the produce industry, fruits and vegetables could be inspected and routed based upon their size (area) and color. Deep learning can also expand on existing methods by looking at a greater number of features for grading.
M. Tim Jones is a veteran embedded firmware architect with over 30 years of architecture and development experience. Tim is the author of several books and many articles across the spectrum of software and firmware development. His engineering background ranges from the development of kernels for geosynchronous spacecraft to embedded systems architecture and protocol development.
