Robotic machine vision robotic solutions for automated inspection and quality control in manufacturing floors

Overview

Today, any business that relies on producing items with precision and high quality could potentially benefit from machine vision as part of its manufacturing process. Robots powered with AI and machine vision could allow machines to “see” the products on the production line and spot any imperfections. Our client was in the business of designing robotic solution for the heavy industry. They wanted us to build a system that can measure the dimensions and detect defects on the manufactured artifacts. The robots attached with camera should be able to determine where the problems are being introduced in a production line so corrective action can be taken. The design had to be an automated visual inspection tool to find even microscopic flaws in products. The system recognizes defects, marks them, and sends alerts. The attached AI system can alert human workers of the flaw before the item winds up in the hands of an unhappy consumer.

Challenges

The key goals of the project are to identify the object of interest on the camera feed and to measure the object. Since the object of interest is not a standard shape, conventional shape recognition methods like Hough Transforms are not suitable for object detection. So we started the data acquisition phase of the project by collecting the video recording from the camera feeds and trim the video segments pertaining to the object of interest. This was then followed by an object annotation phase with the CVAT tool to generate a segmentation dataset of 5000 images. We implemented a neural network training process derived from the CNN architectures including the Xception, Inception, and ResNet. From our RoC studies, it was observed that the ResNet model was performing adequately in terms of both accuracy and execution time. We then ported the ResNet inference program to an edge computing platform that interfaces along with the existing camera module. The edge computing platform read the camera feed and performed real-time object measurement and reported the measurements over a network connection.

Solution

The key goals of the project are to identify the object of interest on the camera feed and to measure the object. Since the object of interest is not a standard shape, conventional shape recognition methods like Hough Transforms are not suitable for object detection. So we started the data acquisition phase of the project by collecting the video recording from the camera feeds and trim the video segments pertaining to the object of interest. This was then followed by an object annotation phase with the CVAT tool to generate a segmentation dataset of 5000 images. We implemented a neural network training process derived from the CNN architectures including the Xception, Inception, and ResNet. From our RoC studies, it was observed that the ResNet model was performing adequately in terms of both accuracy and execution time. We then ported the ResNet inference program to an edge computing platform that interfaces along with the existing camera module. The edge computing platform read the camera feed and performed real-time object measurement and reported the measurements over a network connection.

Impact

We were able to equip their robots with a novel object measurement and fault detection process. The robotic system was able to detect manufacturing defects, thereby eliminating wastes due to defective product discards. The detection model was 90 % accurate and it reduced their material wastage by significant a 35%. The accuracy in the detection model improved their product quality, reduced errors and spared their workforce from reptitive tasks.

Technology stack