ISSUE

STMicroelectronics is a global leader in the production of semiconductors used in various sectors such as automotive, industry, Internet of Things and communication infrastructure. The manufacturing process requires highly technological facilities, such as automated clean rooms, to prevent contamination of electronic chips. To guarantee the quality of the products, the human presence in these factories is limited to the maximum.

With growing demand, it becomes necessary to automate production quality control. In particular, STMicroelectronics needs to detect potential manufacturing defects as early as possible. After the initial phase of wafer production, the silicon plates are cut and mounted in boxes. Defects can then appear and are critical because they can impact the operation of the chips, or reduce the life of the products.

STMicroelectronics uses high precision microscopes to capture images that are very high resolution (50 microns) and very large size (6100×6100 pixels), showing tiny defects of a few pixels, image reduction is impossible, and the amount of images available is limited.

The company set out to develop artificial intelligence (AI) to detect these defects, but several problems emerged. The considerable size of the images causes memory problems when training AI. In addition, creating a training data set is laborious due to the large number of small defects to annotate on each image. Finally, the limited amount of data available poses a challenge to create a powerful and robust AI.

CUSTOMER

STMicroelectronics, Crolles (FRANCE) 

SOLUTION PROVIDED

To address data challenges, Neovision proposed one method to automate data annotation and another to artificially generate new data, which significantly increased the number of training data.

To deal with memory problems, Neovision proposed tools and methods to train AI on very large images without loss of precision.

Finally, to optimize overall performance, Neovision proposed a method to find the best parameters of the neural network used to detect defects.

CUSTOMER BENEFIT

This collaboration has led to the development of a robust and efficient defect detection AI. Neovision’s expertise has enabled STMicroelectronics to integrate state-of-the-art tools, methods and algorithms, which has helped improve the know-how of its teams and the performance of its quality control process.

This collaboration has made it possible to automate the detection of certain critical defects. This results in an improvement in the quality of STMicroelectronics products and an increase in the efficiency of its production.

REALIZATION

The use of a semantic image segmentation model played a key role in this project. It first allowed the automation of data annotation.

The model was selected for its ability to work with a limited amount of data. Pre-trained on a wide variety of objects, this model is generic and is not optimized to work on silicon plate defects. Neovision has therefore optimized this model to work on STMicroelectronics defects using fine tuning. In this way, it was possible to automatically annotate defects.

Manual annotation by bounding box of defects

To compensate for the lack of training images, a Generative AI has been developed. It consists of a method of artificial data simulation where we created images automatically annotated starting from a flawless image and adding random defects with varying density, size and intensity. This increased the size of the training base tenfold.

Example of small simulated defects

For small object management, the method used is to zoom in on the object of interest instead of sending the entire image to the network. The image is cut into small, high-resolution patches, and the training is done on those fragments. The inferences are then merged.

Separation of defects into cropped sub-images

Finally, the semantic segmentation algorithm was trained on all real and simulated data. A suitable cost function for segmentation was chosen and stochastic optimization methods were used to find the best parameters of the model.

Results of the detection algorithms

COMBINED TECHNOLOGIES AND EXPERTISE

Data generation, Generative AI, Image segmentation, Deep learning, Fine tuning

TESTIMONIAL

« Neovision’s contribution has been significant in the progress of projects on several aspects.

Neovision’s knowledge of the field and technology watch enabled us to test innovative solutions that have already led to the desired optimization.

Neovision’s expertise enabled us to broaden the scope of our investigations with advice tailored to the desired level. Last but not least, our ability to meet delivery deadlines and the notion of sharing and collaboration are two points that are much appreciated. »

Loïc Rolland, Manufacturing Data & Analytics,
Technology Solutions