Substrate defect-detection and comparison
Various examples described herein include image-processing tasks of image data used for defect detection and comparison of features on substrates. At least one of a deep-convnet-based and a transformer-based backbone network (a common backbone) that is arranged to convert various types of raw-image data into features that can, in turn, be used by smaller neural networks to perform final calculations of specific tasks. The smaller neural networks perform, for example, final defect-detections, die-to-die image comparisons, anomaly detection, and customer-specific tasks in a fabrication facility. The common-backbone network can initially be trained using self-supervised learning based on the raw-image data, and then transfer learning can be used to train a final application of the task-specific networks. Other systems and methods are also disclosed.
1 . A system to provide comparison data in a fabrication facility, the system comprising:
a common-backbone network based on a machine-learning-based network, the common-backbone network including:
an input component to receive raw-image data;
a self-supervised training component to generate a machine-learning-based comparison labels from the raw-image data; and
an analysis engine to perform anomaly detection and classification of images received from a plurality of task-specific networks provided at edge nodes electronically coupled to the common-backbone network, each of the plurality of task-specific networks being coupled to at least one type of equipment, the analysis engine to convert the raw-image data into features that are to be transmitted to the plurality of task-specific networks to perform final calculations of specific tasks within the fabrication facility.
2 . The system of claim 1 , wherein the raw-image data include acquired images and reference images.
3 . The system of claim 2 , wherein the reference images include CAD-based reference images and reference-die images.
4 . The system of claim 1 , wherein the common-backbone network is further configured to extract image features and align and scale the image features to image data provided to an input data component.
5 . The system of claim 4 , wherein the image features are to be transmitted to the plurality of task-specific networks to make a determination of defects within the fabrication facility.
6 . The system of claim 1 , wherein the raw-image data can be used to train a backbone network without a requirement for labeling each component of the raw-image data.
7 . The system of claim 1 , wherein the analysis engine is further configured to compare images received from the plurality of task-specific networks with related ones of images in the machine-learning-based comparison labels generated from the raw-image data.
8 . The system of claim 1 , wherein the machine-learning-based network is based on at least one of a convolutional-neural network (convnet) and a transformer-based network.
9 . An image-processing system to categorize defects in a fabrication facility, the image-processing system comprising:
a common-backbone network based on a machine-learning-based network; and
a plurality of task-specific networks provided at edge nodes electronically coupled to the common-backbone network, each of the plurality of task-specific networks based on a machine-learning-based network that is separate from the common-backbone network, each of the plurality of task-specific networks configured to be trained for a specific task within the fabrication facility,
wherein the common-backbone network comprises:
an input component to receive raw-image data;
a self-supervised training component to produce a machine-learning-based comparison labels from the raw-image data; and
an analysis engine for performing anomaly detection and classification of images received from the plurality of task-specific networks, the analysis engine for converting the raw-image data into features that are to be transmitted to the plurality of task-specific networks for performing final calculations of specific tasks within the fabrication facility.
10 . The image-processing system of claim 9 , wherein the machine-learning-based comparison labels is based on at least one of a convolutional-neural network (convnet) and a transformer-based network.
11 . The image-processing system of claim 9 , wherein the analysis engine is further configured to compare images received from the plurality of task-specific networks with related ones of images in the machine-learning-based comparison labels generated from the raw-image data.
12 . The image-processing system of claim 9 , wherein each of the plurality of task-specific networks is configured to perform operations of defect detection and die-to-die comparisons based on the features received from the analysis engine of the common-backbone network.
13 . The image-processing system of claim 9 , wherein each of the plurality of task-specific networks are to be coupled to and collect data from at least one type of equipment including a metrology tool and an inspection tool.
14 . The image-processing system of claim 9 , wherein each of the plurality of task-specific networks is trained to ignore normal process variations for a particular process based on a pre-determined tolerance value.
15 . The image-processing system of claim 9 , wherein each of the plurality of task-specific networks are customized with task-specific recipes for a particular process.
16 . The image-processing system of claim 9 , wherein each of the machine-learning-based networks is based on at least one of a convolutional-neural network (convnet) and a transformer-based network.
17 . A method for comparing data in a fabrication facility, the method comprising:
receiving raw-image data as inputs to a common-backbone network, the common-backbone network based on a machine-learning-based network;
using self-supervised training for generating a machine-learning-based comparison labels from the raw-image data;
performing anomaly detection and classification of images received from a plurality of task-specific networks provided at edge nodes electronically coupled to the common-backbone network;
converting the raw-image data into features and transmitting the features to the plurality of task-specific networks; and
performing final calculations of specific tasks within the fabrication facility based on the features.
18 . The method of claim 17 , further comprising using transfer learning for training a final application for each of the plurality of task-specific networks.
19 . The method of claim 17 , further comprising:
extracting image features; and
aligning and scaling the image features to a common-coordinate system from the raw-image data within the common-backbone network.
20 . The method of claim 17 , further comprising comparing images received from the plurality of task-specific networks with related ones of images in the machine-learning-based comparison labels generated from the raw-image data.
21 . The method of claim 17 , further comprising receiving the images from at least one of the plurality of task-specific networks from at least one type of equipment selected from an equipment type including a metrology tool and an inspection tool.
22 . The method of claim 17 , wherein the machine-learning-based network is based on at least one of a convolutional-neural network (convnet) and a transformer-based network.
23 . The system of claim 1 , wherein each of the plurality of task-specific networks being separate from the common-backbone network, each of the plurality of task-specific networks configured to be trained for a specific task within the fabrication facility.
24 . The method of claim 17 , wherein each of the plurality of task-specific networks being separate from the common-backbone network, each of the plurality of task-specific networks configured to be trained for a specific task within the fabrication facility.