Implementing image enhancement for improved defect detection
A system and method for enhancing image quality. The system and method acquire a machine learning model trained for correlating one or more training images and one or more training design images. The system and method receive one or more sample specimen images corresponding to one or more features of a sample specimen. The system and method enhance the one or more sample specimen images by generating one or more enhanced images with the machine learning model based on at least the one or more sample specimen images.
1 . A characterization system comprising:
a controller communicatively coupled to a characterization sub-system, wherein the controller includes one or more processors configured to execute a set of program instructions maintained in memory, wherein the set of program instructions are configured to cause the one or more processors to:
acquire a machine learning model trained for correlating one or more training images and one or more training design images;
receive one or more sample specimen images corresponding to one or more features of a sample specimen;
enhance the one or more sample specimen images by generating one or more enhanced images with the machine learning model based on at least the one or more sample specimen images, wherein the one or more enhanced images have improved image quality relative to the one or more sample specimen images, the improved image quality comprising at least one of increased signal-to-noise ratio, increased resolution, or increased contrast; and
determine, based on the one or more enhanced images, one or more characteristics of the one or more features, wherein the one or more characteristics of the one or more features comprise a defect classification type of at least one of the one or more features, wherein the defect classification type comprises one of a Defect of Interest (DOI) or a nuisance defect.
2 . The characterization system of claim 1 , wherein the acquire the machine learning model trained for correlating the one or more training images and the one or more training design images comprises:
acquire, with the characterization sub-system, the one or more training images comprising one or more training features of one or more training specimens;
receive the one or more training design images corresponding to the one or more training features of the one or more training specimens;
aligning the one or more training images with the one or more training design images corresponding to the one or more training features of the one or more training specimens;
generating the machine learning model based on the one or more training images and the one or more training design images.
3 . The characterization system of claim 1 , wherein the acquire the machine learning model trained for correlating the one or more training images and the one or more training design images comprises:
receive the machine learning model, the machine learning model based on the one or more training images and the one or more training design images corresponding to and aligned with the one or more training images.
4 . The characterization system of claim 1 , wherein the characterization sub-system comprises at least one of an optical inspection sub-system or an optical imaging-based metrology sub-system.
5 . The characterization system of claim 1 , wherein the one or more characteristics of the one or more features comprise: a localization of at least one of the one or more features.
6 . The characterization system of claim 1 , wherein the one or more characteristics of the one or more features comprise: one or more context attributes of at least one of the one or more features.
7 . The characterization system of claim 1 , wherein the one or more characteristics of the one or more features comprise: a classification scheme of the one or more features.
8 . The characterization system of claim 1 , wherein the one or more characteristics of the one or more features comprise: a size measurement of at least one of the one or more features.
9 . The characterization system of claim 1 , wherein the set of program instructions are further configured to cause the one or more processors to adjust a process tool based on the one or more characteristics of the one or more features.
10 . The characterization system of claim 1 , wherein the set of program instructions are further configured to cause the one or more processors to re-identify one or more defects based on the one or more enhanced images.
11 . The characterization system of claim 1 , wherein the set of program instructions are further configured to cause the one or more processors to identify one or more defects based on the one or more enhanced images.
12 . The characterization system of claim 11 , wherein the enhance the one or more sample specimen images and the identify the one or more defects occur in real-time.
13 . The characterization system of claim 11 , wherein the one or more sample specimen images are recorded and the identifying one or more defects occurs during at least one of a playback of the recorded one or more sample specimen images or offline reprocessing of data.
14 . The characterization system of claim 1 , wherein the one or more features are one or more defects, and wherein the set of program instructions are further configured to cause the one or more processors to re-detect at least one of the one or more defects based on the one or more characteristics of the one or more features.
15 . The characterization system of claim 2 , wherein the aligning the one or more training images with the one or more training design images comprises aligning the one or more training images with the one or more training design images to a sub-pixel accuracy.
16 . The characterization system of claim 1 , wherein the training design images comprise at least one of rendered training design images rendered from design data of one or more training specimens or derived training design images derived from the design data of the one or more training specimens.
17 . The characterization system of claim 1 , wherein the machine learning model comprises a deep learning predictive model.
18 . The characterization system of claim 17 , wherein the deep learning predictive model is a Generative Adversarial Network (GAN).
19 . The characterization system of claim 1 , wherein the one or more training images have a first resolution and the one or more training design images have a second resolution that is higher than the first resolution.
20 . The characterization system of claim 19 , wherein the one or more enhanced images have a third resolution that is higher than the first resolution.
21 . The characterization system of claim 1 , wherein the one or more training features are two or more training defects, each training defect of the two or more training defects being from a particular Design Based Group (DBG) of two or more different DBGs.
22 . The characterization system of claim 2 , wherein the generate the machine learning model is further based on at least one of rotated or flipped versions of both the one or more training images and the one or more training design images.
23 . The characterization system of claim 1 , wherein the one or more features include one or more polygon features, wherein the set of program instructions are further configured to cause the one or more processors to determine whether at least one of the one or more polygon features is a failed polygon feature that failed due to a hotspot.
24 . A method for enhancing image quality comprising:
acquiring a machine learning model trained for correlating one or more training images and one or more training design images;
receiving one or more sample specimen images corresponding to one or more features of a sample specimen;
enhancing the one or more sample specimen images by generating one or more enhanced images with the machine learning model based on at least the one or more sample specimen images, wherein the one or more enhanced images have improved image quality relative to the one or more sample specimen images, the improved image quality comprising at least one of increased signal-to-noise ratio, increased resolution, or increased contrast; and
determining, based on the one or more enhanced images, one or more characteristics of the one or more features, wherein the one or more characteristics of the one or more features comprise a defect classification type of at least one of the one or more features, wherein the defect classification type comprises one of a Defect of Interest (DOI) or a nuisance defect.
25 . The method of claim 24 , wherein the acquiring the machine learning model trained for correlating the one or more training images and the one or more training design images comprises:
acquiring the one or more training images comprising one or more training features of a one or more training specimens;
receiving the one or more training design images corresponding to the one or more training features of the one or more training specimens;
aligning the one or more training images with the one or more training design images corresponding to the one or more training features of the one or more training specimens;
generating the machine learning model based on the one or more training images and the one or more training design images.
26 . The method of claim 24 , wherein the acquiring the machine learning model trained for correlating the one or more training images and the one or more training design images comprises:
receiving the machine learning model, the machine learning model based on the one or more training images and the one or more training design images corresponding to and aligned with the one or more training images.
27 . The method of claim 24 , wherein the one or more characteristics of the one or more features comprise: a localization of at least one of the one or more features.
28 . The method of claim 24 , wherein the one or more characteristics of the one or more features comprise: one or more context attributes of at least one of the one or more features.
29 . The method of claim 24 , wherein the one or more characteristics of the one or more features comprise: a classification scheme of the one or more features.
30 . The method of claim 24 , wherein the one or more characteristics of the one or more features comprise: a size measurement of at least one of the one or more features.
31 . The method of claim 24 , further comprising adjusting a process tool based on the one or more characteristics of the one or more features.
32 . The method of claim 24 , further comprising re-identifying one or more defects based on the one or more enhanced images.
33 . The method of claim 24 , further comprising identifying one or more defects based on the one or more enhanced images.
34 . The method of claim 33 , wherein the enhancing the one or more sample specimen images and the identifying one or more defects occur in real-time.
35 . The method of claim 33 , wherein the one or more sample specimen images are recorded and the identifying one or more defects occurs during a playback of the recorded one or more sample specimen images.
36 . The method of claim 24 , wherein the one or more features are one or more defects, and wherein the method further comprises re-detecting at least one of the one or more defects based on the one or more characteristics of the one or more features.
37 . The method of claim 25 , wherein the aligning the one or more training images with the one or more training design images comprises aligning the one or more training images with the one or more training design images to a sub-pixel accuracy.
38 . The method of claim 24 , wherein the training design images comprises at least one of rendered training design images rendered from design data of one or more training specimens or derived training design images derived from the design data of the one or more training specimens.
39 . The method of claim 24 , wherein the machine learning model comprises a deep learning predictive model.
40 . The method of claim 39 , wherein the deep learning predictive model is a Generative Adversarial Network (GAN).
41 . The method of claim 24 , wherein the one or more training images have a first resolution and the one or more training design images have a second resolution that is higher than the first resolution.
42 . The method of claim 41 , wherein the one or more enhanced images have a third resolution that is higher than the first resolution.
43 . The method of claim 24 , wherein the one or more training features are two or more training defects, each training defect of the two or more training defects being from a particular Design Based Group (DBG) of two or more different DBGs.
44 . The method of claim 25 , wherein the generating a machine learning model is further based on at least one of rotated or flipped versions of both the one or more training images and the one or more training design images.
45 . The method of claim 24 , wherein the one or more features include one or more polygon features, wherein the method further comprises determining whether at least one of the one or more polygon features is a failed polygon feature that failed due to a hotspot.