System and method for defect detection using deep learning-based image segmentation
A system may be configured to receive training images of features on the sample. The system may be configured to generate a segmentation model based on the training images. The system may be configured to receive product images of product features and identify, using the segmentation model, features of interest based on the product images. The system may be configured to generate a segmentation model image including segmentation masks corresponding to the identified features of interest and adjust a size of the segmentation masks based on a size of the identified features of interest. The system may be configured to determine a difference-based attribute value for each identified feature of interest based on the adjusted size of the segmentation masks. The system may be configured to determine whether each of the identified features of interest are defects based on the difference-based attribute value and a predetermined threshold.
1 . A system for detecting defects on a sample, the system comprising:
a controller including one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to:
receive one or more product images of a plurality of product features on the sample;
identify, using a deep learning segmentation model, one or more features of interest of a plurality of product features on the sample based on the received one or more product images;
generate a segmentation model image of the sample, wherein the segmentation model image includes a respective segmentation mask corresponding to each identified feature of interest;
adjust a size of each respective segmentation mask based on a size of a corresponding identified feature of interest, such that an adjusted segmentation mask matches the size of the corresponding identified feature of interest;
determine a difference-based attribute value for each identified feature of interest on the sample based on the adjusted size of a corresponding segmentation mask; and
determine whether each identified feature of interest on the sample are defects based on comparing, for each identified feature of interest, the determined difference-based attribute value corresponding to the identified feature of interest to a predetermined threshold.
2 . The system of claim 1 , wherein the set of program instructions are further configured to cause the one or more processors to:
receive one or more training images of the plurality of features on the sample, wherein the plurality of features on the sample are identified on the received one or more training images; and
generate the deep learning segmentation model based on the received one or more training images of the plurality of features on the sample.
3 . The system of claim 2 , wherein the one or more training images include one or more labeled training images, wherein each feature of the plurality of features on the sample are labeled in the one or more labeled training images.
4 . The system of claim 1 , wherein the determine whether each of the identified one or more features of interest on the sample are defects based on the determined difference-based attribute value corresponding to each of the one or more identified features of interest on the sample and a predetermined threshold comprises:
comparing, during runtime, the determined difference-based attribute value to the predetermined threshold for each of the one or more identified features of interest.
5 . The system of claim 4 , wherein a defect instance is determined when the determined difference-based attribute value is greater than the predetermined threshold for each of the one or more identified features of interest.
6 . The system of claim 4 , wherein a non-defective instance is determined when the determined difference-based attribute value is less than the predetermined threshold for each of the one or more identified features of interest.
7 . The system of claim 1 , wherein the deep learning segmentation model is a supervised deep learning model.
8 . The system of claim 1 , wherein the difference-based attribute value includes at least one of:
a grey level value or a standard deviation value.
9 . The system of claim 1 , further comprising:
a characterization sub-system configured to acquire one or more images of the sample.
10 . The system of claim 9 , wherein the characterization sub-system includes a scanning electron microscopy sub-system.
11 . The system of claim 1 , wherein the predetermined threshold includes a user-defined threshold.
12 . A system for detecting defects on a sample, the system comprising:
a characterization sub-system configured to acquire one or more images of the sample; and
a controller communicatively coupled to the characterization sub-system, the controller includes one or more processors configured to execute a set of program instructions stored in memory, the set of program instructions configured to cause the one or more processors to:
receive one or more product images of a plurality of product features on the sample;
identify, using a deep learning segmentation model, one or more features of interest of a plurality of product features on the sample based on the received one or more product images;
generate a segmentation model image of the sample, wherein the segmentation model image includes a respective segmentation mask corresponding to each identified one or more feature of interest;
adjust a size of each respective segmentation mask based on a size of a corresponding identified feature of interest, such that an adjusted segmentation mask matches the size of the corresponding identified feature of interest;
determine a difference-based attribute value for each identified feature of interest on the sample based on the adjusted size of a corresponding segmentation mask; and
determine whether each identified feature of interest on the sample are defects based on comparing, for each identified feature of interest, the determined difference-based attribute value corresponding to the identified feature of interest to a predetermined threshold.
13 . The system of claim 12 , wherein the set of program instructions are further configured to cause the one or more processors to:
receive one or more training images of the plurality of features on the sample, wherein the plurality of features on the sample are identified on the received one or more training images; and
generate the deep learning segmentation model based on the received one or more training images of the plurality of features on the sample.
14 . The system of claim 13 , wherein the one or more training images include one or more labeled training images, wherein each feature of the plurality of features on the sample are labeled in the one or more labeled training images.
15 . The system of claim 12 , wherein the determine whether each of the identified one or more features of interest on the sample are defects based on the determined difference-based attribute value corresponding to each of the one or more identified features of interest on the sample and a predetermined threshold comprises:
comparing, during runtime, the determined difference-based attribute value to the predetermined threshold for each of the one or more identified features of interest.
16 . The system of claim 15 , wherein a defect instance is determined when the determined difference-based attribute value is greater than the predetermined threshold for each of the one or more identified features of interest.
17 . The system of claim 15 , wherein a non-defective instance is determined when the determined difference-based attribute value is less than the predetermined threshold for each of the one or more identified features of interest.
18 . The system of claim 12 , wherein the deep learning segmentation model is a supervised deep learning model.
19 . The system of claim 12 , wherein the difference-based attribute value includes at least one of:
a grey level value or a standard deviation value.
20 . The system of claim 12 , wherein the characterization sub-system includes a scanning electron microscopy sub-system.
21 . The system of claim 12 , wherein the predetermined threshold includes a user-defined threshold.
22 . A method comprising:
receiving one or more product images of a plurality of product features on a sample;
identifying, using a deep learning segmentation model, one or more features of interest of a plurality of product features on the sample based on the received one or more product images;
generating a segmentation model image of the sample, wherein the segmentation model image includes a respective segmentation mask corresponding to each identified one or more feature of interest;
adjusting a size of each respective segmentation mask based on a size of a corresponding identified feature of interest, such that an adjusted segmentation mask matches the size of the corresponding identified feature of interest;
determining a difference-based attribute value for each identified feature of interest on the sample based on the adjusted size of a corresponding segmentation mask; and
determining whether each identified feature of interest on the sample are defects based on comparing, for each identified feature of interest, the determined difference-based attribute value corresponding to the identified feature of interest to a predetermined threshold.
23 . The method of claim 22 , further comprising:
receiving one or more training images of the plurality of features on the sample, wherein the plurality of features on the sample are identified on the received one or more training images; and
generating the deep learning segmentation model based on the received one or more training images of the plurality of features on the sample.
24 . The method of claim 23 , wherein the one or more training images include one or more labeled training images, wherein each feature of the plurality of features on the sample are labeled in the one or more labeled training images.
25 . The method of claim 22 , wherein the determining whether each of the identified one or more features of interest on the sample are defects based on the determined difference-based attribute value corresponding to each of the one or more identified features of interest on the sample and a predetermined threshold comprises:
comparing, during runtime, the determined difference-based attribute value to the predetermined threshold for each of the one or more identified features of interest.
26 . The method of claim 22 , wherein a defect instance is determined when the determined difference-based attribute value is greater than the predetermined threshold for each of the one or more identified features of interest.
27 . The method of claim 22 , wherein a non-defective instance is determined when the determined difference-based attribute value is less than the predetermined threshold for each of the one or more identified features of interest.
28 . The method of claim 22 , wherein the difference-based attribute value includes at least one of:
a grey level value or a standard deviation value.
29 . The method of claim 22 , wherein the predetermined threshold includes a user-defined threshold.