Methods and systems for inspection of semiconductor structures with automatically generated defect features
Methods and systems for improved detection and classification of defects of interest (DOI) is realized based on values of one or more automatically generated attributes derived from images of a candidate defect. Automatically generated attributes are determined by iteratively training, reducing, and retraining a deep learning model. The deep learning model relates optical images of candidate defects to a known classification of those defects. After model reduction, attributes of the reduced model are identified which strongly relate the optical images of candidate defects to the known classification of the defects. The reduced model is subsequently employed to generate values of the identified attributes associated with images of candidate defects having unknown classification. In another aspect, a statistical classifier is employed to classify defects based on automatically generated attributes and attributes identified manually.
1. A method comprising:
generating one or more training images including one or more candidate defects, each of the one or more candidate defects having a known defect classification;
training a deep learning based model having the one or more training images of the one or more candidate defects as input to the deep learning model and the known classification of each of the one or more candidate defects as output of the deep learning model, wherein the deep learning based model is a neural network model including at least one convolutional layer including a plurality of image filters and at least one connected layer including a plurality of nodes;
determining a reduced deep learning model by reducing a number of layers of the neural network model, a number of image filters of the neural network model, a number of nodes of the neural network model, or any combination thereof;
retraining the reduced deep learning based model having the one or more images of the one or more candidate defects as input to the reduced deep learning model and the known classification of each of the one or more candidate defects as output of the reduced deep learning model;
determining a value of each of one or more attributes of the reduced deep learning based model associated with each of the one or more training images; and
training a statistical classifier including the value of each of the one or more attributes of the reduced deep learning based model associated with each of the one or more training images as input to the statistical classifier and the known classification associated with each of the one or more training images as output of the statistical classifier.
2. The method of claim 1 , wherein the one or more attributes includes a maximum pixel intensity of a filtered image, wherein the filtered image is any of the one or more training images operated on by any of the plurality of image filters.
3. The method of claim 1 , wherein the one or more attributes includes a value of one or more nodes of the neural network model.
4. The method of claim 1 , further comprising:
determining a value of each of one or more manually generated attributes associated with each of the one or more candidate defects, wherein the input of the statistical classifier also includes the one or more manually generated attributes.
5. The method of claim 1 , further comprising:
training a statistical model relating values of nodes of a layer of the neural network model to values of nodes of a subsequent layer of the neural network model;
determining a relative measure of importance of the layer, a relative measure of importance of each node of the layer, a relative measure of importance of each image filter, or any combination thereof, based on the trained statistical model, wherein the reducing the number of layers of the neural network model, the number of image filters of the neural network model, the number of nodes of the neural network model, or any combination thereof, is based on the determined relative measure of importance of the layer, the relative measure of importance of each node of the layer, the relative measure of importance of each image filter, or any combination thereof.
6. The method of claim 1 , wherein the one or more training images includes a first image including a first instance of a structure associated with the one or more candidate defects and a second image including a second instance of the structure.
7. The method of claim 6 , wherein the one or more training images includes a difference image, wherein the difference image is a pixel-by-pixel difference between the first and second images.
8. The method of claim 6 , wherein the first and second images are collected from the same wafer.
9. The method of claim 1 , further comprising:
generating one or more sample images including one or more candidate defects, each of the one or more candidate defects having an unknown defect classification;
determining a value of each of the one or more attributes of the reduced deep learning based model associated with each of the one or more sample images; and
determining a classification of each of the one or more candidate defects based on the trained statistical classifier, wherein an input to the trained statistical classifier includes the values of the one or more attributes associated with each of the one or more sample images and the output of the trained statistical classifier is the classification of each of the one or more candidate defects.
10. The method of claim 1 , further comprising:
determining a relative measure of importance of each of the one or more attributes based on the trained statistical classifier;
determining a subset of the one or more attributes based on the relative measure of importance associated with each of the one or more attributes;
retraining the statistical classifier including the subset of the one or more attributes associated with each of the one or more training images as input to the statistical classifier and the known classification associated with each of the one or more training images as output of the statistical classifier.
11. A system comprising:
an illumination source providing a first amount of illumination light to a semiconductor wafer at locations of one or more candidate defects on a semiconductor wafer, each of the one or more candidate defects having a known defect classification;
a detector detecting an amount of light from the semiconductor wafer in response to the first amount of illumination light and generating a plurality of training images including the one or more candidate defects; and
a computing system configured to:
train a deep learning based model having the one or more training images of the one or more candidate defects as input to the deep learning model and the known classification of each of the one or more candidate defects as output of the deep learning model, wherein the deep learning based model is a neural network model including at least one convolutional layer including a plurality of image filters and at least one connected layer including a plurality of nodes;
determine a reduced deep learning model by reducing a number of layers of the neural network model, a number of image filters of the neural network model, a number of nodes of the neural network model, or any combination thereof;
retrain the reduced deep learning based model having the one or more images of the one or more candidate defects as input to the reduced deep learning model and the known classification of each of the one or more candidate defects as output of the reduced deep learning model;
determine a value of each of one or more attributes of the reduced deep learning based model associated with each of the one or more training images; and
train a statistical classifier including the value of each of the one or more attributes of the reduced deep learning based model associated with each of the one or more training images as input to the statistical classifier and the known classification associated with each of the one or more training images as output of the statistical classifier.
12. The system of claim 11 , wherein the one or more attributes includes a maximum pixel intensity of a filtered image, wherein the filtered image is any of the one or more training images operated on by any of the plurality of image filters.
13. The system of claim 11 , wherein the one or more attributes includes a value of one or more nodes of the neural network model.
14. The system of claim 11 , the computing system further configured to:
determine a value of each of one or more manually generated attributes associated with each of the one or more candidate defects, wherein the input of the statistical classifier also includes the one or more manually generated attributes.
15. The system of claim 11 , the computing system further configured to:
train a statistical model relating values of nodes of a layer of the neural network model to values of nodes of a subsequent layer of the neural network model; and
determine a relative measure of importance of the layer, a relative measure of importance of each node of the layer, a relative measure of importance of each image filter, or any combination thereof, based on the trained statistical model, wherein the reducing the number of layers of the neural network model, the number of image filters of the neural network model, the number of nodes of the neural network model, or any combination thereof, is based on the determined relative measure of importance of the layer, the relative measure of importance of each node of the layer, the relative measure of importance of each image filter, or any combination thereof.
16. The system of claim 11 , the illumination source providing a second amount of illumination light to a semiconductor wafer at locations of one or more candidate defects on a semiconductor wafer having an unknown defect classification, the detector detecting a second amount of light from the semiconductor wafer in response to the incident amount of illumination light and generating a plurality of sample images including the one or more candidate defects having unknown defect classification, the computing system further configured to:
determine a value of each of the one or more attributes of the reduced deep learning based model associated with each of the one or more sample images; and
determine a classification of each of the one or more candidate defects based on the trained statistical classifier, wherein an input to the trained statistical classifier includes the values of the one or more attributes associated with each of the one or more sample images and the output of the trained statistical classifier is the classification of each of the one or more candidate defects.
17. A system comprising:
an illumination source providing a first amount of illumination light to a semiconductor wafer at locations of one or more candidate defects on a semiconductor wafer, each of the one or more candidate defects having a known defect classification;
a detector detecting an amount of light from the semiconductor wafer in response to the first amount of illumination light and generating a plurality of training images including the one or more candidate defects; and
a computing system comprising:
one or more processors; and
a non-transitory, computer-readable medium storing instructions that when executed by the one or more processors, cause the computing system to:
train a deep learning based model having the one or more training images of the one or more candidate defects as input to the deep learning model and the known classification of each of the one or more candidate defects as output of the deep learning model, wherein the deep learning based model is a neural network model including at least one convolutional layer including a plurality of image filters and at least one connected layer including a plurality of nodes;
determine a reduced deep learning model by reducing a number of layers of the neural network model, a number of image filters of the neural network model, a number of nodes of the neural network model, or any combination thereof;
retrain the reduced deep learning based model having the one or more images of the one or more candidate defects as input to the reduced deep learning model and the known classification of each of the one or more candidate defects as output of the reduced deep learning model;
determine a value of each of one or more attributes of the reduced deep learning based model associated with each of the one or more training images; and
train a statistical classifier including the value of each of the one or more attributes of the reduced deep learning based model associated with each of the one or more training images as input to the statistical classifier and the known classification associated with each of the one or more training images as output of the statistical classifier.
18. The system of claim 17 , wherein the one or more attributes includes a maximum pixel intensity of a filtered image, wherein the filtered image is any of the one or more training images operated on by any of the plurality of image filters.
19. The system of claim 17 , the non-transitory, computer-readable medium further storing instructions, that when executed by the one or more processors, cause the computing system to:
determine a value of each of one or more manually generated attributes associated with each of the one or more candidate defects, wherein the input of the statistical classifier also includes the one or more manually generated attributes.
20. The system of claim 17 , the non-transitory, computer-readable medium further storing instructions, that when executed by the one or more processors, cause the computing system to:
train a statistical model relating values of nodes of a layer of the neural network model to values of nodes of a subsequent layer of the neural network model; and
determine a relative measure of importance of the layer, a relative measure of importance of each node of the layer, a relative measure of importance of each image filter, or any combination thereof, based on the trained statistical model, wherein the reducing the number of layers of the neural network model, the number of image filters of the neural network model, the number of nodes of the neural network model, or any combination thereof, is based on the determined relative measure of importance of the layer, the relative measure of importance of each node of the layer, the relative measure of importance of each image filter, or any combination thereof.