System and method for defect detection using a conditional masked autoencoder
A method for defect detection includes receiving an optical image of a sample and dividing the received optical image include a plurality of image patches. The method includes generating a masked optical image by selectively masking the image patches based on characterization data. The method includes providing the masked optical image to a first encoder. The method includes receiving a design image and providing it to a second encoder. The method includes generating a first set of feature vectors using the first encoder and generating a second set of feature vectors using the second encoder. The method includes concatenating the feature vectors from the first encoder and the feature vectors from the second encoder and generating a reference image of the sample based on the concatenated feature vectors using a decoder, where the generated reference image is a reconstructed image of the optical image and the design image.
1 . A system for detecting defects on a sample, the system comprising:
a controller including one or more processors configured to execute a set of program instructions stored in memory, wherein a conditional masked autoencoder is stored in the memory, the set of program instructions configured to cause the one or more processors to:
receive an optical image;
generate a plurality of image patches by dividing the received optical image into the plurality of image patches;
generate a masked optical image by selectively masking a first set of image patches of the plurality of image patches based on characterization data, wherein the first set of image patches correspond to one or more defect regions, wherein the plurality of image patches includes a second set of image patches corresponding to one or more defect free regions;
provide the generated masked optical image to a first encoder of the conditional masked autoencoder;
apply the first encoder to the second set of image patches of the generated masked optical image to generate a first set of feature vectors;
receive a design image;
provide the design image to a second encoder of the conditional masked autoencoder;
apply the second encoder to the design image to generate a second set of feature vectors;
concatenate the first set of feature vectors from the first encoder and the second set of feature vectors from the second encoder; and
generate a reference image based on the concatenated feature vectors using a decoder of the conditional masked autoencoder, wherein the generated reference image is a reconstructed image of the optical image and the design image.
2 . The system of claim 1 , wherein the set of program instructions are further configured to cause the one or more processors to:
identify one or more defects on the sample by comparing the generated reference image to a sample image.
3 . The system of claim 1 , wherein each image patch of the plurality of image patches has a patch size of 16×16 pixels.
4 . The system of claim 1 , wherein each image patch of the plurality of image patches has a patch size of 14×14 pixels.
5 . The system of claim 1 , wherein each image patch of the plurality of image patches has a patch size of 28×28 pixels.
6 . The system of claim 1 , wherein each image patch of the plurality of image patches has a patch size of 32×32 pixels.
7 . The system of claim 1 , wherein the characterization data includes defect inspection data from a broadband inspection system.
8 . The system of claim 1 , wherein the conditional masked autoencoder includes a self-supervised deep learning model.
9 . The system of claim 8 , wherein the first encoder and the second encoder include vision transformer encoders, wherein the decoder includes a vision transformer decoder.
10 . The system of claim 1 , further comprising:
an optical sub-system configured to acquire the optical image of the sample.
11 . The system of claim 10 , wherein the optical sub-system includes a scanning electron microscopy sub-system.
12 . The system of claim 11 , wherein the optical image includes a scanning electron microscope (SEM) image.
13 . A system for detecting defects on a sample, the system comprising:
an optical sub-system configured to acquire a plurality of optical images of the sample; and
a controller communicatively coupled to the optical sub-system, the controller includes one or more processors configured to execute a set of program instructions stored in memory, wherein a conditional masked autoencoder is stored in the memory, the set of program instructions configured to cause the one or more processors to:
receive an optical image of the plurality of optical images from the optical sub-system;
generate a plurality of image patches by dividing the received optical image into the plurality of image patches;
generate a masked optical image by selectively masking a first set of image patches of the plurality of image patches based on characterization data, wherein the first set of image patches correspond to one or more defect regions, wherein the plurality of image patches includes a second set of image patches corresponding to one or more defect free regions;
provide the generated masked optical image to a first encoder of the conditional masked autoencoder;
apply the first encoder to the second set of image patches of the generated masked optical image to generate a first set of feature vectors;
receive a design image;
provide the design image to a second encoder of the conditional masked autoencoder;
apply the second encoder to the design image to generate a second set of feature vectors;
concatenate the first set of feature vectors from the first encoder and the second set of feature vectors from the second encoder; and
generate a reference image based on the concatenated feature vectors using a decoder of the conditional masked autoencoder, wherein the generated reference image is a reconstructed image of the optical image and the design image.
14 . The system of claim 13 , wherein the set of program instructions are further configured to cause the one or more processors to:
identify one or more defects on the sample by comparing the generated reference image to an optical image of the plurality of optical images.
15 . The system of claim 13 , wherein each image patch of the plurality of image patches has a patch size of 16×16 pixels.
16 . The system of claim 13 , wherein each image patch of the plurality of image patches has a patch size of 14×14 pixels.
17 . The system of claim 13 , wherein each image patch of the plurality of image patches has a patch size of 28×28 pixels.
18 . The system of claim 13 , wherein each image patch of the plurality of image patches has a patch size of 32×32 pixels.
19 . The system of claim 13 , wherein the characterization data includes defect inspection data from a broadband inspection system.
20 . The system of claim 13 , wherein the conditional masked autoencoder includes a self-supervised deep learning model.
21 . The system of claim 20 , wherein the first encoder and the second encoder include vision transformer encoders, wherein the decoder includes a vision transformer decoder.
22 . The system of claim 13 , wherein the optical sub-system includes a scanning electron microscopy sub-system.
23 . The system of claim 22 , wherein the optical image includes a scanning electron microscope (SEM) image.
24 . A method comprising:
receiving an optical image of a sample;
generating a plurality of image patches by dividing the received optical image into the plurality of image patches;
generating a masked optical image by selectively masking a first set of image patches of the plurality of image patches based on characterization data, wherein the first set of image patches correspond to one or more defect regions, wherein the plurality of image patches includes a second set of image patches corresponding to one or more defect free regions;
providing the generated masked optical image to a first encoder of a conditional masked autoencoder;
applying the first encoder to the second set of image patches of the generated masked optical image to generate a first set of feature vectors;
receiving a design image;
providing the design image to a second encoder of the conditional masked autoencoder;
applying the second encoder to the design image to generate a second set of feature vectors;
concatenating the first set of feature vectors from the first encoder and the second set of feature vectors from the second encoder; and
generating a reference image of the sample based on the concatenated feature vectors using a decoder of the conditional masked autoencoder, wherein the generated reference image is a reconstructed image of the optical image and the design image.
25 . The method of claim 24 , further comprising:
identifying one or more defects on the sample by comparing the generated reference image to an optical image of the plurality of optical images.