Granted Patent B2

US 12,260,545 · App. 17/840,798 · Granted Mar 25, 2025

Inventors: Yuki Doi (Tokyo, JP); Naoaki Kondo (Tokyo, JP); Minoru Harada (Tokyo, JP); Hideki Nakayama (Tokyo, JP); Yohei Minekawa (Tokyo, JP); Yuji Takagi (Tokyo, JP)

Assignee: Hitachi High-Tech Corporation

G06T7/001G01N21/9505G06T7/74G06T2207/20081G06T2207/20092G06T2207/20132G06T2207/20212G06T2207/30148G06T2207/30168

Abstract

Claims (108)

1. A sample observation device comprising:

an imaging device; and

a processor executing learning processing for learning a high-picture quality image estimation model and sample observation processing for performing defect detection, wherein

(A) in the learning processing:

(A 1 ) one or more learning defect positions related to a learning sample are acquired;

(A 2 ) a low-picture quality learning image under a first imaging condition is acquired for each of the learning defect positions;

(A 3 ) a first set value related to an imaging count of a high-picture quality learning image is acquired;

(A 4 ) for each of the learning defect positions;

(A 4 a ) the imaging count of the high-picture quality learning image is determined based on the first set value;

(A 4 b ) one or more imaging points as positions where the high-picture quality learning image is captured are determined based on the imaging count determined in (A 4 a );

(A 4 c ) the high-picture quality learning image under a second imaging condition is acquired for each of the one or more imaging points determined in (A 4 b );

(A 5 ) the high-picture quality image estimation model is learned using the low-picture quality learning image and the high-picture quality learning image; and

(A 6 ) a defect detection parameter is adjusted using the high-picture quality image estimation model, and

(B) in the sample observation processing, based on the adjusted defect detection parameter:

(B 1 ) a first inspection image of a defect position of an observation target sample is acquired under the first imaging condition; and

(B 2 ) a defect candidate of the observation target sample is detected based on the first inspection image, wherein

the processor in determining the imaging point:

determines, based on a feature quantity of a defect candidate in the low-picture quality learning image, a priority of each of the defect candidates;

selects, based on the priority of each of the defect candidates, a plurality of defect candidates to be imaged under the second imaging condition from a plurality of defect candidates of the low-picture quality learning image so as to be equal to or less than the imaging count; and

determines the imaging point for imaging the plurality of selected defect candidates.

2. The sample observation device according to claim 1 , wherein

the processor sets the first set value related to the imaging count based on user input.

3. The sample observation device according to claim 1 , wherein

in a case where several defect candidates are mutually close in the low-picture quality learning image, the processor in determining the imaging point determines the imaging point such that a region centered on the imaging point is one region including the several defect candidates.

4. A sample observation device comprising:

an imaging device; and

a processor executing learning processing for learning a high-picture quality image estimation model and sample observation processing for performing defect detection, wherein

(A) in the learning processing:

(A 1 ) one or more learning defect positions related to a learning sample are acquired;

(A 2 ) a low-picture quality learning image under a first imaging condition is acquired for each of the learning defect positions;

(A 3 ) a first set value related to an imaging count of a high-picture quality learning image is acquired;

(A 4 ) for each of the learning defect positions;

(A 4 a ) the imaging count of the high-picture quality learning image is determined based on the first set value;

(A 4 b ) one or more imaging points as positions where the high-picture quality learning image is captured are determined based on the imaging count determined in (A 4 a );

(A 4 c ) the high-picture quality learning image under a second imaging condition is acquired for each of the one or more imaging points determined in (A 4 b );

(A 5 ) the high-picture quality image estimation model is learned using the low-picture quality learning image and the high-picture quality learning image; and

(A 6 ) a defect detection parameter is adjusted using the high-picture quality image estimation model, and

(B) in the sample observation processing, based on the adjusted defect detection parameter:

(B 1 ) a first inspection image of a defect position of an observation target sample is acquired under the first imaging condition; and

(B 2 ) a defect candidate of the observation target sample is detected based on the first inspection image,

wherein

in the learning processing:

a defect-free learning reference image associated with the low-picture quality learning image is acquired under the first imaging condition for each of the learning defect positions; and

the model is learned using the learning reference image, and

in the sample observation processing:

a defect-free first reference image associated with the first inspection image is acquired under the first imaging condition for each defect position of the observation target sample; and

a position and a feature quantity of the defect candidate are calculated based on comparison between the first inspection image and the first reference image.

5. The sample observation device according to claim 4 , wherein

the processor:

in acquiring the learning reference image, acquires the learning reference image by performing imaging with the imaging device under the first imaging condition; and

in acquiring the first reference image, acquires the first reference image by performing imaging with the imaging device under the first imaging condition.

6. The sample observation device according to claim 4 , wherein

the processor:

in acquiring the learning reference image, combines the learning reference image from the low-picture quality learning image; and

in acquiring the first reference image, combines the first reference image from the first inspection image.

7. The sample observation device according to claim 4 , wherein

the processor in adjusting the defect detection parameter:

crops an image with regard to the defect candidate from the low-picture quality learning image;

estimates the high-picture quality learning image from the cropped image using the high-picture quality image estimation model or selects a high-picture quality learning image including the defect candidate from the acquired high-picture quality learning image;

estimates the learning reference image using the high-picture quality image estimation model with regard to a position associated with the defect candidate from the learning reference image or combines the learning reference image from the high-picture quality learning image;

determines a defect position in the high-picture quality learning image by discriminating whether or not the defect candidate is a defect using the high-picture quality learning image and the learning reference image; and

selects a parameter capable of detecting the determined defect position based on evaluation.

8. The sample observation device according to claim 7 , wherein

the processor causes a screen to display the low-picture quality learning image, the high-picture quality learning image, a result of estimation of a high-picture quality inspection image from the first inspection image by the high-picture quality image estimation model, a result of the adjustment of the defect detection parameter, and a defect position corresponding to a detected defect candidate in the first inspection image after the adjustment of the defect detection parameter.

9. A sample observation method in a sample observation device including an imaging device and a processor executing learning processing for learning a high-picture quality image estimation model and sample observation processing for performing defect detection, wherein the method comprises:

as steps executed by the sample observation device,

(A) in the learning processing:

(A 1 ) one or more learning defect positions related to a learning sample is acquired;

(A 2 ) a low-picture quality learning image under a first imaging condition is acquired for each of the learning defect positions;

(A 3 ) a first set value related to an imaging count of a high-picture quality learning image is acquired;

(A 4 ) for each of the learning defect positions;

(A 4 a ) the imaging count of the high-picture quality learning image is determined based on the first set value;

(A 4 b ) one or more imaging points as positions where the high-picture quality learning image is captured are determined based on the imaging count determined in (A 4 a );

(A 4 c ) the high-picture quality learning image under a second imaging condition is acquired for each of the one or more imaging points determined in (A 4 b );

(A 5 ) the high-picture quality image estimation model is learned using the low-picture quality learning image and the high-picture quality learning image; and

(A 6 ) a defect detection parameter is adjusted using the high-picture quality image estimation model, and

(B) in the sample observation processing, based on the adjusted defect detection parameter:

(B 1 ) a first inspection image of a defect position of an observation target sample is acquired under the first imaging condition; and

(B 2 ) a defect candidate of the observation target sample is detected based on the first inspection image, wherein

the determining of the imaging point further comprises

determining, based on a feature quantity of a defect candidate in the low-picture quality learning image, a priority of each of the defect candidates;

selecting, based on the priority of each of the defect candidates, a plurality of defect candidates to be imaged under the second imaging condition from a plurality of defect candidates of the low-picture quality learning image so as to be equal to or less than the imaging count; and

determining the imaging point for imaging the plurality of selected defect candidates.

10. The sample observation method according to claim 9 , further comprising:

setting, by the processor, the first set value related to the imaging count based on user input.

11. The sample observation method according to claim 9 , wherein

in a case where several defect candidates are mutually close in the low-picture quality learning image, the determining of the imaging point further comprises determines the imaging point such that a region centered on the imaging point is one region including the several defect candidates.

12. The sample observation method according to claim 9 , wherein

the learning processing further comprises

acquiring a defect-free learning reference image associated with the low-picture quality learning image under the first imaging condition for each of the learning defect positions; and

updating the model using the learning reference image; and

the sample observation processing further comprises

acquiring a defect-free first reference image associated with the first inspection image under the first imaging condition for each defect position of the observation target sample; and

calculating a position and a feature quantity of the defect candidate based on comparison between the first inspection image and the first reference image.

13. The sample observation device according to claim 12 , wherein

the acquiring of the learning reference image further comprises acquiring the learning reference image by performing imaging with the imaging device under the first imaging condition; and

the acquiring of the first reference image further comprises acquiring the first reference image by performing imaging with the imaging device under the first imaging condition.

14. The sample observation method according to claim 12 , wherein

the acquiring of the learning reference image further comprises combining the learning reference image from the low-picture quality learning image; and

he acquiring of the first reference image further comprises combining the first reference image from the first inspection image.

15. The sample observation method according to claim 12 , wherein adjusting the defect detection parameter further comprises:

cropping an image with regard to the defect candidate from the low-picture quality learning image;

estimating the high-picture quality learning image from the cropped image using the high-picture quality image estimation model or selects a high-picture quality learning image including the defect candidate from the acquired high-picture quality learning image;

estimating the learning reference image using the high-picture quality image estimation model with regard to a position associated with the defect candidate from the learning reference image or combines the learning reference image from the high-picture quality learning image;

determining a defect position in the high-picture quality learning image by discriminating whether or not the defect candidate is a defect using the high-picture quality learning image and the learning reference image; and

selecting a parameter capable of detecting the determined defect position based on evaluation.

16. The sample observation method according to claim 15 , further comprising:

displaying the low-picture quality learning image, the high-picture quality learning image, a result of estimation of a high-picture quality inspection image from the first inspection image by the high-picture quality image estimation model, a result of the adjustment of the defect detection parameter, and a defect position corresponding to a detected defect candidate in the first inspection image after the adjustment of the defect detection parameter.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (27)