Method and assistance system for checking samples for defects
A method for checking samples for defects is provided, in which image data of the samples are recorded and classified into predeterminable defect categories by a defect detection algorithm, and the samples classified into a defect category are represented in a multi-dimensional confusion matrix as a classification result of the defect detection algorithm, characterized in that miniature images which reproduce the image data are assigned according to the classified defect categories of the image data to segments of the confusion matrix which represent the defect categories, and these miniature images are displayed visually, the miniature image is assigned by an interaction with a user or a software robot to a different segment from the assigned segment of the confusion matrix, and is either provided as training image data for the defect detection algorithm or is output as training image data for the defect detection algorithm.
1 . A method for checking samples for defectiveness, the method comprising:
recording image data of the samples to be checked;
classifying the image data associated with the samples into predefinable defect categories by a defect recognition algorithm;
presenting a number of the samples classified in the predefinable defect categories in a multi-dimensional confusion matrix as a classification result of the defect recognition algorithm;
assigning miniature images which reproduce the image data associated with the samples according to the predefinable defect categories of the image data, to segments of the multi-dimensional confusion matrix which represent the predefinable defect categories, wherein the miniature images are presented visually within the segments, and, as a function of the assigning the miniature images, a size of the miniature images is adapted to optically fit into the segment, wherein each miniature image is a thumbnail representation of image data of a respective sample that is visually embedded within a matrix segment corresponding to the classified defect category of the respective sample, wherein the multi-dimensional confusion matrix comprises a first axis representing defect categories classified by the defect recognition algorithm and a second axis representing defect categories assigned by a user or software robot, and wherein the miniature images are visually embedded within individual matrix segments defined by intersections of the first axis and the second axis;
assigning, by way of an interaction with a user or a software robot, a miniature image into a different segment than the assigned segment of the multi-dimensional confusion matrix, which is either provided as training image data for the defect recognition algorithm or is provided and output as training image data for the defect recognition algorithm, wherein the assigning comprises dragging and dropping the miniature image from one matrix segment to another matrix segment within the multi-dimensional confusion matrix.
2 . The method as claimed in claim 1 , wherein a size of the miniature image is individually adapted after selection of the miniature image.
3 . The method as claimed in claim 1 , wherein a number of miniature images within a segment is optically identified.
4 . The method as claimed in claim 1 , wherein the miniature images are positioned within a segment of the multi-dimensional confusion matrix in a manner sorted according to at least one predefinable criterion, the at least one predefinable criterion being a confidence value, an entropy over all defect categories, a dimension reduction, a similarity, or a distance metric.
5 . The method as claimed in claim 1 , wherein an assignment of one or more miniature images from an assigned segment into a different segment of the multi-dimensional confusion matrix is carried out by way of comparison of selected or selectable miniature image regions on a basis of the at least one criterion or on a basis of at least one further criterion including a poor confidence value, a similar image brightness, a visually similar sample shape and/or a recognizable defect.
6 . An assistance system for checking samples for defectiveness by a defect recognition device, which records image data of the samples to be checked and classifies the image data associated with the samples into predefinable defect categories by a defect recognition algorithm, wherein a number of the samples classified in the defect categories are presentable in a multi-dimensional confusion matrix as a classification result of the defect recognition algorithm, the assistance system comprising:
at least one processing unit having at least one storage unit, wherein the processing unit is configured
to assign miniature images which reproduce the image data associated with the samples, according to the classified defect categories of the image data, to segments of the multi-dimensional confusion matrix which represent the predefinable defect categories, wherein the multi-dimensional confusion matrix comprises a first axis representing defect categories classified by the defect recognition algorithm and a second axis representing defect categories assigned by a user or software robot, and wherein the miniature images are visually embedded within individual matrix segments defined by intersections of the first axis and the second axis, and to present the miniature images visually within the segments, and, as a function of assigning the miniature images, a size of the miniature images is adapted to optically fit into the segment, wherein each miniature image is a thumbnail representation of image data of a respective sample that is visually embedded within a matrix segment corresponding to the sample's classified defect category,
to assign a miniature image, by way of an interaction with a user or a software robot, into a different segment than the assigned segment of the multi-dimensional confusion matrix, wherein the assigning comprises dragging and dropping the miniature image from one matrix segment to another matrix segment within the multi-dimensional confusion matrix, either to provide the miniature image as training image data for the defect recognition algorithm in the at least one storage unit or to provide the miniature image as training image data for the defect recognition algorithm in the at least one storage unit and to output the miniature image at an output unit.
7 . The apparatus as claimed claim 6 , wherein the miniature images are positionable within a segment in a manner sorted according to at least one predefinable criterion.
8 . The apparatus as claimed in claim 6 , wherein the number of miniature images within a segment is optically identifiable.
9 . A computer program product, comprising a non-transitory computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement a method as claimed in claim 1 , comprising program code parts designed to carry out the method.
10 . The method as claimed in claim 1 , wherein the samples are checked for defectiveness while a manufacturing machine is running or after a manufacturing process.
11 . The apparatus as claimed in claim 6 , wherein the samples are checked for defectiveness while a manufacturing machine is running or after a manufacturing process.