Neural network-based classification method and classification device thereof

1. A neural network-based classification device, comprising:

a storage medium, storing a plurality of modules; and

a processor coupled to the storage medium, the processor accessing and executing the modules of the storage medium, wherein the modules comprise:

a neural network, generating a feature map according to input data;

a cropping layer, cropping the feature map to generate a first cropped part and a second cropped part of the feature map;

a first classifier, generating a first probability vector according to the first cropped part;

a second classifier, generating a second probability vector according to the second cropped part, wherein weights of the first classifier are shared with the second classifier; and

a fusion layer, performing a probability fusion on the first probability vector and the second probability vector to generate an estimated probability vector for determining a class of the input data,

wherein if all elements of the estimated probability vector are smaller than a classification threshold value, the cropping layer crops the feature map to generate a third cropped part of the feature map, and the modules further comprise:

a third classifier, generating a third probability vector according to the third cropped part, wherein

the weights of the first classifier are shared with the third classifier, and

the fusion layer performs a probability fusion on the first probability vector, the second probability vector, and the third probability vector to generate a second estimated probability vector for determining the class of the input data.

2. The classification device according to claim 1 , wherein the neural network is a convolutional neural network.

3. The classification device according to claim 2 , wherein the neural network comprises X first groups formed of layers, and the modules further comprise:

a second neural network comprising Y first groups formed of layers and additional (X-Y) second groups formed of layers, wherein the Y first groups formed of layers are included in the X first groups formed of layers, X>Y≥1, and X and Y are integers.

4. The classification device according to claim 3 , wherein the Y first groups formed of layers corresponding to the X first groups formed of layers are a 1 st group in the X first groups formed of layers to a Y th group in the X first groups formed of layers.

5. The classification device according to claim 3 , wherein if all elements of the estimated probability vector are smaller than the classification threshold value, the second neural network outputs a second feature map according to the input data, and the modules further comprise:

a second cropping layer, cropping the second feature map to generate a plurality of cropped parts of the second feature map;

a plurality of classifiers, generating a plurality of probability vectors according to the cropped parts;

a second fusion layer, performing a probability fusion on the probability vectors to generate a second estimated probability vector; and

a final fusion layer, performing a probability fusion on the estimated probability vector and the second estimated probability vector to generate a final estimated probability vector for determining the class of the input data.

6. The classification device according to claim 1 , wherein the operation of sharing the weights of the first classifier with the second classifier comprises:

applying weights identical to the weights of the first classifier to the second classifier.

7. The classification device according to claim 1 , wherein the first classifier comprises:

a pooling layer, receiving the first cropped part of a plurality of channels to generate a plurality of feature points respectively corresponding to the channels;

a classification layer, converting the feature points into a score vector, wherein each element in the score vector corresponds to a class; and

a softmax function layer, normalizing the score vector to generate the first probability vector, wherein each element in the first probability vector corresponds to a class.

8. The classification device according to claim 1 , wherein the fusion layer performs the probability fusion according to one of the following:

method 1: taking an average of the first probability vector and the second probability vector by using a same weight coefficient to generate the estimated probability vector;

method 2: taking an average of the first probability vector and the second probability vector by using different weight coefficients to generate the estimated probability vector;

method 3: selecting an element having a maximum value from the first probability vector and the second probability vector, and determining a probability vector corresponding to the element as the estimated probability vector;

method 4: randomly selecting one of the first probability vector and the second probability vector as the estimated probability vector; and

method 5: randomly discarding a predetermined number of second elements in the first probability vector and the second probability vector, and then performing one of method 1 to method 4, wherein the predetermined number of the second elements respectively correspond to one of a plurality of classes.

9. The classification device according to claim 1 , wherein the input data comprises image data and mirrored image data of the image data.

10. The classification device according to claim 9 , wherein the image data and the mirrored image data are concatenated into new input data.

11. A neural network-based classification method, comprising:

obtaining a neural network and a first classifier;

inputting input data to the neural network to generate a feature map;

cropping the feature map to generate a first cropped part and a second cropped part of the feature map;

inputting the first cropped part to the first classifier to generate a first probability vector;

inputting the second cropped part to a second classifier to generate a second probability vector, wherein weights of the first classifier are shared with the second classifier; and

performing a probability fusion on the first probability vector and the second probability vector to generate an estimated probability vector for determining a class of the input data,

wherein if all elements of the estimated probability vector are smaller than a classification threshold value, the classification method further comprises:

cropping the feature map to generate a third cropped part of the feature map;

inputting the third cropped part to a third classifier to generate a third probability vector, wherein the weights of the first classifier are shared with the third classifier; and

perfoming a probability fusion on the first probability vector, the second probability vector, and the third probability vector to generate a second estimated probability vector for determining the class of input data.

12. The classification method according to claim 11 , wherein the neural network is a convolutional neural network.

13. The classification method according to claim 12 , wherein the neural network comprises X first groups formed of layers, and the classification method further comprises:

obtaining a second neural network comprising Y first groups formed of layers and additional (X-Y) second groups formed of layers, wherein the Y first groups formed of layers are included in the X first groups formed of layers, X>Y≥1, and X and Y are integers.

14. The classification method according to claim 13 , wherein the Y first groups formed of layers corresponding to the X first groups formed of layers are a 1 st group in the X first groups formed of layers to a Y th group in the X first groups formed of layers.

15. The classification method according to claim 13 , wherein if all elements of the estimated probability vector are smaller than the classification threshold value, the classification method further comprises:

inputting the input data to the second neural network to generate a second feature map;

cropping the second feature map to generate a plurality of cropped parts of the second feature map;

inputting the cropped parts to a plurality of classifiers to generate a plurality of probability vectors;

performing a probability fusion on the probability vectors to generate a second estimated probability vector; and

performing a probability fusion on the estimated probability vector and the second estimated probability vector to generate a final estimated probability vector for determining the class of the input data.

16. The classification method according to claim 11 , wherein the step of sharing the weights of the first classifier with the second classifier comprises:

applying weights identical to the weights of the first classifier to the second classifier.

17. The classification method according to claim 11 , wherein the first classifier comprises:

a pooling layer, receiving the first cropped part of a plurality of channels to generate a plurality of feature points respectively corresponding to the channels;

a classification layer, converting the feature points into a score vector, wherein each element in the score vector corresponds to a class; and

a softmax function layer, normalizing the score vector to generate the first probability vector, wherein each element in the first probability vector corresponds to a class.

18. The classification method according to claim 11 , wherein the probability fusion comprises one of the following:

method 1: taking an average of the first probability vector and the second probability vector by using a same weight coefficient to generate the estimated probability vector;

method 2: taking an average of the first probability vector and the second probability vector by using different weight coefficients to generate the estimated probability vector;

method 3: selecting an element having a maximum value from the first probability vector and the second probability vector, and deteiiiiining a probability vector corresponding to the element as the estimated probability vector;

method 4: randomly selecting one of the first probability vector and the second probability vector as the estimated probability vector; and

method 5: randomly discarding a predetermined number of second elements in the first probability vector and the second probability vector, and then performing one of method 1 to method 4, wherein the predetermined number of the second elements respectively correspond to one of a plurality of classes.

19. The classification method according to claim 11 , wherein the input data comprises image data and mirrored image data of the image data.

20. The classification method according to claim 19 , wherein the image data and the mirrored image data are concatenated into new input data.

21. A neural network-based classification device suitable for classifying appearance defects of objects, the classification device comprising:

an automated optical inspection equipment, obtaining image data of the objects; and

a server comprising a storage medium and a processor, the server being coupled to the automated optical inspection equipment and receiving the image data, wherein

the storage medium stores a plurality of modules, and

the processor is coupled to the storage medium and accesses and executes the modules of the storage medium, the modules comprising:

a neural network, generating a feature map according to the image data;

a cropping layer, cropping the feature map to generate a first cropped part and a second cropped part of the feature map;

a first classifier, generating a first probability vector according to the first cropped part;

a second classifier, generating a second probability vector according to the second cropped part, wherein weights of the first classifier are shared with the second classifier; and

a fusion layer, performing a probability fusion on the first probability vector and the second probability vector to generate an estimated probability vector for determining a class of the image data,

wherein if all elements of the estimated probability vector are smaller than a classification threshold value, the cropping layer crops the feature map to generate a third cropped part of the feature map, and the modules further comprises:

a third classifier, generating a third probability vector according to the third cropped part, wherein

the weights of the first classifier are shared with the third classifier, and

the fusion layer preforms a probability fusion on the first probability vector, the second probability vector, and the third probability vector to generate a second estimated probability vector for determining the class of the image data.