Neural network model fusion method and electronic device using the same
A neural network model fusion method and an electronic device using the same are provided. The neural network model fusion method includes the following steps. An image is received. The image is analyzed through several neural network models. The neural network models include at least two of a degree classification model, a multi-class identification model and an object detection model. Several analysis results are obtained according to the neural network models. These analysis results are converted into a number of conversion factors. The conversion factors are inputted into a fusion model to obtain a fusion result.
1. A neural network model fusion method, comprising:
receiving an image;
analyzing the image through a plurality of neural network models comprising at least two of a degree classification model, a multi-class identification model and an object detection model;
obtaining a plurality of analysis results according to the neural network models;
converting the analysis results into a plurality of conversion factors; and
inputting the conversion factors into a fusion model to obtain a fusion result;
wherein the neural network models comprises the degree classification model, the analysis results of the degree classification model are presented as a distribution probability matrix of a plurality of levels, the distribution probability matrix has a plurality of distribution probability values whose sum adds up to 1, and the distribution probability matrix is element-wise multiplied by a weight matrix to obtain a weighted probability matrix used as the conversion factors of the degree classification model.
2. The neural network model fusion method according to claim 1 , wherein each of the analysis results of the degree classification model is a classification result of disease severity.
3. The neural network model fusion method according to claim 1 , wherein the neural network models comprise the multi-class identification model, the analysis results of the multi-class identification model are presented as an individual probability matrix of a plurality of classes, the individual probability matrix has a plurality of individual probabilities each having a value between 0.0 to 1.0, the individual probability matrix is converted into a distribution probability matrix of a plurality of levels through a conversion model, and the distribution probability matrix is used as the conversion factors of the multi-class identification model.
4. The neural network model fusion method according to claim 3 , wherein the quantity of the levels of the distribution probability matrix of the multi-class identification model is the same as that of the distribution probability matrix of the degree classification model.
5. The neural network model fusion method according to claim 3 , wherein each of the analysis results of the multi-class identification model is a multi-lesion classification result.
6. The neural network model fusion method according to claim 1 , wherein the neural network models comprise the multi-class identification model, the analysis results of the multi-class identification model are presented as an individual probability matrix of a plurality of classes, the individual probability matrix is converted into a distribution probability matrix of a plurality of levels through a correspondence table, and the distribution probability matrix is used as the conversion factors of the multi-class identification model.
7. The neural network model fusion method according to claim 6 , wherein the quantity of the levels of the distribution probability matrix of the multi-class identification model is the same as that of the distribution probability matrix of the degree classification model.
8. The neural network model fusion method according to claim 1 , wherein the neural network models comprise the object detection model, the analysis results of the object detection model are presented as at least one individual region description matrix of a plurality of objects, the at least one individual region description matrix is converted into an all-object region description matrix of the objects, and the all-object region description matrix is used as the conversion factors of the object detection model.
9. The neural network model fusion method according to claim 8 , wherein each of the analysis results of the object detection model is a lesion detection result.
10. An electronic device, comprising:
a processor configured to perform a neural network model fusion method, which comprises:
receiving an image;
analyzing the image through a plurality of neural network models comprising at least two of a degree classification model, a multi-class identification model and an object detection model;
obtaining a plurality of analysis results according to the neural network models;
converting the analysis results into a plurality of conversion factors; and
inputting the conversion factors into a fusion model to obtain a fusion result;
wherein the neural network models comprises the degree classification model, the analysis results of the degree classification model are presented as a distribution probability matrix of a plurality of levels, the distribution probability matrix has a plurality of distribution probability values whose sum adds up to 1, and the distribution probability matrix is element-wise multiplied by a weight matrix to obtain a weighted probability matrix used as the conversion factors of the degree classification model.
11. The electronic device according to claim 10 , wherein each of the analysis results of the degree classification model is a classification result of disease severity.
12. The electronic device according to claim 10 , wherein the neural network models comprise the multi-class identification model, the analysis results of the multi-class identification model are presented as an individual probability matrix of a plurality of classes, the individual probability matrix has a plurality of individual probabilities each having a value between 0.0 to 1.0, the individual probability matrix is converted into a distribution probability matrix of a plurality of levels through a conversion model, and the distribution probability matrix is used as the conversion factors of the multi-class identification model.
13. The electronic device according to claim 12 , wherein a quantity of the levels of the distribution probability matrix of the multi-class identification model is the same as that of the distribution probability matrix of the degree classification model.
14. The electronic device according to claim 12 , wherein each of the analysis results of the multi-class identification model is a multi-lesion classification result.
15. The electronic device according to claim 10 , wherein the neural network models comprise the multi-class identification model, the analysis results of the multi-class identification model are presented as an individual probability matrix of a plurality of classes, the individual probability matrix is converted into a distribution probability matrix of a plurality of levels through a correspondence table, and the distribution probability matrix is used as the conversion factors of the multi-class identification model.
16. The electronic device according to claim 15 , wherein the quantity of the levels of the distribution probability matrix of the multi-class identification model is the same as that of the distribution probability matrix of the degree classification model.
17. The electronic device according to claim 10 , wherein the neural network models comprise the object detection model, the analysis results of the object detection model are presented as at least one individual region description matrix of a plurality of objects, the at least one individual region description matrix is converted into an all-object region description matrix of the objects, and the all-object region description matrix is used as the conversion factors of the object detection model.
18. The electronic device according to claim 17 , wherein each of the analysis results of the object detection model is a lesion detection result.