Medical image diagnosis assistance apparatus and method for providing user-preferred style based on medical artificial neural network
Disclosed herein is an artificial neural network-based medical image diagnosis assistance apparatus for assisting in diagnosing a medical image based on a medical artificial neural network. A medical image diagnosis assistance apparatus according to an embodiment of the present invention includes a computing system, and the computing system includes at least one processor. The at least one processor is configured to acquire or receive a first analysis result obtained through the inference of a first artificial neural network about a first medical image, to detect user feedback on the first analysis result input by a user, to determine whether the first analysis result and the user feedback satisfy training conditions, and to transfer the first analysis result and the user feedback satisfying the training conditions to a style learning model so that the style learning model is trained on the first analysis result and the user feedback.
1. An artificial neural network-based medical image diagnosis assistance apparatus for assisting in diagnosing a medical image based on a medical artificial neural network, the medical image diagnosis assistance apparatus comprising a computing system, the computing system comprising at least one processor, wherein the at least one processor is configured to:
acquire or receive a first analysis result obtained through an inference of a first artificial neural network about a first medical image;
detect user feedback on the first analysis result input by a user;
determine whether the first analysis result and the user feedback including a modification of the first analysis result satisfy training conditions to train a style learning model, based on at least one of a similarity between a user-modified first analysis result generated based on the user feedback on the first analysis result and the first analysis result, a size of a difference region between the first analysis result and the user-modified first analysis result, a shape of the difference region, or a distribution of brightness values of the difference region in the first medical image; and
transfer the first analysis result and the user feedback satisfying the training conditions to the style learning model so that the style learning model is trained on the first analysis result and the user feedback.
2. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to determine the similarity between the user-modified first analysis result and the first analysis result based on at least one of a Hausdorff distance between a first set of pixels or voxels included in a first boundary of the first analysis result and a modified first set of pixels or voxels included in a user-modified first boundary of the user-modified first analysis result and a ratio between a first area or first volume of an area surrounded by the first boundary and a modified first area or modified first volume of an area surrounded by the user-modified first boundary.
3. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to:
evaluate whether the user accept the first analysis result based on the user feedback on the first analysis result; and
when the user accepts the first analysis result, determine that the first analysis result and the user feedback satisfy the training conditions.
4. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to:
receive a prediction result predicting whether the user will accept the first analysis result; and
determine whether the first analysis result and the user feedback satisfy the training conditions based on the prediction result.
5. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to:
receive a preliminary first analysis result, which is an output before a final layer inside the first artificial neural network, together with the first analysis result; and
transfer the preliminary first analysis result to the style learning model together with the first analysis result and the user feedback satisfying the training conditions, and control training of the style learning model so that the style learning model is trained on relationships among the first analysis result, the preliminary first analysis result, and the user feedback.
6. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to controls training of the style learning model so that the style learning model is trained on a relationship between the first analysis result and the user feedback based on context information including body part and organ information included in the first medical image.
7. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein:
the style learning model is a second artificial neural network, and the computing system further comprises the second artificial neural network; and
the at least one processor is further configured to:
input the first analysis result and the user feedback satisfying the training conditions to the second artificial neural network; and
control training of the second artificial neural network so that the second artificial neural network is trained on a relevance between the first analysis result and the user feedback satisfying the training conditions.
8. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein:
the computing system further comprises a communication interface configured to transmit and receive data to and from the first artificial neural network outside the computing system; and
the at least one processor is further configured to acquire and receive the first analysis result obtained through the inference of the first artificial neural network about the first medical image via the communication interface.
9. The artificial neural network-based medical image diagnosis assistance apparatus of claim 1 , wherein the at least one processor is further configured to:
provide a first user menu configured to receive information about whether the user has approved the first analysis result for the first medical image; and
when the user has approved the first analysis result via the first user menu, determine that the first analysis result and the user feedback satisfy the training conditions.
10. An artificial neural network-based medical image diagnosis assistance method that is performed by a computing system, the computing system comprising at least one processor, the artificial neural network-based medical image diagnosis assistance method comprising:
acquiring or receiving, by the at least one processor, a first analysis result obtained through an inference of a first artificial neural network about a first medical image;
detecting, by the at least one processor, user feedback on the first analysis result input by a user;
determining, by the at least one processor, whether the first analysis result and the user feedback including a modification of the first analysis result satisfy training conditions to train a style learning model, based on at least one of a similarity between a user-modified first analysis result generated based on the user feedback on the first analysis result and the first analysis result, a size of a difference region between the first analysis result and the user-modified first analysis result, a shape of the difference region, or a distribution of brightness values of the difference region in the first medical image; and
transferring, by the at least one processor, the first analysis result and the user feedback satisfying the training conditions to the style learning model so that the style learning model is trained on the first analysis result and the user feedback.