Processing method of medical image and computing apparatus for processing medical image
An embodiment of the invention provides a processing method of a medical image and a computing apparatus for processing a medical image. In the method, one or more image samples are obtained, a tumor region and an appearance feature thereof in the image sample are marked, and an image recognition model is trained according to the image sample, the tumor region thereof, and an appearance feature of a first tumor. The image sample is an image obtained by photographing an animal body. The appearance feature represents an appearance of the first tumor corresponding to the tumor region. The image recognition model identifies a second tumor in an image to be evaluated is a first type. The first type is related to the tumor region and the appearance feature of the first tumor. Accordingly, a prediction accuracy may be improved.
1. A processing method of a medical image, comprising:
obtaining at least one image sample, wherein the at least one image sample is an image obtained by photographing at least one animal body;
marking a tumor region in the at least one image sample and an appearance feature thereof, wherein the appearance feature represents an appearance of a first tumor corresponding to the tumor region;
training an image recognition model according to the at least one image sample, the tumor region thereof, and the appearance feature of the first tumor, wherein the image recognition model recognizes a second tumor in an image to be evaluated is a first type, and the first type is related to the tumor region and the appearance feature of the first tumor;
respectively inputting a first image to be evaluated and a second image to be evaluated to the trained image recognition model to obtain a first prediction result and a second prediction result, wherein the first image to be evaluated is photographed at a third angle relative to one of the at least one animal body, and the second image to be evaluated is photographed at a fourth angle relative to the same animal body; and
recognizing a second tumor in the first image to be evaluated or the second image to be evaluated is the first type according to a statistical result of the first prediction result and the second prediction result.
2. The processing method of the medical image of claim 1 , wherein the image recognition model recognizes an appearance feature of the second tumor, and the steps of training the image recognition model comprise:
assigning recognizing the second tumor is the first type as a first task;
assigning recognizing the appearance feature of the second tumor as a second task; and
performing a multi-task learning on the image recognition model according to the first task and the second task.
3. The processing method of the medical image of claim 1 , wherein the appearance feature comprises at least one of calcification, mass, local asymmetry, atypical hyperplasia, and absence of features.
4. The processing method of the medical image of claim 1 , further comprising:
segmenting the tumor region from one of the image samples; and
adjusting an image of the tumor region to a uniform size, wherein images of tumor regions in other image samples are all adjusted to the uniform size.
5. The processing method of the medical image of claim 1 , wherein the at least one image sample comprises a first sample and a second sample, and the processing method further comprises:
rotating, flipping, or translating the first sample to generate the second sample.
6. The processing method of the medical image of claim 1 , wherein the at least one image sample comprises a third sample and a fourth sample, the third sample is photographed at a first angle relative to the at least one animal body, and the fourth sample is photographed at a second angle with respect to the at least one animal body, via magnification photography or via compression photography.
7. The processing method of the medical image of claim 1 , wherein the at least one image sample comprises a fifth sample and a sixth sample, the fifth sample and the sixth sample are both for the same animal body, and the processing method further comprises:
using the fifth sample and the sixth sample simultaneously as one of a training sample, a verification sample, and a test sample of the image recognition model.
8. The processing method of the medical image of claim 1 , wherein the step of training the image recognition model comprises:
training the image recognition model via a mixed loss function, wherein the mixed loss function is:
L
(
x
,
w
f
,
w
T
1
,
w
T
2
)
α
1
×
CE
(
y
2
,
T
1
(
f
(
x
,
w
f
)
,
w
T
1
)
)
+
α
2
×
∑
j
=
1
6
CE
(
y
2
j
,
T
2
j
(
f
(
x
,
w
f
)
,
w
T
2
)
)
x is the at least one image sample, w f , w T 1 , w T 2 are weights the image recognition model needs to learn, y 1 is a first true result recognizing the first tumor is the first type, y 2j is a second true result recognizing the appearance feature of the first tumor, a 1 , a 2 are hyperparameters of the image recognition model, f(x, w f ) is a feature encoder of the at least one image sample, T 1 (x, w f ) and T 2 (x, w f ) are classifiers of y 1 , y 2 respectively, CE(y,ŷ)=−[y×log(ŷ)+(1−y)log(1−ŷ)] is a cross entropy loss, y is a true result, and ŷ is a prediction result corresponding to the classifiers.
9. The processing method of the medical image of claim 1 , wherein the first image to be evaluated input to the image recognition model is marked with a tumor region of the second tumor.
10. A computing apparatus for processing a medical image, comprising:
a memory for storing a code; and
a processor coupled to the memory and configured to load and execute the code to:
obtain at least one image sample, wherein the at least one image sample is an image obtained by photographing at least one animal body;
mark a tumor region in the at least one image sample and an appearance feature thereof, wherein the appearance feature represents an appearance of a first tumor corresponding to the tumor region;
train an image recognition model according to the at least one image sample, the tumor region thereof, and the appearance feature of the first tumor, wherein the image recognition model recognizes a second tumor in an image to be evaluated is a first type, and the first type is related to the tumor region and the appearance feature of the first tumor;
input a first image to be evaluated and a second image to be evaluated to the trained image recognition model respectively to obtain a first prediction result and a second prediction result, wherein the first image to be evaluated is photographed at a third angle relative to one of the at least one animal body, and the second image to be evaluated is photographed at a fourth angle relative to the same animal body; and
recognize a second tumor in the first image to be evaluated or the second image to be evaluated is the first type according to a statistical result of the first prediction result and the second prediction result.
11. The computing apparatus for processing the medical image of claim 10 , wherein the image recognition model further recognizes an appearance feature of the second tumor, and the processor is further configured to:
assign recognizing the second tumor is the first type as a first task;
assign recognizing the appearance feature of the second tumor as a second task; and
perform a multi-task learning on the image recognition model according to the first task and the second task.
12. The computing apparatus for processing the medical image of claim 10 , wherein the appearance feature comprises at least one of calcification, mass, local asymmetry, atypical hyperplasia, and absence of features, and the first type is related to invasive breast carcinoma with extensive intraductal component.
13. The computing apparatus for processing the medical image of claim 10 , wherein the processor is further configured to:
segment the tumor region from one of the image samples; and
adjust an image of the tumor region to a uniform size, wherein images of tumor regions in other image samples are all adjusted to the uniform size.
14. The computing apparatus for processing the medical image of claim 10 , wherein the at least one image sample comprises a first sample and a second sample, and the processor is further configured to:
rotate, flip, or translate the first sample to generate the second sample.
15. The computing apparatus for the medical image of claim 10 , wherein the at least one image sample comprises a third sample and a fourth sample, the third sample is photographed at a first angle relative to the at least one animal body, and the fourth sample is photographed at a second angle with respect to the at least one animal body, via magnification photography or via compression photography.
16. The computing apparatus for processing the medical image of claim 10 , wherein the at least one image sample comprises a fifth sample and a sixth sample, the fifth sample and the sixth sample are both for the same animal body, and the processor is further configured to:
use the fifth sample and the sixth sample simultaneously as one of a training sample, a verification sample, and a test sample of the image recognition model.
17. The computing apparatus for processing the medical image of claim 10 , wherein the processor is further configured to:
train the image recognition model via a mixed loss function, wherein the mixed loss function is:
L ( x,w f ,w T 1 ,w T 2 )=α 1 ×CE ( y 1 ,T 1 ( f ( x,w f ), w T1 ))+α 2 ×Σ j=1 6 CE ( y 2j ,T 2j ( f ( x,w f ), w T2 )) (1)
x is the at least one image sample, w f , w T 1 , w T 2 are weights the image recognition model needs to learn, y 1 is a first true result recognizing the first tumor is the first type, y 2j is a second true result recognizing the appearance feature of the first tumor, a 1 , a 2 are hyperparameters of the image recognition model, f(x, w f ) is a feature encoder of the at least one image sample, T 1 (x, w f ) and T 2 (x, w f ) are classifiers of y 1 , y 2 respectively, CE(y,ŷ)=−[y×log(ŷ)+(1−y)log(1−ŷ)] is a cross entropy loss, y is a true result, and ŷ is a prediction result corresponding to the classifiers.
18. The computing apparatus for processing the medical image of claim 10 , wherein the first image to be evaluated input to the image recognition model is marked with a tumor region of the second tumor.