Granted Patent B2

US 12,229,224 · App. 17/469,850 · Granted Feb 18, 2025

Inventors: Lishu Luo (Shenzhen, CN); Hong Shang (Shenzhen, CN); Zhongqian Sun (Shenzhen, CN)

Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

G06F18/2431A61B5/015A61B5/4887A61B5/7267G06T7/0012G06T11/00G06V10/44G06T2207/20084G06T2207/30096G06V2201/03

Abstract

Claims (158)

1. A medical image processing method, performed by a data processing device, the method comprising:

obtaining a medical image;

obtaining a feature map and a medical image content recognition result that correspond to the medical image by using a medical image classification model, or obtaining the feature map and a lesion classification result that correspond to the medical image by using the medical classification model, the feature map including N channels, N being an integer greater than 1;

generating a thermodynamic diagram corresponding to the medical image content recognition result or the lesion classification result according to the feature map and a weight parameter set, the weight parameter set including N weight parameters, the weight parameters having a correspondence with the channels;

obtaining a gradient propagation result according to the medical image content recognition result or the lesion classification result based on a backward gradient propagation algorithm;

generating an object contour map corresponding to the medical image according to the gradient propagation result and convolutional network results of an input layer and previous layers of the medical image classification model, the object contour map describing overall object contour features of the medical image; and

generating an image recognition result corresponding to the medical image according to the thermodynamic diagram and the object contour map.

2. The method according to claim 1 , wherein obtaining the feature map and the medical image content recognition result comprises:

obtaining the feature map corresponding to the medical image by using a convolutional layer of the medical image classification model;

processing the feature map by using a global average pooling (GAP) layer of the medical image classification model, to obtain a feature vector;

calculating C class prediction scores according to the feature vector and the weight parameter set, each class prediction score being corresponding to one class, C being an integer greater than or equal to 1; and

determining the medical image content recognition result from C classes according to the C class prediction scores.

3. The method according to claim 2 , wherein calculating C class prediction scores according to the feature vector and the weight parameter set comprises:

calculating the class prediction score according to equation

Y

c

=

∑

n

w

n

c

⁢

∑

(

x

,

y

)

F

n

(

x

,

y

)

,

wherein Y c represents a class prediction score corresponding to a c th class, C represents a total quantity of classes, F n (x, y) represents a feature map of an n th channel, (x, y) represents a spatial position in the feature map, and w n c represents a weight parameter of the feature map of the n th channel on a prediction being the c th class.

4. The method according to claim 1 , wherein obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result comprises:

obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result based on a guided backpropagation algorithm, the guided backpropagation algorithm being used for back passing a feature corresponding to a position in which both a gradient and an input value are greater than 0.

5. The method according to claim 1 , wherein generating the thermodynamic diagram comprises:

generating the thermodynamic diagram according to equation

P

c

(

x

,

y

)

=

∑

n

w

n

c

⁢

F

n

(

x

,

y

)

,

wherein P c (x, y) represents a thermodynamic diagram corresponding to a c th class, the thermodynamic diagram is obtained through upsampling, F n (x, y) represents a feature map of an n th channel, (x, y) represents a spatial position in the feature map, w n c represents a weight parameter of the feature map of the n th channel on a prediction being the c th class, and the c th class belongs to the lesion classification result.

6. The method according to claim 5 , wherein the lesion recognition result is obtained by using equation

Visual c ( x,y )= G c ( x,y )⊙ P c ( x,y ),

wherein Visual c (x, y) represents a lesion recognition result corresponding to a c th class, G c (x,y) represents an object contour map corresponding to the c th class, P c (x, y) represents a thermodynamic diagram corresponding to the c th class, the thermodynamic diagram is obtained through upsampling, the c th class belongs to the lesion classification result, and ⊙ represents an XNOR operation.

7. The method according to claim 1 , wherein obtaining the gradient propagation result comprises:

obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result based on a guided backpropagation algorithm, the guided backpropagation algorithm being used for back passing a feature corresponding to a position in which both a gradient and an input value are greater than 0.

8. The method according to claim 1 , wherein generating the image recognition result comprises:

multiplying pixels of the object contour map by pixels of the thermodynamic diagram point by point to obtain a classification result interpretation diagram; and

generating the image recognition result based on the classification result interpretation diagram.

9. The method according to claim 1 , wherein the object contour map reserves high-resolution image contour information and has no relationship with a specified class.

10. A medical image processing apparatus, comprising: a memory storing computer program instructions; and a processor coupled to the memory and configured to execute the computer program instructions and perform:

obtaining a medical image;

obtaining a feature map and a medical image content recognition result that correspond to the medical image by using a medical image classification model, or obtaining the feature map and a lesion classification result that correspond to the medical image by using the medical classification model, the feature map including N channels, N being an integer greater than 1;

generating a thermodynamic diagram corresponding to the medical image content recognition result or the lesion classification result according to the feature map and a weight parameter set, the weight parameter set including N weight parameters, the weight parameters having a correspondence with the channels;

obtaining a gradient propagation result according to the medical image content recognition result or the lesion classification result based on a backward gradient propagation algorithm;

generating an object contour map corresponding to the medical image according to the gradient propagation result and convolutional network results of an input layer and previous layers of the medical image classification model, the object contour map describing overall object contour features of the medical image; and

generating an image recognition result corresponding to the medical image according to the thermodynamic diagram and the object contour map.

11. The medical image processing apparatus according to claim 10 , wherein obtaining the feature map and the medical image content recognition result comprises:

obtaining the feature map corresponding to the medical image by using a convolutional layer of the medical image classification model;

processing the feature map by using a global average pooling (GAP) layer of the medical image classification model, to obtain a feature vector;

calculating C class prediction scores according to the feature vector and the weight parameter set, each class prediction score being corresponding to one class, C being an integer greater than or equal to 1; and

determining the medical image content recognition result from C classes according to the C class prediction scores.

12. The medical image processing apparatus according to claim 11 , wherein calculating C class prediction scores according to the feature vector and the weight parameter set comprises:

calculating the class prediction score according to equation

Y

c

=

∑

n

w

n

c

⁢

∑

(

x

,

y

)

F

n

(

x

,

y

)

,

wherein Y c represents a class prediction score corresponding to a c th class, C represents a total quantity of classes, F n (x, y) represents a feature map of an n th channel, (x, y) represents a spatial position in the feature map, and w n c represents a weight parameter of the feature map of the n th channel on a prediction being the c th class.

13. The medical image processing apparatus according to claim 10 , wherein obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result comprises:

obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result based on a guided backpropagation algorithm, the guided backpropagation algorithm being used for back passing a feature corresponding to a position in which both a gradient and an input value are greater than 0.

14. The medical image processing apparatus according to claim 10 , wherein generating the thermodynamic diagram comprises:

generating the thermodynamic diagram according to equation

P

c

(

x

,

y

)

=

∑

n

w

n

c

⁢

F

n

(

x

,

y

)

,

wherein P c (x, y) represents a thermodynamic diagram corresponding to a c th class, the thermodynamic diagram is obtained through upsampling, F n (x, y) represents a feature map of an n th channel, (x, y) represents a spatial position in the feature map, w n c represents a weight parameter of the feature map of the n th channel on a prediction being the c th class, and the c th class belongs to the lesion classification result.

15. The medical image processing apparatus according to claim 14 , wherein the lesion recognition result is obtained by using equation

Visual c ( x,y )= G c ( x,y )⊙ P c ( x,y ),

wherein Visual c (x, y) represents a lesion recognition result corresponding to a c th class, G c (x, y) represents an object contour map corresponding to the c th class, P c (x, y) represents a thermodynamic diagram corresponding to the c th class, the thermodynamic diagram is obtained through upsampling, the c th class belongs to the lesion classification result, and ⊙ represents an XNOR operation.

16. The medical image processing apparatus according to claim 10 , wherein obtaining the gradient propagation result comprises:

obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result based on a guided backpropagation algorithm, the guided backpropagation algorithm being used for back passing a feature corresponding to a position in which both a gradient and an input value are greater than 0.

17. A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform:

obtaining a medical image;

obtaining a feature map and a medical image content recognition result that correspond to the medical image by using a medical image classification model, or obtaining the feature map and a lesion classification result that correspond to the medical image by using the medical classification model, the feature map including N channels, N being an integer greater than 1;

generating a thermodynamic diagram corresponding to the medical image content recognition result or the lesion classification result according to the feature map and a weight parameter set, the weight parameter set including N weight parameters, the weight parameters having a correspondence with the channels;

obtaining a gradient propagation result according to the medical image content recognition result or the lesion classification result based on a backward gradient propagation algorithm;

generating an object contour map corresponding to the medical image according to the gradient propagation result and convolutional network results of an input layer and previous layers of the medical image classification model, the object contour map describing overall object contour features of the medical image; and

generating an image recognition result corresponding to the medical image according to the thermodynamic diagram and the object contour map.

18. The non-transitory computer-readable storage medium according to claim 17 , wherein obtaining the feature map and the medical image content recognition result comprises:

obtaining the feature map corresponding to the medical image by using a convolutional layer of the medical image classification model;

processing the feature map by using a global average pooling (GAP) layer of the medical image classification model, to obtain a feature vector;

calculating C class prediction scores according to the feature vector and the weight parameter set, each class prediction score being corresponding to one class, C being an integer greater than or equal to 1; and

determining the medical image content recognition result from C classes according to the C class prediction scores.

19. The non-transitory computer-readable storage medium according to claim 17 , wherein obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result comprises:

obtaining the gradient propagation result according to the medical image content recognition result or the lesion classification result based on a guided backpropagation algorithm, the guided backpropagation algorithm being used for back passing a feature corresponding to a position in which both a gradient and an input value are greater than 0.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (21)