Three-dimensional medical image recognition method and apparatus, device, storage medium, and product

1 . A three-dimensional (3D) medical image recognition method, comprising:

in a multi-round feature extraction iteration process:

performing, in an i th -round feature extraction process, view rearrangement processing on an (i−1) th -round modified 3D medical image feature to obtain two-dimensional (2D) image features, wherein

2D image features being features of the (i−1) th round modified 3D medical image feature in different views,

performing semantic feature extraction processing on each 2D image feature to obtain image semantic features in the different views, and

performing feature fusion processing on the image semantic features in the different views to obtain an i th -round modified 3D medical image feature; and

performing image recognition processing on an I th -round modified 3D medical image feature obtained through I th -round feature extraction to obtain an image recognition result of a 3D medical image, wherein

i is a positive integer that increases in sequence, 1<i≤I,

I is a positive integer, and

the modified 3D medical image feature in an initial round of the multi-round feature extraction iteration process is obtained by performing feature extraction on an initial 3D medical image feature, the initial 3D medical image feature being obtained by performing initial patch embedding on the 3D medical image.

2 . The method according to claim 1 , wherein the performing semantic feature extraction processing on each 2D image feature to obtain image semantic features in the different views comprises:

performing spatial feature extraction processing on the 2D image feature to obtain a 2D image spatial feature; and

performing semantic feature extraction processing on the 2D image spatial feature based on a main view and an auxiliary view to obtain an image semantic feature, wherein

the main view is a view corresponding to the 2D image feature, and

the auxiliary view is a view different from the main view in a 3D view.

3 . The method according to claim 2 , wherein performing the spatial feature extraction processing on the 2D image feature to obtain a 2D image spatial feature comprises:

performing window division processing on the 2D image feature to obtain N local 2D image features respectively corresponding to N windows, wherein

the N windows not overlapping each other, and

N is a positive integer greater than 1;

performing feature extraction processing on the N local 2D image features to obtain 2D image window features;

performing transposition processing on the N windows; and

performing feature extraction processing on the 2D image window features respectively corresponding to the N windows after transposition to obtain the 2D image spatial feature, wherein transposition is used for changing spatial positions of the N windows.

4 . The method according to claim 3 , wherein performing the feature extraction processing on the 2D image window features respectively corresponding to the N windows after transposition to obtain the 2D image spatial feature comprises:

performing self-attention processing on the 2D image window features respectively corresponding to the N windows after transposition to obtain N self-attention features respectively corresponding to the N windows;

performing feature fusion processing on the N self-attention features to obtain a second image window internal feature;

restoring the second image window internal feature;

performing convolution processing on a restored second image window internal feature to obtain a second image window interactive feature; and

performing feature extraction processing on the second image window interactive feature by using a multilayer perceptron (MLP) to obtain the 2D image spatial feature.

5 . The method according to claim 3 , wherein performing the feature extraction processing on the N local 2D image features to obtain 2D image window features comprises:

performing self-attention processing on the N local 2D image features to obtain N self-attention features respectively corresponding to the N local 2D image features;

performing feature fusion processing on the N self-attention features to obtain a first image window internal feature;

performing convolution processing on the first image window internal feature to obtain a first image window interactive feature; and

performing feature extraction processing on the first image window interactive feature by using an MLP to obtain the 2D image window feature.

6 . The method according to claim 5 , wherein performing the self-attention processing on the N local 2D image features to obtain self-attention features respectively corresponding to the N local 2D image features comprises:

performing self-attention processing based on a query item Q, a key item K, and a value item V corresponding to the N local 2D image feature to obtain the self-attention features of the N local 2D image features.

7 . The method according to claim 6 , wherein

the i th -round feature extraction process comprises a feature encoding process or a feature decoding process;

a K value in a t th -round feature decoding process is obtained through fusion based on a K value in (t−1) th -round feature decoding and a K value in a corresponding feature encoding process;

a V value in the t th -round feature decoding process is obtained through fusion based on a V value in (t−1) th -round feature decoding and a V value in the corresponding feature encoding process; and

a Q value in the t th -round feature decoding process is a Q value in (t−1) th -round feature decoding; and

t is a positive integer greater than 1.

8 . The method according to claim 2 , wherein performing the semantic feature extraction processing on the 2D image spatial feature based on a main view and an auxiliary view to obtain the image semantic feature comprises:

performing feature fusion processing on the 2D image spatial feature and a position encoding feature to obtain a first image semantic feature, wherein

the position encoding feature is used for indicating position information corresponding to the 2D image feature;

performing semantic feature extraction processing on the first image semantic feature in the main view by using an MLP to obtain a main image semantic feature;

performing semantic feature extraction processing on the first image semantic feature in the auxiliary view by using the MLP to obtain an auxiliary image semantic feature; and

performing feature fusion processing on the main image semantic feature and the auxiliary image semantic feature to obtain the image semantic feature.

9 . The method according to claim 1 , wherein performing the feature fusion processing on the image semantic features in the different views to obtain an i th -round modified 3D medical image feature comprises:

performing fusion processing on the image semantic feature and a view feature to obtain a view image semantic feature; and

performing feature fusion processing on each view image semantic feature to obtain the i th round modified 3D medical image feature.

10 . The method according to claim 1 , wherein performing the semantic feature extraction processing on each 2D image feature to obtain image semantic features in the different views comprises:

performing semantic feature extraction processing on the 2D image feature in each view by using a feature extraction network corresponding to a same network parameter to obtain the image semantic features in the different views.

11 . The method according to claim 1 , wherein

the i th -round feature extraction process comprises a feature encoding process or a feature decoding process,

the feature encoding process comprises a modified 3D medical image feature downsampling process,

the feature decoding process comprises a modified 3D medical image feature upsampling process, and

the method further comprises:

before the performing image recognition, processing on an I th -round modified 3D medical image feature obtained through I th -round feature extraction to obtain an image recognition result of the 3D medical image; and

determining, in a case that an upsampling result reaches an original size, the i th round modified 3D medical image feature obtained through extraction as the I th -round modified 3D medical image feature obtained through I th -round feature extraction.

12 . The method according to claim 1 , wherein

the 3D medical image is a computed tomography (CT) image,

a magnetic resonance imaging (MRI) image, or

a positron emission tomography (PET) image.

13 . A non-transitory computer-readable storage medium storing at least one instruction, at least one piece of program, a code set, or an instruction set, and the at least one instruction, wherein

the at least one piece of program, the code set, or the instruction set is loaded and executed by a processor to implement the three-dimensional (3D) medical image recognition method according to claim 1 .

14 . An electronic device, comprising a memory for storing computer-readable instructions, and at least one processor configured to execute the computer-readable instructions to cause the electronic device to:

in a multi-round feature extraction iteration process:

perform, in an i th -round feature extraction process, view rearrangement processing on an (i−1) th -round modified 3D medical image feature to obtain two-dimensional (2D) image features, wherein

different 2D image features are features of the (i−1) th -round modified 3D medical image feature in different views,

perform semantic feature extraction processing on each 2D image feature to obtain image semantic features in different views, and

perform feature fusion processing on the image semantic features in different views to obtain an i th -round modified 3D medical image feature; and

perform image recognition processing on an I th round modified 3D medical image feature obtained through I th -round feature extraction, to obtain an image recognition result of a 3D medical image, wherein

i is a positive integer that increases in sequence, 1<i≤I,

I is a positive integer, and

the modified 3D medical image feature in an initial round of the multi-round feature extraction iteration process is obtained by performing feature extraction on an initial 3D medical image feature, the initial 3D medical image feature being obtained by performing initial patch embedding on the 3D medical image.