Image segmentation method, system, and device, and readable storage medium
Disclosed is an image segmentation method, including: obtaining an original image set; performing feature extraction on the original image set by using a backbone network to obtain a feature map set; performing channel extraction fusion processing on the feature map set by using a channel extraction fusion model to obtain an enhanced feature map set; and segmenting the enhanced feature map set by using a preset convolutional neural network to obtain an image segmentation result. In addition, the present application also provides an image segmentation system and device, and a readable storage medium, which have the beneficial effects above.
1. An image segmentation method, comprising:
obtaining an original image set;
performing feature extraction on the original image set by using a backbone network to obtain a feature map set;
performing channel extraction fusion processing on the feature map set by using a channel extraction fusion model to obtain an enhanced feature map set; and
segmenting the enhanced feature map set by using a preset convolutional neural network to obtain an image segmentation result,
wherein when the channel extraction fusion model comprises a first channel extraction fusion sub-model, the performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a first enhanced feature map subset;
starting a first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a second enhanced feature map subset; and
superimposing the first enhanced feature map subset and the second enhanced feature map subset to obtain the enhanced feature map set,
wherein the starting the first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the first enhanced feature map subset comprises:
calculating a feature vector of each feature map in the feature map set;
calculating a weight value of each feature map by using an activation function according to each feature vector;
sorting each weight value from large to small, and selecting a preset proportion of top-ranking weight values as enhanced weight values; and
selecting a channel corresponding to each feature map according to a channel position corresponding to each enhanced weight value, and performing multiplication to obtain the first enhanced feature map subset,
wherein the starting the first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain the second enhanced feature map subset comprises:
reading the weight value of each feature map, and determining an extraction probability of each channel according to each weight value;
extracting a preset proportion of channels as enhanced channels according to the extraction probability of each channel, and determining feature maps corresponding to the enhanced channels as enhanced feature maps;
performing feature extraction on each enhanced feature map to obtain a feature vector;
calculating a weight value of each enhanced feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each enhanced feature map according to a channel position corresponding to each weight value of each enhanced feature map, and performing multiplication to obtain the second enhanced feature map subset.
2. The method according to claim 1 , wherein when the channel extraction fusion model comprises a second channel extraction fusion sub-model, the performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a third enhanced feature map subset;
starting a second feature extraction process of the second channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a fourth enhanced feature map subset; and
superimposing the third enhanced feature map subset and the fourth enhanced feature map subset to obtain the enhanced feature map set.
3. The method according to claim 2 , wherein the starting the second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the third enhanced feature map subset comprises:
performing a depthwise separable convolution operation on each feature map in the feature map set to obtain a down-sampled feature map;
calculating a feature vector of each down-sampled feature map, and calculating a weight value of each down-sampled feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each down-sampled feature map according to a channel position corresponding to each weight value of each down-sampled feature map, and performing multiplication to obtain the third enhanced feature map subset.
4. The method according to claim 2 , wherein the starting the second feature extraction process of the second channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain the fourth enhanced feature map subset comprises:
halving a length and width of each feature map in the feature map set to obtain a down-sampled feature map;
calculating a feature vector of each down-sampled feature map, and calculating a weight value of each down-sampled feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each down-sampled feature map according to a channel position corresponding to each weight value of each down-sampled feature map, and performing multiplication to obtain the fourth enhanced feature map subset.
5. An image segmentation device, comprising:
a memory, configured to store a computer program; and
a processor, configured to execute the computer program to cause the processor to perform operations comprising:
obtaining an original image set;
performing feature extraction on the original image set by using a backbone network to obtain a feature map set;
performing channel extraction fusion processing on the feature map set by using a channel extraction fusion model to obtain an enhanced feature map set; and
segmenting the enhanced feature map set by using a preset convolutional neural network to obtain an image segmentation result,
wherein when the operation of channel extraction fusion model comprises a first channel extraction fusion sub-model, the performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a first enhanced feature map subset;
starting a first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a second enhanced feature map subset; and
superimposing the first enhanced feature map subset and the second enhanced feature map subset to obtain the enhanced feature map set,
wherein the operation of starting the first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the first enhanced feature map subset comprises:
calculating a feature vector of each feature map in the feature map set;
calculating a weight value of each feature map by using an activation function according to each feature vector;
sorting each weight value from large to small, and selecting a preset proportion of top-ranking weight values as enhanced weight values; and
selecting a channel corresponding to each feature map according to a channel position corresponding to each enhanced weight value, and performing multiplication to obtain the first enhanced feature map subset,
wherein the operation of starting the first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain the second enhanced feature map subset comprises:
reading the weight value of each feature map, and determining an extraction probability of each channel according to each weight value;
extracting a preset proportion of channels as enhanced channels according to the extraction probability of each channel, and determining feature maps corresponding to the enhanced channels as enhanced feature maps;
performing feature extraction on each enhanced feature map to obtain a feature vector;
calculating a weight value of each enhanced feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each enhanced feature map according to a channel position corresponding to each weight value of each enhanced feature map, and performing multiplication to obtain the second enhanced feature map subset.
6. The image segmentation device according to claim 5 , wherein when the channel extraction fusion model comprises a second channel extraction fusion sub-model, the operation of performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a third enhanced feature map subset;
starting a second feature extraction process of the second channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a fourth enhanced feature map subset; and
superimposing the third enhanced feature map subset and the fourth enhanced feature map subset to obtain the enhanced feature map set.
7. The image segmentation device according to claim 6 , wherein the operation of starting the second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the third enhanced feature map subset comprises:
performing a depthwise separable convolution operation on each feature map in the feature map set to obtain a down-sampled feature map;
calculating a feature vector of each down-sampled feature map, and calculating a weight value of each down-sampled feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each down-sampled feature map according to a channel position corresponding to each weight value of each down-sampled feature map, and performing multiplication to obtain the third enhanced feature map subset.
8. The image segmentation device according to claim 6 , wherein the operation of starting the second feature extraction process of the second channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain the fourth enhanced feature map subset comprises:
halving a length and width of each feature map in the feature map set to obtain a down-sampled feature map;
calculating a feature vector of each down-sampled feature map, and calculating a weight value of each down-sampled feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each down-sampled feature map according to a channel position corresponding to each weight value of each down-sampled feature map, and performing multiplication to obtain the fourth enhanced feature map subset.
9. A non-transitory readable storage medium, characterized by storing a computer program that is executed by a processor, and upon execution by the processor, is configured to cause the processor to perform operations comprising:
obtaining an original image set;
performing feature extraction on the original image set by using a backbone network to obtain a feature map set;
performing channel extraction fusion processing on the feature map set by using a channel extraction fusion model to obtain an enhanced feature map set; and
segmenting the enhanced feature map set by using a preset convolutional neural network to obtain an image segmentation result, wherein when the channel extraction fusion model comprises a first channel extraction fusion sub-model, the operation of performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a first enhanced feature map subset;
starting a first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a second enhanced feature map subset; and
superimposing the first enhanced feature map subset and the second enhanced feature map subset to obtain the enhanced feature map set,
wherein the operation of starting the first channel selection process of the first channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the first enhanced feature map subset comprises:
calculating a feature vector of each feature map in the feature map set;
calculating a weight value of each feature map by using an activation function according to each feature vector;
sorting each weight value from large to small, and selecting a preset proportion of top-ranking weight values as enhanced weight values; and
selecting a channel corresponding to each feature map according to a channel position corresponding to each enhanced weight value, and performing multiplication to obtain the first enhanced feature map subset,
wherein the operation of starting the first feature extraction process of the first channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain the second enhanced feature map subset comprises:
reading the weight value of each feature map, and determining an extraction probability of each channel according to each weight value;
extracting a preset proportion of channels as enhanced channels according to the extraction probability of each channel, and determining feature maps corresponding to the enhanced channels as enhanced feature maps;
performing feature extraction on each enhanced feature map to obtain a feature vector;
calculating a weight value of each enhanced feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each enhanced feature map according to a channel position corresponding to each weight value of each enhanced feature map, and performing multiplication to obtain the second enhanced feature map subset.
10. The non-transitory readable storage medium according to claim 9 , wherein when the channel extraction fusion model comprises a second channel extraction fusion sub-model, the operation of performing channel extraction fusion processing on the feature map set by using the channel extraction fusion model to obtain the enhanced feature map set comprises:
starting a second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain a third enhanced feature map subset;
starting a second feature extraction process of the second channel extraction fusion sub-model to perform feature extraction on the feature map set to obtain a fourth enhanced feature map subset; and
superimposing the third enhanced feature map subset and the fourth enhanced feature map subset to obtain the enhanced feature map set.
11. The non-transitory readable storage medium according to claim 10 , wherein the operation of starting the second channel selection process of the second channel extraction fusion sub-model to perform channel selection on the feature map set to obtain the third enhanced feature map subset comprises:
performing a depthwise separable convolution operation on each feature map in the feature map set to obtain a down-sampled feature map;
calculating a feature vector of each down-sampled feature map, and calculating a weight value of each down-sampled feature map by using an activation function according to each feature vector; and
selecting a channel corresponding to each down-sampled feature map according to a channel position corresponding to each weight value of each down-sampled feature map, and performing multiplication to obtain the third enhanced feature map subset.