Systems and methods for image segmentation using a scalable and compact convolutional neural network
Embodiments of the disclosure provide systems and methods for segmenting an image. An exemplary system includes a communication interface configured to receive the image acquired by an image acquisition device. The system further includes a memory configured to store a multi-level learning network comprising a plurality of convolution blocks cascaded at multiple levels. The system also includes a processor configured to apply a first convolution block and a second convolution block of the multi-level learning network to the image in series. The first convolution block is applied to the image and the second convolution block is applied to a first output of the first convolution block. The processor is further configured to concatenate the first output of the first convolution block and a second output of the second convolution block to obtain a feature map and obtain a segmented image based on the feature map.
1. A method for segmenting an image, comprising:
receiving, by a communication interface, the image acquired by an image acquisition device;
retrieving a multi-level learning network comprising a plurality of convolution blocks cascaded at multiple levels;
serially applying, by a processor, the convolution blocks of levels previous to a current level in the multi-level learning network to obtain a current level feature map;
determining a current level segmented image based on the current level feature map;
determining, by the processor, an improvement of the current level segmented image; and
based on a comparison of the improvement with a threshold, stopping applying any additional convolution block in the multi-level learning network or applying, by the processor, the convolution block of a level next to the current level to an output from the convolution block of the current level.
2. The method of claim 1 , further comprising:
determining the improvement of the current level segmented image exceeds the threshold;
applying the convolution block of the level next to the current level to an output from the convolution block of the current level;
concatenating an output of the next level convolution block with the current level feature map to obtain a next level feature map; and
obtaining a next level segmented image based on the next level feature map.
3. The method of claim 1 , further comprising:
determining the improvement of the current level segmented image is below the threshold; and
stopping applying any additional convolution block in the multi-level learning network.
4. The method of claim 2 , wherein applying the next level convolution block to the output of the current level convolution block further comprises:
applying a plurality of convolution layers to the output of the current level convolution block in parallel to obtain respective intermediate feature maps; and
determining the output of the next level convolution block by concatenating the intermediate feature maps.
5. The method of claim 2 , further comprising:
determining that a first spatial resolution of the output of the next level convolution block is different from a second spatial resolution of the current level feature map; and
adjusting the output of the next level convolution block from the first spatial resolution to the second spatial resolution before concatenating with the current level feature map.
6. A system for segmenting an image, comprising:
a communication interface configured to receive the image acquired by an image acquisition device;
a memory configured to store a multi-level learning network comprising a plurality of convolution blocks cascaded at multiple levels; and
a processor configured to:
apply a first convolution block and a second convolution block of the multi-level learning network to the image in series, wherein the first convolution block is applied to the image and the second convolution block is applied to a first output of the first convolution block;
concatenate the first output of the first convolution block and a second output of the second convolution block to obtain a feature map, wherein concatenating the first output and the second output includes combining values of the first output and the second output pixel b pixel; and
obtain a segmented image based on the feature map.
7. The system of claim 6 , wherein the second convolution block includes a plurality of convolution layers connected in parallel, wherein the processor is further configured to:
determine intermediate feature maps by applying respective parallel convolution layers to the first output of the first convolution block; and
determine the second output of the second convolution block by concatenating the intermediate feature maps.
8. The system of claim 6 , wherein the first output of the first convolution block has a first spatial resolution, wherein the processor is further configured to:
down-sample the first output from the first spatial resolution to a second spatial resolution lower than the first spatial resolution; and
determine the second output of the second convolution block in the second spatial resolution.
9. The system of claim 8 , wherein the processor is configured to up-sample the second output from the second spatial resolution to the first spatial resolution before concatenating the first output with the second output.
10. The system of claim 8 , wherein the first output of the first convolution block is down-sampled by applying a pooling layer or an atrous convolution.
11. The system of claim 6 , wherein the processor is further configured to:
continue to obtain a next level segmented image by applying a next level convolution block to an output from a current level convolution block until a difference between a current level segmented image and a previous level segmented image is smaller than a threshold.
12. The system of claim 11 , wherein the next level convolution block is connected in series with the current level convolution block, wherein the processor is further configured to:
obtain a next level feature map by concatenating an output from the next level convolution block with a current level feature map; and
obtain the next level segmented image based on the next level feature map.
13. A method for segmenting an image, comprising:
receiving, by a communication interface, the image acquired by an image acquisition device;
retrieving a multi-level learning network comprising a plurality of convolution blocks cascaded at multiple levels;
applying, by a processor, a first convolution block and a second convolution block of the multi-level learning network to the image in series, wherein the first convolution block is applied to the image and the second convolution block is applied to a first output of the first convolution block;
concatenating, by the processor, the first output of the first convolution block and a second output of the second convolution block to obtain a feature map, wherein
concatenating the first output and the second output includes combining values of the first output and the second output pixel by pixel; and
obtaining a segmented image based on the feature map.
14. The method of claim 13 , wherein applying the second convolution block further comprises:
applying a plurality of convolution layers to the first output of the first convolution block in parallel to obtain respective intermediate feature maps; and
determining the second output of the second convolution block by concatenating the intermediate feature maps.
15. The method of claim 13 , wherein the first output of the first convolution block has a first spatial resolution, wherein the method further comprises:
down-sampling the first output from the first spatial resolution to a second spatial resolution lower than the first spatial resolution; and
determining the second output of the second convolution block in the second spatial resolution.
16. The method of claim 15 , further comprising up-sampling the second output from the second spatial resolution to the first spatial resolution before concatenating the first output with the second output.
17. The method of claim 13 , further comprising continuing to obtain a next level segmented image by applying a next level convolution block to an output from a current level convolution block until a difference between a current level segmented image and a previous level segmented image is smaller than a threshold.
18. The method of claim 17 , wherein continuing to obtain the next level segmented image further comprises:
obtaining a next level feature map by concatenating an output from the next level convolution block with a current level feature map; and
obtaining the next level segmented image based on the next level feature map.