Apparatus and method for image segmentation using a deep convolutional neural network with a nested U-structure

1. A non-transitory computer readable storage medium with instructions which, when executed by a processor, cause the processor to:

receive an ultrasound image;

apply the ultrasound image to a sequence of encoders that includes a first encoder and one or more subsequent encoders, wherein each encoder in the sequence of encoders comprises a U network encoder which performs convolution neural network processing in an orthogonal dimension, and the one or more subsequent encoders receive a down-sampled version of the ultrasound image from a prior encoder, the sequence of encoders forming a first dimension orthogonal to the orthogonal dimension;

apply a result of the sequence of encoders to a sequence of decoders that includes a first decoder and one or more subsequent decoders, wherein a respective decoder in the one or more subsequent decoders performs convolution neural network processing of an up-sampled version of the ultrasound image from the first decoder, and the sequence of decoders forms a second dimension parallel to the first dimension;

produce probability segmentation maps from paired encoders and decoders in the sequence of encoders and the sequence of decoders; and

combine the probability segmentation maps to form a final probability segmentation output.

2. The non-transitory computer readable storage medium of claim 1 , wherein the instructions further cause a respective encoder in the sequence of encoders and a respective decoder in the sequence of decoders to receive input feature maps and generate multi-scale features and local features.

3. The non-transitory computer readable storage medium of claim 1 , wherein the ultrasound image has a shape with an associated height, width, and input channel number.

4. The non-transitory computer readable storage medium of claim 3 , wherein the final probability segmentation output is characterized as Y output =K(W,X input ), wherein X input is the ultrasound image, Y output is a probability map with pixel values ranging from zero to one with the shape of the associated height and width, and W denotes weights of kernel filters.

5. The non-transitory computer readable storage medium of claim 1 , wherein the final probability segmentation output is a multi-channel feature map fed to an activation function to produce multi-class segmentation results.

6. The non-transitory computer readable storage medium of claim 1 , wherein the final probability segmentation output is characterized as a feature map with regression results without using activation functions.

7. The non-transitory computer readable storage medium of claim 3 , wherein the input channel number is two or more.

8. The non-transitory computer readable storage medium of claim 1 , wherein a respective encoder in the sequence of encoders and a respective decoder in the sequence of decoders have configurable heights.

9. The non-transitory computer readable storage medium of claim 1 , wherein a respective encoder in the sequence of encoders and a respective decoder in the sequence of decoders have configurable filter numbers.

10. The non-transitory computer readable storage medium of claim 1 , wherein each decoder in the one or more subsequent decoders concatenates an up-sampled feature map from a previous decoder with a down-sampled feature from a paired encoder.

11. The non-transitory computer readable storage medium of claim 1 , wherein the instructions to combine the probability segmentation maps include instructions which, when executed by the processor, cause the processor to:

concatenate the probability segmentation maps to form concatenated probability segmentation maps.

12. The non-transitory computer readable storage medium of claim 11 , wherein the instructions further cause the processor to:

apply a convolution layer to the concatenated probability segmentation maps to form convoluted concatenated probability segmentation maps.

13. The non-transitory computer readable storage medium of claim 12 , wherein the instructions further cause the processor to apply a sigmoid function to the convoluted concatenated probability segmentation maps to form the final probability segmentation output.

14. A method, comprising

at a computer system that includes a processor and memory:

receiving an ultrasound image;

applying the ultrasound image to a sequence of encoders that includes a first encoder and one or more subsequent encoders, wherein each encoder in the sequence of encoders comprises a U network encoder which performs convolution neural network processing in an orthogonal dimension, and the one or more subsequent encoders receive a down-sampled version of the ultrasound image from a prior encoder, the sequence of encoders forming a first dimension orthogonal to the orthogonal dimension;

applying a result of the sequence of encoders to a sequence of decoders that includes a first decoder and one or more subsequent decoders, wherein a respective decoder in the one or more subsequent decoders performs convolution neural network processing of an up-sampled version of the ultrasound image from the first decoder, and the sequence of decoders forms a second dimension parallel to the first dimension;

producing probability segmentation maps from paired encoders and decoders in the sequence of encoders and the sequence of decoders; and

combining the probability segmentation maps to form a final probability segmentation output.

15. The method of claim 14 , further comprising:

causing a respective encoder in the sequence of encoders and a respective decoder in the sequence of decoders to receive input feature maps and generate multi-scale features and local features.

16. The method of claim 15 , wherein the ultrasound image has a shape with an associated height, width, and input channel number.

17. The method of claim 16 , wherein the final probability segmentation output is characterized as Y output =K(W,X input ), wherein X input is the ultrasound image, Y output is a probability map with pixel values ranging from zero to one with the shape of the associated height and width, and W denotes weights of kernel filters.

18. The method of claim 14 , wherein the final probability segmentation output is a multi-channel feature map fed to an activation function to produce multi-class segmentation results.

19. A computer system, comprising

a processor; and

memory storing one or more programs, the one or more programs including instructions for:

receiving an ultrasound image;

applying the ultrasound image to a sequence of encoders that includes a first encoder and one or more subsequent encoders, wherein each encoder in the sequence of encoders comprises a U network encoder which performs convolution neural network processing in an orthogonal dimension, and the one or more subsequent encoders receive a down-sampled version of the ultrasound image from a prior encoder, the sequence of encoders forming a first dimension orthogonal to the orthogonal dimension;

applying a result of the sequence of encoders to a sequence of decoders that includes a first decoder and one or more subsequent decoders, wherein a respective decoder in the one or more subsequent decoders performs convolution neural network processing of an up-sampled version of the ultrasound image from the first decoder, and the sequence of decoders forms a second dimension parallel to the first dimension;

producing probability segmentation maps from paired encoders and decoders in the sequence of encoders and the sequence of decoders; and

combining the probability segmentation maps to form a final probability segmentation output.

20. The computer system of claim 19 , wherein the memory further includes for causing a respective encoder in the sequence of encoders and a respective decoder in the sequence of decoders to receive input feature maps and generate multi-scale features and local features.