Method and device for stratified image segmentation

1. A method for stratified image segmentation, comprising:

obtaining a three-dimensional (3D) image data set representative of a region comprising at least three levels of objects;

generating a first segmentation result indicating boundaries of anchor-level objects in the region based on a first neural network (NN) model corresponding to the anchor-level objects;

generating a second segmentation result indicating boundaries of mid-level objects in the region based on the first segmentation result and a second NN model corresponding to the mid-level objects; and

generating a third segmentation result indicating small-level objects in the region based on the first segmentation result, a third NN model corresponding to the small-level objects, and cropped regions corresponding to the small-level objects, wherein generating the third segmentation result comprises: for each small-level object,

detecting a center region of the small-level object based on the first segmentation result and the 3D image data set;

determining a volume of interest (VOI) within the 3D image data set based on the center region of the small-level object; and

segmenting the small-level object from the VOI using the third NN model.

2. The method according to claim 1 , wherein the image data, set comprises a plurality of voxels, and each voxel corresponds to a vector valued mask that provides a probability of the voxel belonging to each of the multiple objects according to the first, second, and third segmentation results.

3. The method according to claim 1 , wherein generating the second segmentation result comprises:

combining location references and semantically-based cues from the first segmentation result to generate the second segmentation result.

4. The method according to claim 1 , wherein the center region of the small-level object is detected based on location references provided by the first segmentation result and a heat map regression method based on a fourth NN model.

5. The method according to claim 1 , wherein at least one of the first, second, and third NN models is a progressive holistically-nested network model.

6. The method according to claim 1 , wherein the first, second, and third NN models are identified through differentiable neural architecture search (NAS).

7. The method according to claim 6 , wherein a search space for the NAS includes 2D, 3D, and Pseudo-3D (P3D) convolutions for a first kernel size, and 2D, 3D, and Pseudo-3D (P3D) convolutions for a second kernel size.

8. The method according to claim 1 , wherein the 3D image data set is a data set of radiotherapy computed tomography (RTCT) images.

9. The method according to claim 8 , wherein the RTCT in ages are scanned from a head and neck region.

10. A device for stratified image segmentation, comprising:

a memory; and

a processor coupled to the memory and configured to:

obtain a three-dimensional (3D) image data set representative of a region comprising at least three levels of objects;

generate a first segmentation result indicating boundaries of anchor-level objects in the region based on a first neural network (NN) model corresponding to the anchor-level objects;

generate a second segmentation result indicating boundaries of mid-level objects in the region based on the first segmentation result and a second NN model corresponding to the mid-level objects; and

generate a third segmentation result indicating small-level objects in the region based on the first segmentation result, a third NN model corresponding to the small-level objects, and cropped regions corresponding to the small-level objects, wherein the processor is further configured to: for each small-level object,

detect a center region of the small-level object based on the first segmentation result and the 3D image data set;

determine a volume of interest (VOI) within the 3D image data set based on the center region of the small-level object; and

segment the small-level object from the VOI using the third NN model.

11. The device according to claim 10 , wherein the image data set comprises a plurality of voxels, and each voxel corresponds to a vector valued mask that provides a probability of the voxel belonging to each of the multiple objects according to the first, second, and third segmentation results.

12. The device according to claim 10 , wherein for generating the second segmentation result, the processor is further configured to:

combine location references and semantically-based cues from the first segmentation result to generate the second segmentation result.

13. The device according to claim 10 , wherein the center region of the all-level object is detected based on location references provided by the first segmentation result and a heat map regression method based on a fourth NN model.

14. The device according to claim 10 , wherein at least one of the first, second, and third NN models is a progressive holistically-nested network model.

15. The device according to claim 10 , wherein the first, second, and third NN models are identified through differentiable neural architecture search (NAS).

16. The device according to claim 15 , wherein a search space for the NAS includes 2D, 3D, and Pseudo-3D (P3D) convolutions for a first kernel size, and 2D, 3D, and Pseudo-3D (P3D) convolutions for a second kernel size.

17. The device according to claim 10 , wherein the 3D image data set is a data set of radiotherapy computed tomography (RTCT) images.

18. A non-transitory computer readable storage medium, storing computer instructions that, when being executed by a processor, causing the processor to perform:

obtaining a three-dimensional (3D) image data set representative of a region comprising at least three levels of objects;

generating a first segmentation result indicating boundaries of anchor-level objects in the region based on a first neural network (NN) model corresponding to the anchor-level objects;

generating a second segmentation result indicating boundaries of objects in the region based on the first segmentation result and a second NN model corresponding to the objects; and

generating a third segmentation result indicating small-level objects in the region based on the first segmentation result, a third NN model corresponding to the small-level objects, and cropped regions corresponding to the small-level objects, wherein generating the third segmentation result comprises: for each small-level object,

detecting a center region of the small-level object based on the first segmentation result and the 3D image data set;

determining a volume of interest (VOI) within the 3D image data set based on the center region of the s all-level object; and

segmenting the small-level object from the VOI using the third NN model.