Semantic segmentation for cancer detection in digital breast tomosynthesis

1. A method for segmenting a set of structures, including pathologic breast structures and regular breast structures, in a digital breast tomosynthesis (DBT) volume, comprising:

pre-classifying the DBT volume to provide a pre-classification result, the pre-classification result distinguishing between dense tissue and fatty tissue;

localizing a set of structures in the DBT volume by using at least one multi-stream deep image-to-image network, taking as input at least one image channel; and

segmenting the set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using at least one multi-stream deep image-to-image network, taking as input at least one image channel for providing a three-dimensional probabilistic map, wherein at least one of

the localizing is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue and

the segmenting is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue.

2. The method of claim 1 , wherein the localizing is executed by using an image-to-image multi-scale multi-streams deep learning network, taking as input at least one image channel to generate inverse distance maps to a center of each structure of interest.

3. The method of claim 2 , wherein the inverse distance maps are at least one of Euclidean, geodesic, are computed according to the center of each structure of interest, and are signed distance maps according to a boundary of the structure of interest.

4. The method of claim 1 , wherein for training the network, a Tversky loss function is used.

5. The method of claim 4 , wherein the Tversky loss function is a DICE loss function.

6. The method of claim 1 , wherein the DBT volume is forwarded as input to multiple resolution-specific streams and as an output of each of the resolution-specific streams a set of deconvolution layers is followed by activation layers and wherein different outputs of the resolution-specific streams are concatenated using a concatenation layer.

7. The method of claim 1 , wherein the localizing is based on generating a bounding box or an ellipsoid for the set of structures.

8. The method of claim 7 , wherein the segmenting is based on procedure, comprising:

pre-segmenting different types of structures, represented in the DBT volume, including regular structures and suspicious structures by way of filters and multi-stream deep image-to-image network;

structure-specific segmenting by generating, for each lesion bounding box, a candidate structure-specific prediction channel, encoding a probability for each voxel to belong to a specific type of structure, including a regular structure or a suspicious structure;

structure-specific refining of the structure-specific prediction channels by modeling neighborhood dependencies; and

aggregating the structure-specific prediction channels, once refined, by fusing the structure-specific prediction channels and probabilities and class probabilities by applying:

Pred=Σ c∈classes P ( c )·Pred c .

9. The method of claim 8 , wherein the segmenting further comprises:

calculating a class probability by classifying the candidate structure into a specific structure class by using a multi-stream deep image-to-image network.

10. The method of claim 1 , further comprising:

providing result data for each voxel of the DBT volume, and encoding a probability of belonging to a specific type of structure for breast cancer diagnosis.

11. A memory storing software code portions for executing the method of claim 1 , when the software code portions are run on a computer.

12. A non-transitory computer readable medium including program code for carrying out the method of claim 1 when the program code is run on a computer.

13. The method of claim 1 , wherein the localizing is based on a procedure, comprising:

extracting regions of interest;

generating bounding boxes;

adjusting the bounding boxes;

rejecting false positive structures; and

pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

14. The method of claim 1 , wherein the localizing is executed by using an image-to-image multi-scale multi-streams deep learning network, taking as input at least one image channel to generate inverse distance maps to a center of each structure of interest.

15. The method of claim 14 , wherein the inverse distance maps are at least one of Euclidean, geodesic, are computed according to the center of each structure of interest, and are signed distance maps according to a boundary of the structure of interest.

16. A method for segmenting a set of structures, including pathologic breast structures and regular breast structures, in a digital breast tomosynthesis (DBT) volume, comprising:

localizing a set of structures in the DBT volume by using at least one multi-stream deep image-to-image network, taking as input at least one image channel; and

segmenting the set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using at least one multi-stream deep image-to-image network, taking as input at least one image channel for providing a three-dimensional probabilistic map, wherein the localizing is based on a procedure, comprising:

extracting regions of interest;

generating bounding boxes;

adjusting the bounding boxes;

rejecting false positive structures; and

pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

17. A non-transitory computer readable medium including program code for carrying out the method of claim 16 when the program code is run on a computer.

18. A memory storing software code portions for executing the method of claim 16 , when the software code portions are run on a computer.

19. An apparatus for segmenting different types of structures, including cancerous lesions and regular structures, in a digital breast tomosynthesis (DBT) volume, comprising:

a localizer to localize a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and

a segmenter to segment the localized set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using a deep convolutional neural network for providing a three-dimensional probabilistic map, wherein localizer is configured to localize the set of structures based on a procedure, comprising:

extracting regions of interest;

generating bounding boxes;

adjusting the bounding boxes;

rejecting false positive structures; and

pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

20. The apparatus of claim 19 , further comprising:

a pre-classifier, configured to provide a pre-classification result, the pre-classification result differentiating in the DBT volume between dense tissue and fatty tissue, wherein the pre-classification result is forwarded to at least one of the localizer and the segmenter, for specific processing of dense tissue and fatty tissue.

21. The apparatus of claim of claim 20 , wherein at least one of

the specific processing includes localizing, by the localizer, is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue; and

the specific processing includes segmenting, by the segmenter, is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue.

22. An apparatus for segmenting different types of structures, including cancerous lesions and regular structures, in a digital breast tomosynthesis (DBT) volume, comprising:

a pre-classifier to pre-classify the DBT volume to provide a pre-classification result, the pre-classification result distinguishing between dense tissue and fatty tissue;

localizer to localize a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and

a segmenter to segment the localized set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using a deep convolutional neural network for providing a three-dimensional probabilistic map, wherein at least one of

the localizer is configured to localize the set of structures based on the pre-classification result and is configured to execute with a specific model for dense tissue and with a specific model for fatty tissue and

the segmenter is configured to segment the localized set based on the pre-classification result and is configured to execute with a specific model for dense tissue and with a specific model for fatty tissue.

23. The apparatus of claim 22 , wherein the localizer is configured to localize the set of structures based on a procedure, comprising:

extracting regions of interest;

generating bounding boxes;

adjusting the bounding boxes;

rejecting false positive structures; and

pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.