Automatic bounding region annotation for localization of abnormalities

1. A method, in a data processing system comprising at least one processor and at least one memory, the method comprising:

automatically segmenting an input image of a subject into one or more segments, at least by processing the input image through a machine learning trained artificial neural network computer model trained to identify the one or more segments in input images, to generate a mask corresponding to recognized internal structures of the subject, wherein the machine learning trained artificial neural network computer model is trained based on normal input images in which no anomalies are present in the normal input images;

generating a template data structure based on an input set of annotated images, wherein the template data structure specifies standardized internal structure zones of the subject;

automatically registering the mask with the template data structure to generate a template registered mask identifying standardized internal structure zones present within the mask;

automatically generating bounding region annotations for each standardized internal structure zone present in the template registered mask;

automatically correlating the bounding region annotations with labels indicating whether or not the bounding region comprises an anomaly in the input image based on an analysis of a received natural language text description of the input image; and

automatically storing the bounding region annotations and labels in association with the input image to provide an automatically annotated and labeled image data structure.

2. The method of claim 1 , wherein the subject is a human being, the input image is a medical image of an anatomical region of the human being, and the internal structures of the subject are anatomical structures within a body of the human being.

3. The method of claim 1 , wherein generating the template data structure based on the input set of annotated images comprises:

receiving a set of annotated input images comprising boundary region annotations specifying internal structure zones of corresponding subjects in corresponding annotated input images of the set of annotated input images; and

performing a registration operation on the set of annotated input images to generate the template data structure specifying standardized internal structure zones based on the boundary region annotations in the set of annotated input images.

4. The method of claim 1 , wherein automatically generating bounding region annotations for each standardized internal structure zone present in the template registered mask comprises performing an affine transformation on the template registered mask to predict coordinates in the input image for the contour boundaries of the standardized internal structure zones identified in the template registered mask.

5. The method of claim 1 , wherein automatically generating bounding region annotations for each standardized internal structure zone present in the template registered mask comprises:

generating a first set of bounding region annotations based on the standardized internal structure zones present in the template registered mask; and

performing a bounding region correction operation on the first set of bounding region annotations based on an application of one or more heuristic algorithms to the first set of bounding region annotations to generate a second set of bounding region annotations, wherein the one or more heuristic algorithms correlate image features with bounding regions and ratios of dimensions of bounding regions.

6. The method of claim 1 , wherein automatically correlating the bounding region annotations with labels further comprises:

performing computer executed natural language processing on the natural language text description of the image to extract from the natural language text description of the input image a reference to an anomaly;

correlating the reference to the anomaly with a standardized internal structure zone based on an ontology or anomaly-to-location mapping rule data structure;

identifying a bounding region annotation in the generated bounding region annotations corresponding to the standardized internal structure zone correlated with the reference to the anomaly; and

associating a label of the anomaly with the identified bounding region.

7. The method of claim 1 , wherein the automatically annotated and labeled image data structure is stored as part of a training dataset for training an artificial intelligence computing system to perform image analysis operations on other input images, and wherein the method further comprises performing a machine learning training of the artificial intelligence computing system based on the training dataset to generate a trained artificial intelligence computing system trained to perform image analysis of the other input images.

8. The method of claim 1 , wherein:

the input image is one of a plurality of medical images of a portion of an anatomical structure of a plurality of human beings,

the plurality of medical images are obtained from a medical image dataset repository based on an input specifying a medical image modality and portion of anatomical structure of interest; and

the template data structure is one of a plurality of template data structures, and is specific to the modality and portion of anatomical structure of interest.

9. The method of claim 1 , wherein the automatically annotated and labeled image data structure is provided to an image viewer application, and wherein the image viewer application renders the automatically annotated and labeled image data structure as a graphical image in which the bounding region annotations and labels are displayed.

10. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:

automatically segment an input image of a subject into one or more segments, at least by processing the input image through a machine learning trained artificial neural network computer model trained to identify the one or more segments in input images, to generate a mask corresponding to recognized internal structures of the subject, wherein the machine learning trained artificial neural network computer model is trained based on normal input images in which no anomalies are present in the normal input images;

generate a template data structure based on an input set of annotated images, wherein the template data structure specifies standardized internal structure zones of the subject;

automatically register the mask with the template data structure to generate a template registered mask identifying standardized internal structure zones present within the mask;

automatically generate bounding region annotations for each standardized internal structure zone present in the template registered mask;

automatically correlate the bounding region annotations with labels indicating whether or not the bounding region comprises an anomaly in the input image based on an analysis of a received natural language text description of the input image; and

automatically store the bounding region annotations and labels in association with the input image to provide an automatically annotated and labeled image data structure.

11. The computer program product of claim 10 , wherein the subject is a human being, the input image is a medical image of an anatomical region of the human being, and the internal structures of the subject are anatomical structures within a body of the human being.

12. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to generate the template data structure based on the input set of annotated images at least by:

receiving a set of annotated input images comprising boundary region annotations specifying internal structure zones of corresponding subjects in corresponding annotated input images of the set of annotated input images; and

performing a registration operation on the set of annotated input images to generate the template data structure specifying standardized internal structure zones based on the boundary region annotations in the set of annotated input images.

13. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to automatically generate bounding region annotations for each standardized internal structure zone present in the template registered mask at least by performing an affine transformation on the template registered mask to predict coordinates in the input image for the contour boundaries of the standardized internal structure zones identified in the template registered mask.

14. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to automatically generate bounding region annotations for each standardized internal structure zone present in the template registered mask at least by:

generating a first set of bounding region annotations based on the standardized internal structure zones present in the template registered mask; and

performing a bounding region correction operation on the first set of bounding region annotations based on an application of one or more heuristic algorithms to the first set of bounding region annotations to generate a second set of bounding region annotations, wherein the one or more heuristic algorithms correlate image features with bounding regions and ratios of dimensions of bounding regions.

15. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to automatically correlate the bounding region annotations with labels further at least by:

performing computer executed natural language processing on the natural language text description of the image to extract from the natural language text description of the input image a reference to an anomaly;

correlating the reference to the anomaly with a standardized internal structure zone based on an ontology or anomaly-to-location mapping rule data structure;

identifying a bounding region annotation in the generated bounding region annotations corresponding to the standardized internal structure zone correlated with the reference to the anomaly; and

associating a label of the anomaly with the identified bounding region.

16. The computer program product of claim 10 , wherein the automatically annotated and labeled image data structure is stored as part of a training dataset for training an artificial intelligence computing system to perform image analysis operations on other input images, and wherein the computer readable program further causes the data processing system to perform a machine learning training of the artificial intelligence computing system based on the training dataset to generate a trained artificial intelligence computing system trained to perform image analysis of the other input images.

17. The computer program product of claim 10 , wherein:

the input image is one of a plurality of medical images of a portion of an anatomical structure of a plurality of human beings,

the plurality of medical images are obtained from a medical image dataset repository based on an input specifying a medical image modality and portion of anatomical structure of interest; and

the template data structure is one of a plurality of template data structures, and is specific to the modality and portion of anatomical structure of interest.

18. A data processing system comprising:

at least one processor; and

at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to:

automatically segment an input image of a subject into one or more segments, at least by processing the input image through a machine learning trained artificial neural network computer model trained to identify the one or more segments in input images, to generate a mask corresponding to recognized internal structures of the subject, wherein the machine learning trained artificial neural network computer model is trained based on normal input images in which no anomalies are present in the normal input images;

generate a template data structure based on an input set of annotated images, wherein the template data structure specifies standardized internal structure zones of the subject;

automatically register the mask with the template data structure to generate a template registered mask identifying standardized internal structure zones present within the mask;

automatically generate bounding region annotations for each standardized internal structure zone present in the template registered mask;

automatically correlate the bounding region annotations with labels indicating whether or not the bounding region comprises an anomaly in the input image based on an analysis of a received natural language text description of the input image; and

automatically store the bounding region annotations and labels in association with the input image to provide an automatically annotated and labeled image data structure.