Development of medical imaging AI analysis algorithms leveraging image segmentation
A medical image may be anatomically segmented, such as by an automated segmentation model, before presentation to a reading physician, who can then step through the anatomical segments which may have already been associated with an initial estimate of whether there is a finding. Based on indications provided by the reading physician, the system may optimize feature detection algorithms of segment-specific diagnostic models that are configured to identify characteristics of medical images of the specific anatomical segments.
1 . A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
for each of a plurality of medical images:
accessing the medical image;
applying a segmentation algorithm to the medical image to determine a plurality of anatomical segments indicated in the medical image, wherein each of the anatomical segments is associated with an anatomical component or an anatomical system;
for each of a plurality of users:
displaying the medical image on a display device of a user;
for each of the plurality of anatomical segments, displaying user interface controls usable to indicate whether a finding is observed by the user;
receiving, via the user interface controls, user input indicating that either:
the anatomical segment shows a finding, or
the anatomical segment does not show a finding;
storing, for each anatomical segment, metadata comprising: (i) segment boundaries, (ii) anatomical identifiers, and (iii) user-indicated finding status, in a training data set; and
for each of the plurality of anatomical segments:
training a segment-specific diagnostic model to detect findings in the anatomical segment of medical images not included in the plurality of medical images, wherein the segment-specific diagnostic model:
accesses the training data set to identify a first set of medical images with the anatomical segment identified as no finding and a second set of medical images with the anatomical segment identified as finding detected, and
trains the segment-specific diagnostic model based on differences between the first and second sets of medical images.
2 . The method of claim 1 , further comprising:
accessing a medical image not included in the plurality of medical images;
applying the segmentation algorithm to the medical image to determine the plurality of anatomical segments of patient anatomy indicated in the medical image; and
for each of the anatomical segments identified in the medical image:
selecting a segment-specific diagnostic model associated with the anatomical segment;
applying the segment-specific diagnostic model to at least portions of the medical image associated with the anatomical segment, wherein the segment-specific diagnostic model provides an indication of whether the anatomical segment is more likely normal or abnormal.
3 . The method of claim 2 , further comprising:
displaying, in a user interface, an indication of any anatomical segments with findings.
4 . The method of claim 2 , further comprising:
prepopulating an itemized report with the indications of findings and associated anatomical segments.
5 . The method of claim 4 , wherein the anatomical segments associated with findings are indicated in the report.
6 . The method of claim 4 , wherein the anatomical segments associated with findings include a link or reference to a medical image associated with the finding.
7 . The method of claim 2 , wherein the segment-specific diagnostic model determines indications of finding vs no finding based on one or more of an indication or a clinical question.
8 . The method of claim 1 , wherein the anatomical components include one or more of: lungs, vasculature, cardiac, mediastinum, pleura, or bone.
9 . The method of claim 1 , wherein the anatomical systems include one or more of: digestive system, musculoskeletal system, nervous system, endocrine system, reproductive system, urinary system, or immune system.
10 . The method of claim 1 , wherein the anatomical segments are associated with corresponding sections of a medical report.
11 . The method of claim 1 , wherein the plurality of anatomical segments are stored in data structure in association with a type of the medical image.
12 . The method of claim 1 , further comprising:
wherein the segmentation algorithm accesses a medical report associated with the medical image to determine whether there is a finding or no finding for each of the anatomical segments indicating in the medical report.
13 . The method of claim 12 , wherein said determining whether there is a finding or no finding for each of the anatomical segments indicating in the medical report is based at least partly on natural language processing of textual descriptions associated with respective anatomical segments.
14 . The method of claim 1 , wherein the segment-specific diagnostic models are trained using itemized reports wherein at least one report item corresponds to an anatomical segment defined in an image.
15 . The method of claim 1 , wherein the segment-specific diagnostic models are trained using one or more artificial intelligence algorithms to classify items in a medical report as finding or no finding.
16 . A computing system comprising:
a hardware computer processor; and
a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising:
accessing the medical image;
applying a segmentation algorithm to the medical image to determine a plurality of anatomical segments indicated in the medical image, wherein each of the anatomical segments is associated with an anatomical component or an anatomical system;
for each of a plurality of users:
displaying the medical image on a display device of a user;
for each of the plurality of anatomical segments, displaying user interface controls usable to indicate whether a finding is observed by the user;
receiving, via the user interface controls, user input indicating that either:
the anatomical segment shows a finding, or
the anatomical segment does not show a finding;
storing, for each anatomical segment, metadata comprising: (i) segment boundaries, (ii) anatomical identifiers, and (iii) user-indicated finding status, in a training data set; and
for each of the plurality of anatomical segments:
training a segment-specific diagnostic model to detect findings in the anatomical segment of medical images not included in the plurality of medical images, wherein the segment-specific diagnostic model:
accesses the training data set to identify a first set of medical images with the anatomical segment identified as no finding and a second set of medical images with the anatomical segment identified as finding detected, and
trains the segment-specific diagnostic model based on differences between the first and second sets of medical images.