Methods and systems for automated follow-up reading of medical image data

1 . A method for generating display data of a medical image data set, the method comprising:

receiving a target medical image series of a patient at a first point in time;

determining a target body region represented by the target medical image series;

selecting a reference medical image study from a plurality of candidate medical image studies based on a comparison of the target body region with a plurality of candidate body regions, wherein each of the plurality of candidate body regions corresponds to one of the plurality of candidate medical image studies and each candidate medical image study comprises a plurality of candidate medical image series of the patient at a second point in time, the selecting the reference medical image study from the plurality of candidate medical image studies including

identifying, from the plurality of candidate medical image studies, a plurality of relevant medical image studies based on the comparison of the target body region with the plurality of candidate body regions,

providing an indication of the plurality of relevant medical image studies to a user via a user interface,

receiving a user selection via the user interface, the user selection indicating at least one of the plurality of relevant medical image studies, and

selecting the at least one of the plurality of relevant medical image studies as the reference medical image study;

selecting, from the plurality of candidate medical image series of the reference medical image study, a reference medical image series based on a degree of comparability with the target medical image series;

performing a registration of the target medical image series and the reference medical image series; and

generating display data to cause a display device to display a rendering of the reference medical image series based on the registration.

2 . The method of claim 1 , wherein

the target medical image series and each of the plurality of candidate medical image studies are respectively associated to one or more attributes each having an attribute value comprising a text string indicating content of the target medical image series or context of a candidate medical image study of the plurality of candidate medical image studies, and at least one of,

(i) the determining the target body region includes,

obtaining one or more text strings of the target medical image series, and

inputting the one or more text strings of the target medical image series into a trained machine learning model, the trained machine learning model trained to output a body region based on an input of the one or more text strings, and obtaining an output from the trained machine learning model to determine the body region represented by the target medical image series; or

(ii) at least one of the plurality of candidate body regions is determined by,

obtaining one or more text strings of the candidate medical image study, and

inputting the one or more text strings of the candidate medical image study into the trained machine learning model and obtaining an output from the trained machine learning model to determine the at least one of the plurality of candidate body regions.

3 . The method of claim 2 , wherein the trained machine learning model is a trained neural network comprising a trained character-based neural network configured to take as input individual characters of the one or more text strings of the target medical image series or the one or more text strings of the candidate medical image study, and inputting the one or more text strings of the target medical image series or the one or more text strings of the candidate medical image study into the trained neural network comprises inputting individual characters of the one or more text strings of the target medical image series or the one or more text strings of the candidate medical image study into the trained character-based neural network.

4 . The method of claim 1 , wherein

each of the target medical image series and the plurality of candidate medical image series are associated with one or more attributes each having an attribute value indicative of an imaging parameter used to capture the target medical image series or a candidate medical image series of the plurality of candidate medical image series; and

the degree of comparability is based on determining a correspondence of one or more attribute values between the target medical image series and a respective one of the plurality of candidate medical image series.

5 . The method of claim 1 , wherein determining the degree of comparability comprises:

obtaining, for the target medical image series, a first feature vector;

obtaining, for each candidate medical image series comprised in the reference medical image study, a second feature vector; and

determining a comparability metric indicative of the degree of comparability between the first feature vector and the second feature vector.

6 . The method of claim 5 , wherein

the first feature vector includes an indication of an imaging modality of the target medical image series and the second feature vector includes an indication of an imaging modality of a candidate medical image data series of the plurality of candidate medical image series; and

the comparability metric comprises an imaging modality relevance score between the imaging modality of the target medical image series and the imaging modality of the candidate medical image data series.

7 . The method of claim 1 , further comprising:

obtaining a region of interest of the target medical image series, wherein

the reference medical image series depicts a reference image volume, the method further comprising obtaining a plurality of candidate slices respectively depicting a certain section of the reference image volume,

the performing the registration comprises identifying, from the plurality of candidate slices, at least one reference slice based on degrees of similarity between image data comprised in the region of interest and individual candidate slices of the plurality of candidate slices, and

the generating the display data causes the display device to display a rendering of the at least one reference slice.

8 . The method of claim 7 , further comprising:

extracting an image descriptor from the target medical image series; and

extracting a corresponding image descriptor from each of the plurality of candidate slices,

wherein the degrees of similarity are respectively based on a comparison between the image descriptor of the target medical image series and the image descriptor of each of the plurality of candidate slices.

9 . The method of claim 7 , wherein

the identifying the at least one reference slice comprises applying a trained function on the target medical image series and the reference medical image series, and

the trained function is adapted to determine degrees of similarities between two-dimensional medical images.

10 . The method of claim 1 , further comprising:

obtaining a region of interest of the target medical image series, wherein the performing the registration comprises:

generating a first local image descriptor based on image data of the region of interest;

generating a second local image descriptor for each of a plurality of candidate locations in the reference medical image series, each second local image descriptor being generated based on image data of the reference medical image series located relative to a respective candidate location of the plurality of candidate locations;

calculating local image similarity metrics indicating a degree of similarity between the first local image descriptor and the second local image descriptor for each of the plurality of candidate locations;

selecting a candidate location from among the plurality of candidate locations based on the local image similarity metrics; and

determining a location corresponding to the region of interest in the reference medical image series based on the candidate location, wherein

the generating the display data includes generating the rendering based on image data of the reference medical image series relative to the candidate location.

11 . The method of claim 1 , wherein

the identifying the plurality of relevant medical image studies comprises determining, for each relevant medical image study of the plurality of relevant medical image studies, a degree of conformance with the target medical image series based on the comparison of the target body region with the plurality of candidate body regions; and

the providing the indication of the plurality of relevant medical image studies provides the degree of conformance for each of the plurality of relevant medical image studies to the user via the user interface, wherein

the degree of conformance is based on an anatomical overlap between the target medical image series and the plurality of relevant medical image studies based on the comparison of the target body region and the plurality of candidate body regions.

12 . The method of claim 1 , further comprising:

obtaining a region of interest of the target medical image series, wherein the reference medical image study comprises one or more annotations corresponding to the reference medical image series and the method further comprises:

obtaining, from the one or more annotations, a reference annotation relevant for at least one of the target medical image series or the region of interest, and

annotating the target medical image series or the region of interest with the reference annotation, the reference annotation including one or more first words.

13 . The method of claim 12 , further comprising:

obtaining a medical report associated with the reference medical image study;

obtaining one or more sections of text of the medical report, each section comprising one or more second words;

for each of the one or more sections and the reference annotation, comparing one or more of the one or more second words to the one or more first words of the reference annotation to identify a match; and

associating the reference annotation with at least one of the one or more sections based on the match.

14 . The method of claim 1 , further comprising:

identifying the user to be provided with the display data; and

obtaining one or more prior actions of the user, the one or more prior actions being directed to at least one of,

a study selection action of selecting a reference medical image study from a plurality of candidate medical image studies, or

a series selecting action of selecting a reference medical image series from a plurality of candidate medical image series of a medical image study;

wherein at least one of the selecting the reference medical image study or the selecting the reference medical image series are based on the one or more prior actions.

15 . A system for supporting evaluation of a target medical image series of a patient acquired at a first point in time, the system comprising:

an interface unit configured to provide a rendering of the target medical image series to a user; and

a computing unit configured to cause the system to,

determine a target body region represented by the target medical image series,

select a reference medical image study from a plurality of candidate medical image studies based on a comparison of the target body region with a plurality of candidate body regions, wherein each of the plurality of candidate body regions corresponds to one of the plurality of candidate medical image studies and each candidate medical image study comprises a plurality of candidate medical image series of the patient at a second point in time, selecting the reference medical image study from the plurality of candidate medical image studies including

identifying, from the plurality of candidate medical image studies, a plurality of relevant medical image studies based on the comparison of the target body region with the plurality of candidate body regions,

providing an indication of the plurality of relevant medical image studies to the user via the interface unit,

receiving a user selection via the interface unit, the user selection indicating at least one of the plurality of relevant medical image studies, and

selecting the at least one of the plurality of relevant medical image studies as the reference medical image study,

select, from the plurality of candidate medical image series of the reference medical image study, a reference medical image series based on a degree of comparability with the target medical image series,

perform a registration of the target medical image series and the reference medical image series, and

generate display data to cause the interface unit to display a rendering of the reference medical image series based on the registration.

16 . A computer program product comprising program elements that, when executed by a computing unit of a system for supporting evaluation of a medical image series, cause the system to perform the method of claim 1 .

17 . A non-transitory computer-readable medium having program elements that, when executed by a computing unit of a system for supporting evaluation of a medical image series, cause the system to perform the method of claim 1 .