Technique for multi-modality medical image clinical workflow guidance

1 . A computer-implemented method for multi-modality medical image clinical workflow guidance, the method comprising:

receiving a user input in relation to a first medical image data set acquired by a first medical imaging modality, the first medical imaging modality comprising computed tomography, the user input comprising at least one of a medical measurement and a marked region of interest on the first medical image data set;

generating, by a semantic image understanding algorithm, extended metadata for the first medical image data set and at least one second medical image data set acquired by transthoracic echocardiography, the extended metadata comprising at least a cardiac phase identification and an image quality assessment score;

assessing, by a clinical-concept-to-medical-image linking algorithm, the received user input by mapping the user input to a clinical concept within a textual clinical concept algorithm using the extended metadata, wherein the clinical-concept-to-medical-image linking algorithm comprises the textual clinical concept algorithm and the semantic image understanding algorithm connected by iterative cycles of prompts and queries; and

outputting an indication of a clinical workflow guidance in relation to the at least one second medical image data set, wherein the indication comprises an automatic anatomy alignment of the first medical image data set and the at least one second medical image data set for side-by-side display on a user interface based on the clinical concept and the extended metadata.

2 . The computer-implemented method according to claim 1 , wherein the first medical image data set and/or the at least one second medical image data set comprises a two-dimensional and/or a three-dimensional image data set.

3 . The computer-implemented method according to claim 1 , wherein the multi-modality medical imaging comprises cardiac and/or cardiovascular imaging.

4 . The computer-implemented method according to claim 1 , wherein the semantic image understanding algorithm generates metadata in relation to the first medical image data set and/or the at least one second medical image data set, wherein the metadata are indicative of at least one of:

a view classification;

one or more anatomical landmarks and/or anatomical structures;

a zoom level;

a cardiac phase identification;

a contrast enhancement;

an image quality assessment; and/or

a score for use cases.

5 . The computer-implemented method according to claim 4 , wherein the metadata are indicative of the score for use cases, and wherein the score is based on an image quality assessment at a predetermined phase within the cardiac cycle and/or anatomical structures comprised in the corresponding medical image data set.

6 . The computer-implemented method according to claim 1 , wherein the clinical-concept-to-medical-image linking algorithm comprises at least one trained artificial intelligence model.

7 . The computer-implemented method according to claim 6 , wherein the at least one trained artificial intelligence model comprises two jointly trained artificial intelligence models.

8 . The computer-implemented method according to claim 6 , wherein the at least one trained AI model is trained based on training data comprising:

input training data, the input training data comprising medical image data sets from multiple medical imaging modalities comprising the first medical imaging modality and the at least one second medical imaging modality, textual guidelines in relation to the clinical workflow, and functional couplings among anatomical structures and/or anatomical views in the medical image data sets; and

output training data, the output training data comprising results of user interactions in relation to the input medical image data sets.

9 . The computer-implemented method according to claim 1 , wherein the user input is received by a user interface.

10 . The computer-implemented method according to claim 1 , further comprising:

accessing a storage for retrieving the at least one second medical image data set from a medical information system comprising at least one database for imaging and clinical data.

11 . A system for multi-modality medical image clinical workflow guidance, the system comprising:

a first interface configured for receiving a user input in relation to a first medical image data set acquired by a first medical imaging modality, the first medical imaging modality comprising computed tomography, the user input comprising at least one of a medical measurement and a marked region of interest on the first medical image data set;

a processor configured to generate using a semantic image understanding algorithm, extended metadata for the first medical image data set and at least one second medical image data set acquired by transthoracic echocardiography, the extended metadata comprising at least a cardiac phase identification and an image quality assessment score, the processor further configured to assess using a clinical-concept-to-medical-image linking algorithm the received user input by mapping the user input to a clinical concept within a textual clinical concept algorithm using the extended metadata, wherein the clinical-concept-to-medical-image linking algorithm comprises the textual clinical concept algorithm and the semantic image understanding algorithm connected by iterative cycles of prompts and queries; and

a second interface configured to output an indication of a clinical workflow guidance in relation to the at least one second medical image data set, wherein the indication comprises an automatic anatomy alignment of the first medical image data set and the at least one second medical image data set for side-by-side display on a user interface based on the clinical concept and the extended metadata.

12 . The system according to claim 11 , further comprising:

a memory configured for storing the at least one second medical image data set, wherein the storage is accessible for retrieval of the at least one second medical image data set from a medical information system comprising at least one database for imaging and clinical data; and

a user interface configured to forward one or more user inputs to the first interface and to output one or more indications, received from the second interface, to the user.

13 . The system according to claim 12 , further comprising a third interface configured to access the memory.

14 . A non-transitory computer-readable medium on which instructions are stored that can be read and executed by a computer for multi-modality medical image clinical workflow guidance, the instructions being to:

receive a user input in relation to a first medical image data set acquired by a first medical imaging modality, the first medical imaging modality comprising computed tomography, the user input comprising at least one of a medical measurement and a marked region of interest on the first medical image data set;

generate, by a semantic image understanding algorithm, extended metadata for the first medical image data set and at least one second medical image data set acquired by transthoracic echocardiography, the extended metadata comprising at least a cardiac phase identification and an image quality assessment score;

assess, by a clinical-concept-to-medical-image linking algorithm, the received user input by mapping the user input to a clinical concept within a textual clinical concept algorithm using the extended metadata, wherein the clinical-concept-to-medical-image linking algorithm comprises the textual clinical concept algorithm and the semantic image understanding algorithm connected by iterative cycles of prompts and queries; and

output an indication of a clinical workflow guidance in relation to the at least one second medical image data set, wherein the indication comprises an automatic anatomy alignment of the first medical image data set and the at least one second medical image data set for side-by-side display on a user interface based on the clinical concept and the extended metadata.