AI-based analysis of coronary angiograms
Systems and methods for performing a medical imaging analysis task are provided. One or more input medical images of a patient are received. The one or more input medical images are encoded into embeddings using a machine learning based encoder network. A medical imaging analysis task is performed based on the embeddings. Results of the medical imaging analysis task are output.
1 . A computer-implemented method comprising:
receiving a plurality of input medical images of a patient, the plurality of input medical images comprising a query image and one or more candidate images;
encoding the plurality of input medical images into embeddings using a machine learning based encoder network;
performing a medical imaging analysis task by determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network, wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network comprises:
ranking the embeddings for the one or more candidate images based on a similarity to the embeddings for the query image using the machine learning based encoder network; and
outputting results of the medical imaging analysis task.
2 . The computer-implemented method of claim 1 , further comprising:
extracting patches from the plurality of input medical images,
wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
3 . The computer-implemented method of claim 1 , wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network further comprises:
generating matching scores between the embeddings using the machine learning based encoder network.
4 . The computer-implemented method of claim 1 , wherein the plurality of input medical images comprises coronary angiography images of the patient.
5 . A computer-implemented method comprising:
receiving a plurality of input medical images of a temporal sequence of medical images of a patient;
extracting patches from the plurality of input medical images;
generating temporal relationships for the patches;
encoding the plurality of input medical images into embeddings using a machine learning based encoder network, wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches, with the temporal relationships, into the embeddings using a transformer-based encoder network;
performing a medical imaging analysis task based on the embeddings, wherein performing a medical imaging analysis task based on the embeddings comprises:
performing the medical imaging analysis task based on the embeddings using a transformer-based decoder network; and
outputting results of the medical imaging analysis task.
6 . The computer-implemented method of claim 5 , further comprising:
extracting patches from the plurality of input medical images,
wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
7 . The computer-implemented method of claim 5 , wherein the plurality of input medical images comprises coronary angiography images of the patient.
8 . An apparatus comprising:
means for receiving a plurality of input medical images of a temporal sequence of medical images of a patient;
means for extracting patches from the plurality of input medical images;
means for generating temporal relationships for the patches;
means for encoding the plurality of input medical images into embeddings using a machine learning based encoder network, wherein the means for encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
means for encoding the patches, with the temporal relationships, into the embeddings using a transformer-based encoder network;
means for performing a medical imaging analysis task based on the embeddings, wherein the means for performing a medical imaging analysis task based on the embeddings comprises:
means for performing the medical imaging analysis task based on the embeddings using a transformer-based decoder network; and
means for outputting results of the medical imaging analysis task.
9 . The apparatus of claim 8 , further comprising:
means for extracting patches from the plurality of input medical images,
wherein the means for encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
means for encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
10 . The apparatus of claim 8 , wherein the plurality of input medical images comprises coronary angiography images of the patient.
11 . An apparatus comprising:
means for receiving a plurality of input medical images of a patient, the plurality of input medical images comprising a query image and one or more candidate images;
means for encoding the plurality of input medical images into embeddings using a machine learning based encoder network;
means for performing a medical imaging analysis task by determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network, wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network comprises:
means for ranking the embeddings for the one or more candidate images based on a similarity to the embeddings for the query image using the machine learning based encoder network; and
means for outputting results of the medical imaging analysis task.
12 . The apparatus of claim 11 , wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network further comprises:
means for generating matching scores between the embeddings using the machine learning based encoder network.
13 . The apparatus of claim 11 , further comprising:
means for extracting patches from the plurality of input medical images,
wherein the means for encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
means for encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
14 . The apparatus of claim 11 , wherein the plurality of input medical images comprises coronary angiography images of the patient.
15 . A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving a plurality of input medical images of a patient, the plurality of input medical images comprising a query image and one or more candidate images;
encoding the plurality of input medical images into embeddings using a machine learning based encoder network;
performing a medical imaging analysis task by determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network, wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network comprises:
ranking the embeddings for the one or more candidate images based on a similarity to the embeddings for the query image using the machine learning based encoder network; and
outputting results of the medical imaging analysis task.
16 . The non-transitory computer readable medium of claim 15 , further comprising:
extracting patches from the plurality of input medical images,
wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
17 . The non-transitory computer readable medium of claim 15 , wherein the plurality of input medical images comprises coronary angiography images of the patient.
18 . A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving a plurality of input medical images of a temporal sequence of medical images of a patient;
extracting patches from the plurality of input medical images;
generating temporal relationships for the patches;
encoding the plurality of input medical images into embeddings using a machine learning based encoder network, wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches, with the temporal relationships, into the embeddings using a transformer-based encoder network;
performing a medical imaging analysis task based on the embeddings, wherein performing a medical imaging analysis task based on the embeddings comprises:
performing the medical imaging analysis task based on the embeddings using a transformer-based decoder network; and
outputting results of the medical imaging analysis task.
19 . The non-transitory computer readable medium of claim 18 , further comprising:
extracting patches from the plurality of input medical images,
wherein encoding the plurality of input medical images into embeddings using a machine learning based encoder network comprises:
encoding the patches into the embeddings based on one or more of spatial, temporal, anatomical, cardiac phase, and angulation information extracted from the plurality of input medical images using a transformer-based encoder network.
20 . The non-transitory computer readable medium of claim 18 , wherein determining at least one of the one or more candidate images as corresponding to the query image based on the embeddings using the machine learning based encoder network further comprises:
generating matching scores between the embeddings using the machine learning based encoder network.
21 . The non-transitory computer readable medium of claim 18 , wherein the plurality of input medical images comprises coronary angiography images of the patient.