Explainable deep learning camera-agnostic diagnosis of obstructive coronary artery disease
A deep learning model for the detection of obstructive coronary artery disease (CAD) can take a set of polar maps and patient information as input, then output obstructive CAD scoring data, such as probabilities of obstructive CAD associated with various cardiac territories, as well as an attention map and a CAD scoring map. The model can operate agnostic of camera type used to capture the set of polar maps. The attention map indicates regions of the polar maps important to the deep learning process for that particular set of polar maps. The attention map and obstructive CAD scoring data can be used to generate a CAD scoring map showing CAD probability by segment on a standard 17-segment model of a left ventricle. The attention map and/or CAD scoring map can act as easily explainable tools for interpreting the results of a myocardial perfusion imaging study.
1 . A system for detecting coronary obstructions, comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
creating a first set of training data associated with a myocardial perfusion imaging study comprising data collected using a first type of camera and a second set of training data associated with a myocardial perfusion imaging study collected using a second type of camera wherein the second type of camera is a different type of camera than the first type of camera;
training a deep learning model using the first set of training data and the second set of training data, wherein the deep learning model comprises an input layer, at least one convolution layer, at least one pooling layer, and at least one fully connected layer;
receiving input data associated with a myocardial perfusion imaging study of a patient, wherein the input data comprises a set of polar maps generated by a camera and patient information, wherein the set of polar maps comprises a perfusion polar map, a motion polar map, and a thickening polar map, and wherein the patient information comprises at least one selected from the set consisting of sex information, age information, and cardiac volume information;
providing the input data excluding the type of camera used to generate the set of polar maps to the deep learning model
generating obstructive coronary artery disease (CAD) scoring data in response to providing the input data to the deep learning model, wherein the obstructive CAD scoring data is indicative of i) a general probability for obstructive CAD, ii) a probability for obstructive CAD in a left anterior descending artery (LAD) territory, iii) a probability for obstructive CAD in a left circumflex artery (LCx) territory, iv) and a probability for obstructive CAD in a right coronary artery (RCA) territory, or v) any combination of i-iv;
generating an attention map in response to providing the input data to the deep learning model, wherein the attention map is indicative of regions of importance associated with the obstructive CAD scoring data; and
presenting, on a display device, the obstructive CAD scoring data and the attention map.
2 . The system of claim 1 , wherein generating the attention map comprises applying gradients of a predicted vessel to a final convolution layer of the at least one convolution layer to produce the attention map, wherein the regions of importance are indicative of regions of one or more polar maps of the set of polar maps with high informational weight in the deep learning model.
3 . The system of claim 1 , further comprising:
generating a coronary artery disease probability map based on the attention map, the coronary artery disease probability map comprising a set of segments indicative of segments of a left ventricle, wherein generating the coronary artery disease probability map comprises determining severity measurements for each segment of the set of segments using the attention map and the obstructive CAD scoring data; and
presenting, on the display device, the coronary artery disease probability map.
4 . The system of claim 1 , wherein the first set of training data is collected using a vacuum-tube-photomultiplier-based camera and the second set of training data is collected using a cadmium-zinc-telluride-based camera.
5 . The system of claim 1 , wherein providing the input data to the deep learning model comprises:
providing the set of polar maps to the input layer of the deep learning model; and
providing the patient information to the at least one fully connected layer of the deep learning model.
6 . The system of claim 5 , wherein the at least one fully connected layer includes a fully connected output layer, and wherein providing the patient information to the at least one fully connected layer comprises providing the patient information to the fully connected output layer.
7 . The system of claim 1 , wherein providing the set of polar maps to the deep learning model comprises providing the set of polar maps without pre-defined coronary territories.
8 . The system of claim 1 , further comprising a scanner coupled to the one or more processors for performing a myocardial imaging study, the scanner comprising a scanner camera wherein the set of polar maps comprise the myocardial imaging study and is generated by the scanner camera.
9 . The system of claim 1 , wherein the attention map further comprises:
a polar map from the set of polar maps; and
a coarse localization map overlayed onto the polar map from the set of polar maps, wherein the coarse localization map identifies regions of the polar maps from the set of polar maps contributing to at least one of the probability for obstructive CAD in the LAD territory, the probability for obstructive CAD in the LCx territory, and the probability for obstructive CAD in the RCA territory.
10 . A computer-implemented method for detecting coronary obstructions, comprising:
creating a first set of training data associated with a myocardial perfusion imaging study comprising data collected using a first type of camera and a second set of training data associated with a myocardial perfusion imaging study collected using a second type of camera wherein the second type of camera is a different type of camera than the first type of camera;
training a deep learning model using the first set of training data and the second set of training data, wherein the deep learning model comprises an input layer, at least one convolution layer, at least one pooling layer, and at least one fully connected layer;
receiving input data associated with a myocardial perfusion imaging study of a patient, wherein the input data comprises a set of polar maps generated by a camera and patient information, wherein the set of polar maps comprises a perfusion polar map, a motion polar map, and a thickening polar map, and wherein the patient information comprises at least one selected from the set consisting of sex information, age information, and cardiac volume information;
providing the input data excluding the type of camera used to generate the set of polar maps to the deep learning model
generating obstructive coronary artery disease (CAD) scoring data in response to providing the input data to the deep learning model, wherein the obstructive CAD scoring data is indicative of i) a general probability for obstructive CAD, ii) a probability for obstructive CAD in a left anterior descending artery (LAD) territory, iii) a probability for obstructive CAD in a left circumflex artery (LCx) territory, iv) and a probability for obstructive CAD in a right coronary artery (RCA) territory, or v) any combination of i-iv;
generating an attention map in response to providing the input data to the deep learning model, wherein the attention map is indicative of regions of importance associated with the obstructive CAD scoring data; and
presenting, on a display device, the obstructive CAD scoring data and the attention map.
11 . The method of claim 10 , wherein generating the attention map comprises applying gradients of a predicted vessel to a final convolution layer of the at least one convolution layer to produce the attention map, wherein the regions of importance are indicative of regions of one or more polar maps of the set of polar maps with high informational weight in the deep learning model.
12 . The method of claim 10 , further comprising generating a coronary artery disease probability map based on the attention map, the coronary artery disease probability map comprising a set of segments indicative of segments of a left ventricle, wherein generating the coronary artery disease probability map comprises determining severity measurements for each segment of the set of segments using the attention map and the obstructive CAD scoring data.
13 . The method of claim 10 , wherein the first set of training data is collected using a vacuum-tube-photomultiplier-based camera and the second set of training data is collected using a cadmium-zinc-telluride-based camera.
14 . The method of claim 10 , wherein providing the input data to the deep learning model comprises:
providing the set of polar maps to the input layer of the deep learning model; and
providing the patient information to the at least one fully connected layer of the deep learning model.
15 . The method of claim 14 , wherein the at least one fully connected layer includes a fully connected output layer, and wherein providing the patient information to the at least one fully connected layer comprises providing the patient information to the fully connected output layer.
16 . The method of claim 10 , wherein providing the set of polar maps to the deep learning model comprises providing the set of polar maps without pre-defined coronary territories.
17 . A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:
creating a first set of training data associated with a myocardial perfusion imaging study comprising data collected using a first type of camera and a second set of training data associated with a myocardial perfusion imaging study collected using a second type of camera wherein the second type of camera is a different type of camera than the first type of camera;
training a deep learning model using the first set of training data and the second set of training data, wherein the deep learning model comprises an input layer, at least one convolution layer, at least one pooling layer, and at least one fully connected layer;
receiving input data associated with a myocardial perfusion imaging study of a patient, wherein the input data comprises a set of polar maps generated by a camera and patient information, wherein the set of polar maps comprises a perfusion polar map, a motion polar map, and a thickening polar map, and wherein the patient information comprises at least one selected from the set consisting of sex information, age information, and cardiac volume information;
providing the input data excluding the type of camera used to generate the set of polar maps to the deep learning model
generating obstructive coronary artery disease (CAD) scoring data in response to providing the input data to the deep learning model, wherein the obstructive CAD scoring data is indicative of i) a general probability for obstructive CAD, ii) a probability for obstructive CAD in a left anterior descending artery (LAD) territory, iii) a probability for obstructive CAD in a left circumflex artery (LCx) territory, iv) and a probability for obstructive CAD in a right coronary artery (RCA) territory, or v) any combination of i-iv;
generating an attention map in response to providing the input data to the deep learning model, wherein the attention map is indicative of regions of importance associated with the obstructive CAD scoring data; and
presenting, on a display device, the obstructive CAD scoring data and the attention map.
18 . The computer-program product of claim 17 , wherein generating the attention map comprises applying gradients of a predicted vessel to a final convolution layer of the at least one convolution layer to produce the attention map, wherein the regions of importance are indicative of regions of one or more polar maps of the set of polar maps with high informational weight in the deep learning model.
19 . The computer-program product of claim 17 , further comprising generating a coronary artery disease probability map based on the attention map, the coronary artery disease probability map comprising a set of segments indicative of segments of a left ventricle, wherein generating the coronary artery disease probability map comprises determining severity measurements for each segment of the set of segments using the attention map and the obstructive CAD scoring data.
20 . The computer-program product of claim 17 , wherein the first set of training data is collected using a vacuum-tube-photomultiplier-based camera and the second set of training data is collected using a cadmium-zinc-telluride-based camera.
21 . The computer-program product of claim 17 , wherein providing the input data to the deep learning model comprises:
providing the set of polar maps to the input layer of the deep learning model; and
providing the patient information to the at least one fully connected layer of the deep learning model.
22 . The computer-program product of claim 21 , wherein the at least one fully connected layer includes a fully connected output layer, and wherein providing the patient information to the at least one fully connected layer comprises providing the patient information to the fully connected output layer.
23 . The computer-program product of claim 17 , wherein providing the set of polar maps to the deep learning model comprises providing the set of polar maps without pre-defined coronary territories.
24 . A method for treating obstructive coronary artery disease (CAD), comprising:
creating a first set of training data associated with a myocardial perfusion imaging study comprising data collected using a first type of camera and a second set of training data associated with a myocardial perfusion imaging study collected using a second type of camera wherein the second type of camera is a different type of camera than the first type of camera;
training a deep learning model using the first set of training data and the second set of training data, wherein the deep learning model comprises an input layer, at least one convolution layer, at least one pooling layer, and at least one fully connected layer;
receiving input data associated with a myocardial perfusion imaging study of a patient, wherein the input data comprises a set of polar maps generated by a camera and patient information, wherein the set of polar maps comprises a perfusion polar map, a motion polar map, and a thickening polar map, and wherein the patient information comprises at least one selected from the set consisting of sex information, age information, and cardiac volume information;
providing the input data excluding the type of camera used to generate the set of polar maps to the deep learning model
generating obstructive coronary artery disease (CAD) scoring data in response to providing the input data to the deep learning model, wherein the obstructive CAD scoring data is indicative of i) a general probability for obstructive CAD, ii) a probability for obstructive CAD in a left anterior descending artery (LAD) territory, iii) a probability for obstructive CAD in a left circumflex artery (LCx) territory, iv) and a probability for obstructive CAD in a right coronary artery (RCA) territory, or v) any combination of i-iv;
generating an attention map in response to providing the input data to the deep learning model, wherein the attention map is indicative of regions of importance associated with the obstructive CAD scoring data;
presenting, on a display device, the obstructive CAD scoring data and the attention map;
applying a treatment for obstructive CAD based on the CAD scoring data.