Multi-task learning framework for fully automated assessment of coronary artery disease
Systems and methods for automated assessment of a vessel are provided. One or more input medical images of a vessel of a patient are received. A plurality of vessel assessment tasks for assessing the vessel is performed using a machine learning based model trained using multi-task learning. The plurality of vessel assessment tasks are performed by the machine learning based model based on shared features extracted from the one or more input medical images. Results of the plurality of vessel assessment tasks or a combination of the results of the plurality of vessel assessment tasks are output.
1. A method comprising:
receiving one or more input medical images of a vessel of a patient;
performing a plurality of vessel assessment tasks for assessing the vessel using a machine learning based model trained using multi-task learning, the plurality of vessel assessment tasks performed by the machine learning based model based on shared features extracted from the one or more input medical images, the plurality of vessel assessment tasks comprising at least one of 1) classification of artifacts in the one or more input medical images as one of imaging artifacts and image processing artifacts or 2) classification of anomalies in the one or more input medical images as one of myocardial bridging and anomalies from prior interventions; and
outputting results of the plurality of vessel assessment tasks or a combination of the results of the plurality of vessel assessment tasks.
2. The method of claim 1 , wherein the plurality of vessel assessment tasks further comprises image-based stenosis grading of a stenosis in the vessel and wherein the image-based stenosis grading of the stenosis in the vessel is performed without using segmentation results of lumen of the vessel.
3. The method of claim 1 , wherein the plurality of vessel assessment tasks further comprises image-based stenosis grading of a stenosis in the vessel and lumen segmentation from the one or more input medical images, and wherein results of the image-based stenosis grading and results of the lumen segmentation are consistent.
4. The method of claim 1 , wherein the plurality of vessel assessment tasks further comprises a determination of one or more hemodynamic indices and lumen segmentation from the one or more input medical images, and wherein results of the determination of the one or more hemodynamic indices and results of the lumen segmentation are consistent.
5. The method of claim 1 , wherein the plurality of vessel assessment tasks further comprises at least one of detection and classification of disease in the vessel, detection and classification of artifacts in the one or more input medical images, detection and classification of anomalies in the one or more input medical images, image-based stenosis grading of a stenosis in the vessel, and lumen segmentation from the one or more input medical images.
6. The method of claim 5 , wherein the detection and classification of the disease in the vessel comprises classification of the disease as one of calcified, noncalcified, mixed calcified and noncalcified, and high risk.
7. The method of claim 1 , wherein the machine learning based model is trained using unannotated clinical reports.
8. The method of claim 1 , wherein outputting results of the plurality of vessel assessment tasks or a combination of the results of the plurality of vessel assessment tasks comprises:
outputting a heatmap for each of the plurality of vessel assessment tasks.
9. The method of claim 1 , wherein performing a plurality of vessel assessment tasks for assessing the vessel using a machine learning based model trained using multi-task learning comprises:
determining a confidence measure for the results of the plurality of vessel assessment tasks.
10. The method of claim 1 , wherein performing a plurality of vessel assessment tasks for assessing the vessel using a machine learning based model trained using multi-task learning comprises:
determining a confidence measure for each of the results of the plurality of vessel assessment tasks.
11. An apparatus comprising:
means for receiving one or more input medical images of a vessel of a patient;
means for performing a plurality of vessel assessment tasks for assessing the vessel using a machine learning based model trained using multi-task learning, the plurality of vessel assessment tasks performed by the machine learning based model based on shared features extracted from the one or more input medical images, the plurality of vessel assessment tasks comprising at least one of 1) classification of artifacts in the one or more input medical images as one of imaging artifacts and image processing artifacts or 2) classification of anomalies in the one or more input medical images as one of myocardial bridging and anomalies from prior interventions; and
means for outputting results of the plurality of vessel assessment tasks or a combination of the results of the plurality of vessel assessment tasks.
12. The apparatus of claim 11 , wherein the plurality of vessel assessment tasks further comprises image-based stenosis grading of a stenosis in the vessel and wherein the image-based stenosis grading of the stenosis in the vessel is performed without using segmentation results of lumen of the vessel.
13. The apparatus of claim 11 , wherein the plurality of vessel assessment tasks further comprises image-based stenosis grading of a stenosis in the vessel and lumen segmentation from the one or more input medical images, and wherein results of the image-based stenosis grading and results of the lumen segmentation are consistent.
14. The apparatus of claim 11 , wherein the plurality of vessel assessment tasks further comprises a determination of one or more hemodynamic indices and lumen segmentation from the one or more input medical images, and wherein results of the determination of the one or more hemodynamic indices and results of the lumen segmentation are consistent.
15. A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving one or more input medical images of a vessel of a patient;
performing a plurality of vessel assessment tasks for assessing the vessel using a machine learning based model trained using multi-task learning, the plurality of vessel assessment tasks performed by the machine learning based model based on shared features extracted from the one or more input medical images, the plurality of vessel assessment tasks comprising at least one of 1) classification of artifacts in the one or more input medical images as one of imaging artifacts and image processing artifacts or 2) classification of anomalies in the one or more input medical images as one of myocardial bridging and anomalies from prior interventions; and
outputting results of the plurality of vessel assessment tasks or a combination of the results of the plurality of vessel assessment tasks.
16. The non-transitory computer readable medium of claim 15 , wherein the plurality of vessel assessment tasks further comprises at least one of detection and classification of disease in the vessel, detection and classification of artifacts in the one or more input medical images, detection and classification of anomalies in the one or more input medical images, image-based stenosis grading of a stenosis in the vessel, and lumen segmentation from the one or more input medical images.
17. The non-transitory computer readable medium of claim 16 , wherein the detection and classification of the disease in the vessel comprises classification of the disease as one of calcified, noncalcified, mixed calcified and noncalcified, and high risk.