Method and system for machine learning based assessment of fractional flow reserve
A method and system for determining fractional flow reserve (FFR) for a coronary artery stenosis of a patient is disclosed. In one embodiment, medical image data of the patient including the stenosis is received, a set of features for the stenosis is extracted from the medical image data of the patient, and an FFR value for the stenosis is determined based on the extracted set of features using a trained machine-learning based mapping. In another embodiment, a medical image of the patient including the stenosis of interest is received, image patches corresponding to the stenosis of interest and a coronary tree of the patient are detected, an FFR value for the stenosis of interest is determined using a trained deep neural network regressor applied directly to the detected image patches.
1. A method for determining fractional flow reserve (FFR) for a stenosis of interest for a patient, comprising:
receiving a medical image of the patient including the stenosis of interest;
detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient; and
determining an FFR value for the stenosis of interest using a trained deep neural network regressor applied directly to the detected image patches without first extracting features from the medical image, wherein the trained deep neural network regressor is trained using a first set of training image patches without corresponding FFR values to train weights of layers other than a final layer of the deep neural network regressor and a second set of training image patches including identified stenosis image patches and corresponding FFR values to refine the weights of all layers including the final layer of the deep neural network regressor.
2. The method of claim 1 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting image patches corresponding to the stenosis of interest, coronary ostia, coronary vessels, and coronary bifurcation and trifurcations.
3. The method of claim 1 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting the image patches in a series of marginal parameter spaces using a respective trained deep neural network for each of the marginal parameter spaces.
4. The method of claim 3 , wherein the series of marginal parameter spaces comprises a position parameter space, a position-orientation parameter space, and a position-orientation-scale parameter space.
5. The method of claim 3 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a deep neural network regressor trained that inputs hypotheses in the respective parameter space and for each hypothesis, outputs a displacement vector that provides a prediction for an image patch in the respective parameter space.
6. The method of claim 3 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a discriminative deep neural network that inputs hypotheses in the respective parameters space, and for each hypothesis outputs a probability for the image patch corresponding to the hypothesis.
7. The method of claim 3 , wherein the respective deep neural network for each of the marginal parameter spaces is a deep multi-layer neural network trained using one of a convolutional neural network (CNN), a stacked restricted Boltzmann machine (RBM), or a stacked auto-encoder (AE).
8. The method of claim 1 , wherein the trained deep neural network regressor is a deep neural network with a plurality of layers and the final layer calculates a stenosis specific FFR value.
9. The method of claim 8 , wherein the trained deep neural network is trained by tuning weights for each layer other than the final layer using the first set of training image patches without corresponding FFR values using restricted Boltzmann machines (RBM) contrastive divergences or Auto-encoders algorithms and then refining the weights for each layer including the final layer based on the second set of training image patches with corresponding FFR values using gradient descent back-propagation.
10. The method of claim 1 , wherein the trained deep neural network regressor includes a first hidden layer at which learned weights map raw image data from the detected image patches to a first set of latent variables, a second hidden layer at which learned weights map the first set of latent variables to a second set of latent variables, a third hidden layer at which learned weights map the second set of latent variables to a third set of latent variables, and the final layer at which learned weights map the third set of latent variables to a stenosis specific FFR value.
11. An apparatus for determining fractional flow reserve (FFR) for a stenosis of interest for a patient, comprising:
a processor; and
a memory storing computer program instructions, which when executed by the processor cause the processor to perform operations comprising:
receiving a medical image of the patient including the stenosis of interest;
detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient; and
determining an FFR value for the stenosis of interest using a trained deep neural network regressor applied directly to the detected image patches without first extracting features from the medical image, wherein the trained deep neural network regressor is trained using a first set of training image patches without corresponding FFR values to train weights of layers other than a final layer of the deep neural network regressor and a second set of training image patches including identified stenosis image patches and corresponding FFR values to refine the weights of all layers including the final layer of the deep neural network regressor.
12. The apparatus of claim 11 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting image patches corresponding to the stenosis of interest, coronary ostia, coronary vessels, and coronary bifurcation and trifurcations.
13. The apparatus of claim 11 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting the image patches in a series of marginal parameter spaces using a respective trained deep neural network for each of the marginal parameter spaces.
14. The apparatus of claim 13 , wherein the series of marginal parameter spaces comprises a position parameter space, a position-orientation parameter space, and a position-orientation-scale parameter space.
15. The apparatus of claim 13 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a deep neural network regressor trained that inputs hypotheses in the respective parameter space and for each hypothesis, outputs a displacement vector that provides a prediction for an image patch in the respective parameter space.
16. The apparatus of claim 13 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a discriminative deep neural network that inputs hypotheses in the respective parameters space, and for each hypothesis outputs a probability for the image patch corresponding to the hypothesis.
17. The apparatus of claim 13 , wherein the respective deep neural network for each of the marginal parameter spaces is a deep multi-layer neural network trained using one of a convolutional neural network (CNN), a stacked restricted Boltzmann machine (RBM), or a stacked auto-encoder (AE).
18. The apparatus of claim 11 , wherein the trained deep neural network regressor is a deep neural network with a plurality of layers and the final layer calculates a stenosis specific FFR value.
19. The apparatus of claim 18 , wherein the trained deep neural network is trained by tuning weights for each layer other than the final layer using the first set of training image patches without corresponding FFR values using restricted Boltzmann machines (RBM) contrastive divergences or Auto-encoders algorithms and then refining the weights for each layer including the final layer based on the second set of training image patches with corresponding FFR values using gradient descent back-propagation.
20. A non-transitory computer readable medium storing computer program instructions for determining fractional flow reserve (FFR) for a stenosis of interest for a patient, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving a medical image of the patient including the stenosis of interest;
detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient; and
determining an FFR value for the stenosis of interest using a trained deep neural network regressor applied directly to the detected image patches without first extracting features from the medical image, wherein the trained deep neural network regressor is trained using a first set of training image patches without corresponding FFR values to train weights of layers other than a final layer of the deep neural network regressor and a second set of training image patches including identified stenosis image patches and corresponding FFR values to refine the weights of all layers including the final layer of the deep neural network regressor.
21. The non-transitory computer readable medium of claim 20 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting image patches corresponding to the stenosis of interest, coronary ostia, coronary vessels, and coronary bifurcation and trifurcations.
22. The non-transitory computer readable medium of claim 20 , wherein detecting image patches corresponding to the stenosis of interest and a coronary tree of the patient comprises:
detecting the image patches in a series of marginal parameter spaces using a respective trained deep neural network for each of the marginal parameter spaces.
23. The non-transitory computer readable medium of claim 22 , wherein the series of marginal parameter spaces comprises a position parameter space, a position-orientation parameter space, and a position-orientation-scale parameter space.
24. The non-transitory computer readable medium of claim 22 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a deep neural network regressor trained that inputs hypotheses in the respective parameter space and for each hypothesis, outputs a displacement vector that provides a prediction for an image patch in the respective parameter space.
25. The non-transitory computer readable medium of claim 22 , wherein the respective trained deep neural network for each of the marginal parameter spaces comprises a discriminative deep neural network that inputs hypotheses in the respective parameters space, and for each hypothesis outputs a probability for the image patch corresponding to the hypothesis.
26. The non-transitory computer readable medium of claim 22 , wherein the respective deep neural network for each of the marginal parameter spaces is a deep multi-layer neural network trained using one of a convolutional neural network (CNN), a stacked restricted Boltzmann machine (RBM), or a stacked auto-encoder (AE).
27. The non-transitory computer readable medium of claim 20 , wherein the trained deep neural network regressor is a deep neural network with a plurality of layers and the final layer calculates a stenosis specific FFR value.
28. The non-transitory computer readable medium of claim 27 , wherein the trained deep neural network is trained by tuning weights for each layer other than the final layer using the first set of training image patches without corresponding FFR values using restricted Boltzmann machines (RBM) contrastive divergences or Auto-encoders algorithms and then refining the weights for each layer including the final layer based on the second set of training image patches with corresponding FFR values using gradient descent back-propagation.