Anatomical and functional assessment of coronary artery disease using machine learning

1 . A computer-implemented method for generating enhanced segmentation images for use in determining a state of occlusion within a vessel inspection region, the method comprising:

receiving, by one or more processors, angiography image data of a vessel inspection region containing a vessel tree, the angiography image data comprising one or more two-dimensional (2D) angiography images;

applying, by the one or more processors, the angiography image data to an angiogram-level vessel segmentation machine learning model trained to generate a segmented image of the vessel tree;

applying, by the one or more processors, the angiography image data to a stenosis machine learning model trained to identify and segment stenoses within the vessel tree to generate segmented patch images;

determining, by the one or more processes, a degree of stenosis for each segmented patch image within a vessel tree;

stitching, by the one or more processors, each segmented patch image into the segmented image of the vessel tree to form an enhanced segmentation image of the vessel tree; and

storing the enhanced segmentation image and the determined degree of stenosis for each stenosis, for use in reconstructing a three-dimensional (3D) vessel tree model, performing flow extraction, and determining a state of occlusion in the vessel inspection region.

2 . The computer-implemented method of claim 1 , wherein the angiogram-level vessel segmentation machine learning model is a convolutional neural network.

3 . The computer-implemented method of claim 2 , wherein the convolution neural network has an APN pre-processing neural network and a Deeplabv3+ architecture.

4 . The computer-implemented method of claim 1 , wherein the stenosis machine learning model comprises a stenosis localization machine learning model trained to identify stenosis locations within the vessel tree and a stenosis segmentation machine learning model trained to generate segmented patch images corresponding to each identified stenosis.

5 . The computer-implemented method of claim 4 , wherein the stenosis localization machine learning model is a first convolutional neural network and wherein the stenosis segmentation machine learning model is a second convolutional neural network.

6 . The computer-implemented method of claim 5 , wherein the first convolutional neural network is an object-detector.

7 . The computer-implemented method of claim 5 , wherein the second convolution neural network has an APN pre-processing network and a VGG U-Net architecture.

8 . The computer-implemented method of claim 1 , wherein the angiogram-level vessel segmentation machine learning model and the stenosis machine learning model each have an angiography processing network (APN) trained to remove from respective input data image catheters, overlapping bony structures, and/or other image artifacts.

9 . The computer-implemented method of claim 4 , wherein the stenosis localization machine learning model is trained to identify centroid coordinates for each stenosis in angiography image data.

10 . The computer-implemented method of claim 4 , wherein the stenosis segmentation machine learning model is trained to generate a segmented patch image by classifying vessel and background pixels in a cropped patch image of the angiography image data.

11 . The computer-implemented method of claim 10 , wherein the cropped patch image of the angiography image data is given by centroid coordinate of an identified stenosis and an N×N pixel dimension.

12 . The computer-implemented method of claim 1 , wherein stitching each generated segmented patch image into the segmented image comprises an insertion and a smoothing process.

13 . The computer-implemented method of claim 12 , wherein the insertion process comprises inserting the segmented patch image into the segmented image using centroid coordinates for an identified stenosis.

14 . The computer-implemented method of claim 12 , wherein the smoothing process comprises, by the one or more processors, performing a padding process, an upscaling process, a Gaussian filter process, an erosion process, a downscaling process, a thresholding process, and/or a boundary smoothing process based on linear interpolation and a Savitzky-Golay filter.

15 . The computer-implemented method of claim 1 , comprising the determination of the degree of stenosis for each stenosis location in each segmented patch image.

16 . The computer-implemented method of claim 15 , wherein the degree of stenosis determined for each segmented patch image comprises percentage of stenosis data, stenosis length data, and stenosis eccentricity data.

17 . The computer-implemented method of claim 1 , further comprising:

feeding the enhanced segmentation image to a vessel tree geometric model constructor; and

generating, in the vessel tree geometric model constructor, a segmented geometric vessel tree model.

18 . The computer-implemented method of claim 17 , wherein the segmented geometric vessel tree model is a 3D model.

19 . The computer-implemented method of claim 17 , wherein segmented geometric vessel tree geometric model is a 1D model.

20 . The computer-implemented method of claim 1 , wherein the angiography image data comprises dynamic angiography images corresponding to a progression of fluid flow within the vessel tree, the method further comprising:

applying the dynamic angiography images to a flow data generator configured to extract flow data for the vessel tree; and

applying a segmented geometric vessel tree model generated by a vessel tree geometric model constructor, the extracted flow data, and pressure data obtained from a patient, to a reduced order model (ROM) configured to determine the state of occlusion in the vessel inspection region.

21 . The computer-implemented method of claim 20 , wherein the ROM for determining the state of occlusion in the vessel inspection region is a multi-fidelity neural network, configured to:

receive, by one or more processors, the segmented geometric vessel tree model represented by a matrix Mi×Ni×4, where Mi is the number of branches in the vessel tree i, Ni is the number of points on each branch centerline, and 4 is the data encoded in each point of the branch centerline, specifically its three-dimensional coordinates (x,y,z) and radius r;

using a feature extraction machine learning method, obtain a lower-rank representation of the geometry of the segmented geometric vessel tree model;

using a low-fidelity model, represent physics of flow and pressure in each vessel tree;

using a high-fidelity model, represent physics of flow and pressure in each vessel tree; and

using a machine learning method, establish relationships between the low-fidelity model and the high-fidelity model and their corresponding representation of the physics of flow and pressure in each vessel tree.

22 . The computer-implemented method of claim 21 , wherein the feature extraction machine learning model is a convolutional neural network (FECNN).

23 . The computer-implemented method of claim 22 , wherein the convolutional neural network is an autoencoder for dimensionality reduction.

24 . The computer-implemented method of claim 23 , wherein the autoencoder contains an encoder to compress the input geometry for each vessel tree to latent space features, which provide a low-rank representation of the input geometry for each vessel tree.

25 . The computer-implemented method of claim 23 , wherein the autoencoder contains a decoder to recover the input geometry for each vessel tree i from the latent space features.

26 . The computer-implemented method of claim 23 , wherein the autoencoder is trained to learn geometry features of the vessel tree, i, including number of branches, M i , number of stenoses, stenosis severity, and distribution of branch diameters.

27 . The computer-implemented method of claim 21 , wherein the low-fidelity model is given by either a lumped-parameter model, a 1-D linear model of the Navier-Stokes equations, a 1-D non-linear model of the Navier-Stokes, or a coarse-grid 3-D Navier-Stokes model.

28 . The computer-implemented method of claim 27 , wherein the low-fidelity model includes a model of pressure drop through each stenosis in the vessel tree.

29 . The computer-implemented method of claim 28 , wherein the model of pressure drop through the stenosis is obtained via order reduction of 3-D Navier-Stokes equations using assumptions of steady-state, axi-symmetry, and incompressibility, along with variable separation of the axial velocity.

30 . The computer-implemented method of claim 28 , wherein the model of pressure drop through the stenosis is obtained via a tensor basis reduction for the axial velocity.

31 . The computer-implemented method of claim 21 , wherein the high-fidelity model is given by either a fine-grid 3-D Navier-Stokes model of pressure and flow, or an invasive FFR assessment for each vessel tree.

32 . The computer-implemented method of claim 21 , wherein the machine learning method to establish relationships between the low-fidelity and high-fidelity models is a multi-fidelity neural network.

33 . The computer-implemented method of claim 32 , wherein the multi-fidelity neural network consists of the following components:

an encoder to compress hemodynamic data from the low-fidelity model to low-fidelity feature maps in the latent space;

a decoder to reconstruct the hemodynamic data for the high-fidelity model from high-fidelity feature maps in the latent space;

a fully connected neural network to map the relationship between low-fidelity and high-fidelity feature maps in latent space; and

skip connections between encoder and decoder to enhance the relationship between high-fidelity and low-fidelity feature maps.

34 . The computer-implemented method of claim 33 , wherein the inputs of the fully-connected neural network are:

geometric features of the vessel tree; and

the low-fidelity feature maps.

35 . The computer-implemented method of claim 33 , wherein the outputs of the fully-connected neural network are:

the high-fidelity feature maps.