Machine learning approach for coronary 3D reconstruction from X-ray angiography images
A method of performing 3D vessel tree reconstruction includes providing segmented binary angiography images, applying a distance transform to the images, and generating distance transformed binary angiography images. The set of distance transformed binary angiography images are provided to a trained 3D vessel reconstruction machine learning model capable of reconstructing 3D vessels. The 3D vessel tree reconstruction machine learning model includes a multi-stage convolutional neural network comprising a multi-stage architecture with (i) a vessel centerline stage, and (ii) a radius reconstruction stage. Resultant 3D reconstructed vessel trees may be used in performing clinical assessment of coronary vessel health, and occlusion.
1 . A computer-implemented method for training a vessel tree reconstruction system, the method comprising:
selecting a set of three or more binary angiography images corresponding to three or more different projection angles for each synthetic vessel tree of a set of synthetic vessel trees;
applying a Euclidean distance transform encoding a two-dimensional (2D) diameter of each branch to the binary angiography images and generating Euclidean distance transformed angiography images; and
training, based on the set of Euclidean distance transformed angiography images, a 3D vessel tree reconstruction machine learning model capable of reconstructing 3D vessels, wherein the training of the 3D vessel tree reconstruction machine learning model includes a multi-stage convolutional neural network comprising a multi-stage architecture with (i) a vessel centerline stage, and (ii) a radius reconstruction stage.
2 . The computer-implemented method of claim 1 , wherein the 3D vessel tree reconstruction machine learning model is a multi-stage convolutional neural network having a three-stage architecture with (i) a first stage including a classification convolutional neural network backbone, (ii) second stage, being the centerline stage, including a multilayer perceptron (MLP) with ReLU activation and batch normalization between layers of the MLP for centerline reconstruction, (iii) a third stage, being the radius reconstruction stage, including a MLP with ReLU activation and batch normalization between layers of the MLP for radius reconstruction, with an output of the second stage including a M i ×N i ×3 matrix, with the 3 representing the number of dimensions of the centerline coordinates in cartesian coordinates for each point N i for each branch M i , of each 3D vessel tree i; and wherein the third stage is performed for each branch M i and the output of the third stage is an N i *1 matrix of radii for each point N i of each centerline.
3 . The computer-implemented method of claim 1 , further comprising:
generating a set of synthetic vessel trees, each synthetic vessel tree i represented by a tree matrix M i ×N i ×4, where M i is a number of branches in the synthetic vessel tree i with each branch having a branch centerline, N i is a number of points on each branch centerline, and 4 is the dimension of the data encoded in each point of the branch centerline, including three spatial dimensional coordinates and a radius;
generating, from the M i ×N i ×4 structure, a three-dimensional (3D) volume including a coronary vessel tree comprising M i branches; and
generating, from each synthetic vessel tree, the set of binary angiography images by cone-beam projecting each respective synthetic vessel tree using a plurality of projection angles.
4 . The computer-implemented method of claim 3 , further comprising generating the set of synthetic vessel trees for both a left coronary artery tree (LCA) and right coronary artery tree (RCA), each consisting of an arbitrary number of branches.
5 . The computer-implemented method of claim 3 , wherein generating the set of synthetic vessel trees comprises:
obtaining 3D medical imaging data of a coronary vessel tree of a subject;
identifying branch centerlines from the 3D medical imaging data; and
producing for each branch centerline a statistical distribution of control points and corresponding standard deviations in three-dimensional space.
6 . The computer-implemented method of claim 5 , comprising randomly generating for each branch centerline (i) the length of the centerline; (ii) a distribution of centerline control points, (iii) a most proximal radius of each branch centerline, and (iv) a radius tapering factor across the length of the branch.
7 . The computer-implemented method of claim 5 , comprising imposing one or more stenosis profiles to one or more branches of each synthetic vessel tree.
8 . The computer-implemented method of claim 7 , wherein the one or more stenosis profiles comprises a certain analytical profile, such as Gaussian or sinusoidal profile.
9 . The computer-implemented method of claim 7 , wherein each of the one or more stenosis profiles comprises at least one of parametric stenosis position data, stenosis severity data, and stenosis length data.
10 . The computer-implemented method of claim 5 , further comprising generating a 3D volume including the coronary vessel tree by a lofting of contours having a radius, r, for each point, N i , of each branch, M i , of the vessel tree, defining an analytical surface representation of the boundaries of each coronary vessel tree.
11 . The computer-implemented method of claim 2 , further including utilizing loss functions for training the multi-stage convolutional neural network, wherein the loss functions include one or more of mean square errors between data associated with the set of vessel trees and reconstructed vessel trees including one or more of data on vessel tree length, diameters, vessel tortuosity, and stenosis patterns, and further including regularization terms to prioritize certain features of the trees, including without limitation vessel tree length, diameters, vessel tortuosity, and stenosis patterns.
12 . The computer-implemented method of claim 2 , wherein the classification convolutional neural network backbone used to train the first stage is also used to train at least one of the second stage MLP and third stage MLP.
13 . The computer-implemented method of claim 2 , wherein each MLP is trained independently from each other, and each stage is trained independently.
14 . A method of performing 3D vessel tree reconstruction comprising:
providing segmented binary angiography images to a preprocessing module;
applying, by the preprocessing module, a Euclidean distance transform encoding a two-dimensional (2D) diameter of each branch to the binary angiography images and generating Euclidean distance transformed angiography images; and
feeding the Euclidean distance transformed angiography images to a trained 3D vessel tree reconstruction machine learning model capable of reconstructing 3D vessels, wherein the trained 3D vessel tree reconstruction machine learning model includes a multi-stage convolutional neural network comprising a multi-stage architecture with (i) a vessel centerline stage, and (ii) a radius reconstruction stage.
15 . The method of claim 14 , wherein a data set indicative of the 3D reconstruction of the vessel tree comprises a M i ×N i ×4 matrix, where M i is a number of branches in the vessel tree i with each branch having a branch centerline, N i is a number of points on each branch centerline, and 4 is the dimension of the data encoded in each point of the branch centerline, specifically including three spatial dimensional coordinates and a radius.
16 . The method of claim 14 , wherein the trained machine learning model is a multi-stage convolutional neural network having a three-stage architecture with (i) a first stage including a classification convolutional neural network backbone, (ii) second stage, being the centerline stage, including a multilayer perceptron (MLP) with ReLU activation and batch normalization between layers of the MLP for centerline reconstruction, (iii) a third stage, being the radius reconstruction stage, including a MLP with ReLU activation and batch normalization between layers of the MLP for radius reconstruction, with an output of the second stage including a M i ×N i ×3 matrix, with the 3 representing the number of dimensions of the centerline coordinates in cartesian coordinates for each point N i for each branch M i , and wherein the third stage is performed for each branch and the output of the third stage is an N i *1 matrix of radii for each point of each centerline.
17 . The method of claim 14 , further comprising generating a low-fidelity vessel tree representation including a vessel tree i represented by a tree matrix M i ×N i ×4, where M i is a number of branches in the vessel tree i with each branch having a branch centerline, N i is a number of points on each branch centerline, and 4 is the dimension of the data encoded in each point of the branch centerline, including three spatial dimensional coordinates and a radius.
18 . The method of claim 17 , further comprising generating, from the M i ×N i ×4 structure, a high-fidelity vessel tree representation including a three-dimensional (3D) volume including a coronary vessel tree comprising Mi branches.
19 . The method of claim 18 , further comprising generating the 3D volume by performing a 2D surface spline between the outer radii of adjacent points of each branch centerline.
20 . The method of claim 18 , further comprising:
identifying a vessel inspection region of the vessel tree; and
determining, from the low-fidelity vessel tree representation or from the high-fidelity vessel tree representation, a state of vessel occlusion for the one or more vessels within the vessel inspection region.