3D vessel centerline reconstruction from 2D medical images
Systems and methods for determining a 3D centerline of a vessel are provided. A current state observation of an artificial agent is determined based on one or more image view sets, each including 2D medical images of a vessel, a current position of the artificial agent in the 2D medical images, and a start position and a target position in the 2D medical images. Policy values are calculated for a plurality of actions for moving the artificial agent in 3D based on the current state observation using a trained machine learning model. The artificial agent is moved according to a particular action based on the policy values. The steps of determining, calculating, and moving are repeated for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position. The 3D centerline of the vessel is determined as the 3D path.
1. A method for determining a 3D centerline of a vessel in one or more two dimensional (2D) medical images, comprising:
determining a current state observation of an artificial agent based on one or more image view sets each comprising one or more 2D medical images of a vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images;
calculating policy values for a plurality of actions for moving the artificial agent in three dimensions (3D) based on the current state observation using a trained machine learning model;
moving the artificial agent according to a particular action, of the plurality of actions, based on the policy values;
repeating the determining, the calculating, and the moving steps for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position; and
determining a 2D centerline of the vessel as the 3D path.
2. The method of claim 1 , further comprising:
determining ( 212 ) a 2D centerline of the vessel based on the 3D centerline of the vessel.
3. The method of claim 1 , wherein repeating the determining, the calculating, and the moving steps for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position comprises:
repeating the determining, the calculating, and the moving steps until the current position of the artificial agent is the target position or a maximum number of iterations has been performed.
4. The method of claim 1 , wherein the plurality of actions comprises discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upwards direction, and a downwards direction.
5. The method of claim 1 , wherein the plurality of actions comprises continuous movements in any direction.
6. The method of claim 1 , wherein at least one of the one or more image view sets further comprises at least one of a vesselness image, an intensity image, one or more orientation images, branch overlapping scores, and key point detections.
7. The method of claim 1 , wherein the one or more image view sets is a single image view set comprising a time of arrival of a contrast agent and blood flow velocity or contrast agent propagation speed.
8. The method of claim 1 , wherein calculating policy values for a plurality of actions for moving the artificial agent in three dimensions (3D) based on the current state observation using a trained machine learning model comprises:
calculating the policy values for the plurality of actions based on the current state observation using a trained deep neural network (DNN) trained using deep reinforcement learning.
9. The method of claim 1 , further comprising:
detecting a proximal point and a plurality of respective distal points in a vessel tree; and
performing the determining the current state observation, the calculating, the moving, the repeating, and the determining the 3D centerline of the vessel for each proximal point-distal point pair using the proximal point as the start position and the respective distal point as the target position to determine a 3D centerline of the vessel tree.
10. The method of claim 1 , wherein the one or more 2D medical images of the vessel are channels of an angiographic x-ray image.
11. An apparatus for determining a 3D centerline of a vessel in one or more two dimensional (2D) medical images, comprising:
means for determining a current state observation of an artificial agent based on one or more image view sets each comprising one or more 2D medical images of a vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images;
means for calculating policy values for a plurality of actions for moving the artificial agent in three dimensions (3D) based on the current state observation using a trained machine learning model;
means for moving the artificial agent according to a particular action, of the plurality of actions, based on the policy values;
means for repeating the determining, the calculating, and the moving steps for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position; and
means for determining a 3D centerline of the vessel as the 3D path.
12. The apparatus of claim 11 , further comprising:
means for determining a 2D centerline of the vessel based on the 3D centerline of the vessel.
13. The apparatus of claim 11 , wherein the means for repeating the determining, the calculating, and the moving steps for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position comprises:
means for repeating the determining, the calculating, and the moving steps until the current position of the artificial agent is the target position or a maximum number of iterations has been performed.
14. The apparatus of claim 11 , wherein at least one of the one or more image view sets further comprises at least one of a vesselness image, an intensity image, one or more orientation images, branch overlapping scores, and key point detections.
15. A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
determining a current state observation of an artificial agent based on one or more image view sets each comprising one or more two dimensional (2D) medical images of a vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images;
calculating policy values for a plurality of actions for moving the artificial agent in three dimensions (3D) based on the current state observation using a trained machine learning model;
moving the artificial agent according to a particular action, of the plurality of actions, based on the policy values;
repeating the determining, the calculating, and the moving steps for a plurality of iterations to move the artificial agent along a 3D path between the start position and the target position; and
determining a 3D centerline of the vessel as the 3D path.
16. The non-transitory computer readable medium of claim 15 , wherein the plurality of actions comprises discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upwards direction, and a downwards direction.
17. The non-transitory computer readable medium of claim 15 , wherein the plurality of actions comprises continuous movements in any direction.
18. The non-transitory computer readable medium of claim 15 , wherein the one or more image view sets is a single image view set comprising a time of arrival of a contrast agent and blood flow velocity or contrast agent propagation speed.
19. The non-transitory computer readable medium of claim 15 , the operations further comprising:
detecting a proximal point and a plurality of respective distal points in a vessel tree; and
performing the determining the current state observation, the calculating, the moving, the repeating, and the determining the 3D centerline of the vessel for each proximal point-distal point pair using the proximal point as the start position and the respective distal point as the target position to determine a 3D centerline of the vessel tree.
20. The non-transitory computer readable medium of claim 15 , wherein the one or more 2D medical images of the vessel are channels of an angiographic x-ray image.