Method to project a two dimensional image/photo onto a 3D reconstruction, such as an epicardial view of heart
A three-dimensional (3D) electrical mapping system and method may be used to generate a 3D image of the epicardial surface of a heart by integrating one or more epicardial images with a 3D image of the cardiac structure that may be generated by real-time 3D location and mapping system for cardiac mapping and ablation. The visual textural representation of the epicardial surface of the heart may be reconstructed using, for example, an image sensor or camera-based catheter to collect images of the epicardial surface. For each image that is captured, the system and method may store the image data along with the corresponding catheter location, orientation and/or distance information relative to the cardiac structure. The location, orientation, and/or distance information may be used to reconstruct a 3D textural model of the epicardial surface of the cardiac structure.
1. A system configured to generate a three-dimensional (3D) texture map of an epicardial surface of a cardiac structure, the system comprising:
a processing device configured to acquire anatomical data of the cardiac structure and generate a 3D model of the cardiac structure;
a catheter configured to be inserted into an epicardium of the cardiac structure, the catheter comprising:
an image sensor configured to capture a plurality of images of the epicardial surface of the cardiac structure and provide the plurality of images to the processing device, wherein the plurality of images include textural details of the epicardial surface;
at least one location sensor configured to generate, for each of the plurality images, corresponding location, orientation and distance information and provide the corresponding location, orientation and distance information to the processing device;
the processing device further configured to generate the 3D texture map of the epicardial surface of the cardiac structure by selecting a subset of images from the plurality of images of the epicardial surface of the cardiac structure based on the corresponding location, orientation and distance information to minimize redundant information in the subset of images, and stitching the subset of images to the 3D model of the cardiac structure using the corresponding location, orientation and distance information;
the processing device configured to provide the 3D texture map of the epicardial surface of the cardiac structure to a visual display device; and
the visual display device configured to visually display the 3D texture map of the epicardial surface of the cardiac structure.
2. The system of claim 1 , wherein the processing device is further configured to store the plurality of images and the corresponding location, orientation and distance information in a local storage.
3. The system of claim 1 , wherein the 3D texture map of the epicardial surface of the cardiac structure shows at least one of: a coronary artery, small vessels, adipose tissue, or a scar lesion.
4. The system of claim 1 , wherein the corresponding location, orientation and distance information is relative to an external reference.
5. The system of claim 1 , wherein:
the processing device is configured to generate the 3D model of the cardiac structure using a multi-view 3D reconstruction algorithm.
6. The system of claim 1 , wherein the image sensor is a charge-coupled device (CCD) image sensor.
7. The system of claim 1 , wherein the catheter includes one or more electrodes for performing ablation on the epicardium of the cardiac structure 7 based on the 3D texture map of the epicardial surface of the cardiac structure.
8. The system of claim 7 , wherein the one or more electrodes are configured to perform the ablation to treat ventricular tachycardia (VT).
9. The system of claim 1 further comprising:
extra-cardiac sensors configured to detect electrical activity of the cardiac structure and provide information on the electrical activity of the cardiac structure to the processing device.
10. A method for generating a three-dimensional (3D) texture map of an epicardial surface of a cardiac structure, the method comprising:
acquiring anatomical data of the cardiac structure and generating a 3D model of the cardiac structure;
capturing a plurality of images of the epicardial surface of the cardiac structure, wherein the plurality of images include textural details of the epicardial surface;
generating, for each of the plurality images, corresponding location, orientation and distance information;
generating the 3D texture map of the epicardial surface of the cardiac structure by selecting a subset of images from the plurality of images of the epicardial surface of the cardiac structure based on the corresponding location, orientation and distance information to minimize redundant information in the subset of images, and stitching the subset of images to the 3D model of the cardiac structure using the corresponding location, orientation and distance information; and
visually displaying the 3D texture map of the epicardial surface of the cardiac structure.
11. The method of claim 10 , further comprising:
storing the plurality of images and the corresponding location, orientation and distance information.
12. The method of claim 10 , wherein the 3D texture map of the epicardial surface of the cardiac structure shows at least one of: a coronary artery, small vessels, adipose tissue, or a scar lesion.
13. The method of claim 10 , wherein the corresponding location, orientation and distance information is relative to an external reference.
14. The method of claim 10 , wherein:
the generating the 3D model of the cardiac structure uses a multi-view 3D reconstruction algorithm.
15. The method of claim 10 , wherein the plurality of images of the epicardial surface of the cardiac structure are captured using a charge-coupled device (CCD) image sensor mounted to a catheter inserted into an epicardium of the cardiac structure.
16. The method of claim 10 , further comprising performing ablation on an epicardium of the cardiac structure based on the 3D texture map of the epicardial surface of the cardiac structure.
17. The method of claim 16 , wherein the ablation is used to treat ventricular tachycardia (VT).
18. The method of claim 16 , further comprising:
detecting electrical activity of the cardiac structure; and
providing information on the electrical activity of the cardiac structure for performing the ablation.