Image integration based registration and navigation for endoscopic surgery
A real-time tracking of surgical tools relative to a pre-operative surgical plan and intra-operative images involves an image-based registration and tool tracking registration. The image-based registration implements an integration of a plurality of images of an anatomical region of a body including a pre-operative scan image ( 31 ) (e.g., a 3D CT/MRI image), an intra-operative fluoroscopic image ( 42 ) (e.g., an 2D X-ray image) and an intra- operative endoscopic image ( 23 ) (e.g., an 2D arthroscopic image). The tool tracking registration implements a representation within the pre-operative scan image ( 31 ) and/or the intra-operative fluoroscopic image ( 42 ) of a tracking of one or more surgical tools within the intra-operative endoscopic image ( 23 ).
1. A surgical navigation system ( 60 ) for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image ( 31 ), an intra-operative fluoroscopic image ( 42 ) and an intra-operative endoscopic image ( 23 ), the surgical navigation system ( 60 ) comprising:
an image integrator ( 61 ) operable to generate an integrated image matrix (T x-E ) including an integration of a fluoroscopic image matrix (T x-CT ) and an endoscopic image matrix (T CT-E ),
wherein the fluoroscopic image matrix (T x-CT ) includes a transformation between the intra-operative fluoroscopic image ( 42 ) and the pre-operative scan image ( 31 ), and
wherein the endoscopic image matrix (T CT-E ) includes a transformation between the pre-operative scan image ( 31 ) and the intra-operative endoscopic image ( 23 ); and
a tool tracker ( 62 ) operable to generate an integrated tracking matrix (T T-X ) and a scan tracking matrix (T T-CT ),
wherein the integrated tracking matrix (T T-X ) includes an integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ),
wherein the scan tracking matrix (T T-CT ) includes an integration of the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ), and
wherein the endoscopic tracking matrix (T T-E ) represents a tracking of a surgical tool ( 21 ) within the intra-operative endoscopic image ( 23 ).
2. The surgical navigation system ( 60 ) of claim 1 , wherein the image integrator ( 61 ) is further operable to execute a photogrammetric reconstruction of a surface of an object within the intra-operative endoscopic image ( 23 ) as a basis for the transformation between the pre-operative scan image ( 31 ) and the intra-operative endoscopic image ( 23 ).
3. The surgical navigation system ( 60 ) of claim 1 , wherein the integration of the fluoroscopic image matrix (T X-CT ) and the endoscopic image matrix (T CT-E ) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T X-CT ) and the endoscopic image matrix (T CT-E ).
4. The surgical navigation system ( 60 ) of claim 1 , wherein the integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ) includes an execution of a matrix multiplication involving the integrated image matrix (T X-E ) and the endoscopic tracking matrix (T T-E ).
5. The surgical navigation system ( 60 ) of claim 1 , wherein the integration of the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ) includes an execution of a matrix multiplication involving the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ).
6. The surgical navigation system ( 60 ) of claim 1 , further comprising:
a display device for displaying the intra-operative fluoroscopic image ( 42 ) having an overlay ( 43 ) of a surgical tool ( 21 ) as a function of the integrated tracking matrix (T T-X ).
7. The surgical navigation system ( 60 ) of claim 1 , further comprising:
a display device for displaying the pre-operative scan image ( 31 ) having an overlay ( 43 ) of a surgical tool ( 21 ) as a function of the scan tracking matrix (T T-CT ).
8. The surgical navigation system ( 60 ) of claim 1 , wherein the tool tracker ( 62 ) is further operable to execute a photogrammetric tracking of the surgical tool ( 21 ) within the intra-operative endoscopic image ( 23 ).
9. The surgical navigation system ( 60 ) of claim 1 , wherein the pre-operative scan image ( 31 ) is selected from a group consisting of a computed tomography image, magnetic resonance image, an ultrasound image and an X-ray image.
10. A surgical navigation system ( 60 ) for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image ( 31 ), an intra-operative fluoroscopic image ( 42 ) and an intra-operative endoscopic image ( 23 ), the surgical navigation system ( 60 ) comprising:
an image integrator ( 61 ) operable to generate an integrated image matrix (T X-E ) including an integration of a fluoroscopic image matrix (T X-CT ) and an endoscopic image matrix (T CT-E ),
wherein the fluoroscopic image matrix (T X-CT ) includes a transformation between the intra-operative fluoroscopic image ( 42 ) and the pre-operative scan image ( 31 ), and
wherein the endoscopic image matrix (T CT-E ) includes a transformation between the pre-operative scan image ( 31 ) and the intra-operative endoscopic image ( 23 ); and
a tool tracker ( 62 ) operable to generate an integrated tracking matrix (T T-X ),
wherein the integrated tracking matrix (T T-X ) includes an integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ), and
wherein the endoscopic tracking matrix (T T-E ) represents a tracking of a surgical tool ( 21 ) within the intra-operative endoscopic image ( 23 ).
11. The surgical navigation system ( 60 ) of claim 10 , the image integrator ( 61 ) is further operable to reconstruct a surface of an object within the intra-operative endoscopic image ( 23 ) as a basis for the transformation between the pre-operative scan image ( 31 ) and the intra-operative endoscopic image ( 23 ).
12. The surgical navigation system ( 60 ) of claim 10 , wherein the integration of the fluoroscopic image matrix (T X-CT ) and the endoscopic image matrix (T CT-E ) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T X-CT ) and the endoscopic image matrix (T CT-E ).
13. The surgical navigation system ( 60 ) of claim 10 , wherein the integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ) includes an execution of a matrix multiplication involving the integrated image matrix (T X-E ) and the endoscopic tracking matrix (T T-E ).
14. The surgical navigation system ( 60 ) of claim 10 , further comprising:
a display device for displaying the intra-operative fluoroscopic image ( 42 ) having an overlay ( 43 ) of a surgical tool ( 21 ) as a function of the integrated tracking matrix (T T-X ).
15. The surgical navigation system ( 60 ) of claim 10 , wherein the tool tracker ( 62 ) is further operable to execute a photogrammetric tracking of the surgical tool ( 21 ) within the intra-operative endoscopic image ( 23 ).
16. A surgical navigation method for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image ( 31 ), an intra-operative fluoroscopic image ( 42 ) and an intra-operative endoscopic image ( 23 ), comprising:
generating an integrated image matrix (T X-E ) including an integration of a fluoroscopic image matrix (T X-CT ) and an endoscopic image matrix (T CT-E ),
wherein the fluoroscopic image matrix (T) includes a transformation between the intra-operative fluoroscopic image ( 42 ) and the pre-operative scan image ( 31 ), and wherein the endoscopic image matrix (T CT-E ) includes a transformation between the pre-operative scan image ( 31 ) and the intra-operative endoscopic image ( 23 ); and
generating an integrated tracking matrix (T T-X ) and a scan tracking matrix (T T-CT ),
wherein the integrated tracking matrix (T T-X ) includes an integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ),
wherein the scan tracking matrix (T T-CT ) includes an integration of the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ), and
wherein the endoscopic tracking matrix (T T-E ) represents a tracking of a surgical tool ( 21 ) within the intra-operative endoscopic image ( 23 ).
17. The surgical navigation method of claim 16 , wherein the integration of the fluoroscopic image matrix (T X-CT ) and the endoscopic image matrix (T CT-E ) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T. X-CT ) and the endoscopic image matrix (T CT-E ).
18. The surgical navigation method of claim 16 , wherein the integration of the integrated image matrix (T X-E ) and an endoscopic tracking matrix (T T-E ) includes an execution of a matrix multiplication involving the integrated image matrix (T X-E ) and the endoscopic tracking matrix (T T-E ).
19. The surgical navigation method of claim 16 , wherein the integration of the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ) includes an execution of a matrix multiplication involving the endoscopic image matrix (T CT-E ) and the endoscopic tracking matrix (T T-E ).
20. The surgical navigation method of claim 16 , further comprising: displaying the intra-operative fluoroscopic image ( 42 ) having an overlay ( 43 ) of a surgical tool ( 21 ) as a function of the integrated tracking matrix (T T-X ); and displaying the pre-operative scan image ( 31 ) having an overly ( 43 ) of a surgical tool ( 21 ) as a function of the scan tracking matrix (T T-CT ).