System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities
A system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple modalities which continuously and immediately depicts the current state and condition of the medical procedure site synchronously with respect to each modality and without undue latency is disclosed. The composite real-time dynamic imagery may be provided by spatially registering multiple real-time dynamic video streams from the multiple modalities to each other. Spatially registering the multiple real-time dynamic video streams to each other may provide a continuous and immediate depiction of the medical procedure site with an unobstructed and detailed view of a region of interest at the medical procedure site at multiple depths. A user may thereby view a single, accurate, and current composite real-time dynamic imagery of a region of interest at the medical procedure site as the user performs a medical procedure.
1. A method for image guidance comprising:
obtaining a first real-time imaging stream from a first medical device;
obtaining a second real-time imaging stream from a second medical device;
determining a relative position and orientation of the first medical device with respect to the second medical device based at least in part on first position and orientation data associated with the first medical device and second position and orientation data associated with the second medical device;
determining a relative position and orientation of the first real-time imaging stream with respect to the second real-time imaging stream based at least in part on the determined relative position and orientation of the first medical device with respect to the second medical device; and
causing one or more displays to display a composite medical image comprising:
at least a portion of the first real-time imaging stream, and
at least a portion of the second real-time imaging stream spatially registered with the at least a portion of the first real-time imaging stream based at least in part on the determined relative position and orientation of the first real-time imaging stream with respect to the second real-time imaging stream.
2. The method of claim 1 , wherein the first real-time imaging stream depicts at least a portion of a scene from a first point-of-view of the first medical device, wherein the second real-time imaging stream depicts at least a portion of the scene from a second point-of-view corresponding to the second medical device.
3. The method of claim 1 , wherein the composite medical image depicts the at least a portion of the second real-time imaging stream from a point-of-view of the first medical device.
4. The method of claim 1 , wherein the first real-time imaging stream is based at least in part on a first modality, wherein the second real-time imaging stream is based at least in part on a second modality that is different from the first modality.
5. The method of claim 1 , wherein the composite medical image depicts the at least a portion of the second real-time imaging stream within the at least a portion of the first real-time imaging stream.
6. The method of claim 1 , wherein the at least a portion of the first real-time imaging stream is aligned with the at least a portion of the second real-time imaging stream such that the composite medical image forms a composite representation of a scene.
7. The method of claim 1 , wherein said determining the relative position and orientation of the first medical device with respect to the second medical device comprises determining a relationship between a pose of the first medical device with respect to a first coordinate system and a pose of the second medical device with respect to a second coordinate system.
8. The method of claim 1 , wherein the first real-time imaging stream is based at least in part on a modality that provides two-dimensional imagery.
9. The method of claim 1 , wherein the first real-time imaging stream is based at least in part on a modality that provides three-dimensional imagery.
10. The method of claim 1 , wherein said causing the one or more displays to display a composite medical image comprises causing the one or more displays to display the composite medical image in a virtual 3D space.
11. The method of claim 1 , wherein at least one of the first real-time imaging stream or the second real-time imaging stream corresponds to a real-time dynamic video stream of a scene.
12. The method of claim 1 , wherein at least one of the first position and orientation data or the second position and orientation data comprises spatial state data associated with at least one of the first medical device or the second medical device.
13. A system for medical procedure image guidance comprising an image guidance system having one or more processors, the one or more processors configured to:
determine a relative position and orientation of a first medical device with respect to a second medical device based at least in part on first position and orientation data associated with the first medical device and second position and orientation data associated with the second medical device;
determine a relative position and orientation of a first real-time imaging stream received from the first medical device with respect to a second real-time imaging stream received from the second medical device based at least in part on the determined relative position and orientation of the first medical device with respect to the second medical device; and
cause one or more displays to display a composite medical image comprising:
at least a portion of the first real-time imaging stream, and
at least a portion of a second real-time imaging stream spatially registered with the at least a portion of the first real-time imaging stream based at least in part on the determined relative position and orientation of the first real-time imaging stream with respect to the second real-time imaging stream.
14. The system of claim 13 , wherein the first real-time imaging stream depicts at least a portion of a scene from a first point-of-view of the first medical device, wherein the second real-time imaging stream depicts at least a portion of the scene from a second point-of-view corresponding to the second medical device.
15. The system of claim 13 , wherein the composite medical image depicts a scene from a point-of-view of the first medical device.
16. The system of claim 13 , wherein the composite medical image depicts the at least a portion of the second real-time imaging stream within the at least a portion of the first real-time imaging stream.
17. The system of claim 13 , wherein the at least a portion of the first real-time imaging stream is aligned with the at least a portion of the second real-time imaging stream such that the composite medical image forms a composite representation of a scene.
18. The system of claim 13 , wherein the first real-time imaging stream is based at least in part on a modality that provides three-dimensional imagery.
19. The system of claim 13 , wherein at least one of the first real-time imaging stream or the second real-time imaging stream corresponds to a real-time dynamic video stream of a scene.