Augmented reality interventional system providing contextual overlays
An augmented reality interventional system which provides contextual overlays ( 116 ) to assist or guide a user ( 101 ) or enhance the performance of the interventional procedure by the user that uses an interactive medical device ( 102 ) to perform the interventional procedure. The system includes a graphic processing module ( 110 ) that is configured to generate at least one contextual overlay on an augmented reality display device system ( 106 ). The contextual overlays may identify a component ( 104 ) or control of the interactive medical device. The contextual overlays may also identify steps of a procedure to be performed by the user and provide instructions for performance of the procedure. The contextual overlays may also identify a specific region of the environment to assist or guide the user or enhance the performance of the interventional procedure by identifying paths or protocols to reduce radiation exposure.
1. A medical interventional system, comprising:
an interactive medical device including a plurality of components;
an augmented reality display device worn by a user and configured to show an image of at least one component of the plurality of components of the interactive medical device viewed by the user during an interventional procedure in an environment using the interactive medical device; and
a processor connected to memory and configured to generate a contextual overlay over the image of the at least one component shown on the augmented reality display device, wherein the processor is configured to generate the contextual overlay to include identification of control of the at least one component to assist the user in performing the interventional procedure using the interactive medical device.
2. The system as recited in claim 1 , wherein the processor is further configured to:
receive feedback from at least one of the user, a subject, and the interactive medical device during the interventional procedure; and
receive the feedback and provide the contextual overlay based on the feedback.
3. The system as recited in claim 1 , wherein the processor is further configured to receive input from the user to determine the contextual overlay that is generated.
4. The system as recited in claim 1 , wherein the processor is further configured to generate the contextual overlay to include an identification of a step of the interventional procedure to be performed by the user.
5. The system as recited in claim 4 , wherein the processor is further configured to identify the step of the interventional procedure including identifying one or more specific controls to be manipulated by the user and describing a manipulation of the one or more specific controls.
6. The system as recited in claim 4 , wherein the processor is further configured to:
receive feedback from at least one of the user, a subject, and the interactive medical device during the interventional procedure; a history of signals from the at least one component of the plurality of components of the interactive medical device; or a database;
determine the step for the user to perform for the interventional procedure based on the feedback; and
send signals concerning the contextual overlay to be generated for the step.
7. The system as recited in claim 1 , wherein the processor is further configured to generate the contextual overlay to include identification of a specific region in the environment.
8. The system as recited in claim 7 , wherein the processor is further configured to generate the contextual overlay to include identification of a desired position for a c-arm of the interactive medical device.
9. The system as recited in claim 1 , wherein the processor is further configured to:
determine a position of a target isocenter for a c-arm of the interactive medical device and a current isocenter for the c-arm; and
based on the position of the target isocenter and a current isocenter, generate the contextual overlay to include identification of the target isocenter and another contextual overlay to include identification of the current isocenter in the environment.
10. The system as recited in claim 1 , wherein the processor is further configured to generate the contextual overlay to include identification of at least one region that must be clear of obstructions for performing the interventional procedure.
11. The system as recited in claim 10 , further comprising:
determine positions of objects in the environment and compare the positions of objects with the at least one region that must be clear of obstructions; and
generate the contextual overlay to include identification of objects in the at least one region that must be clear of obstructions.
12. A method for assisting performance of an interventional procedure by a user in an environment, the method comprising:
providing an interactive medical device including a plurality of components;
displaying, by an augmented reality display device worn by the user, an image of at least one component of the plurality of components of the interactive medical device viewed by the user while performing the interventional procedure; and
generating a contextual overlay over the image of the at least one component displayed by the augmented reality display device, the contextual overlay identifying the at least one component to assist the user's performance of the interventional procedure.
13. The method as recited in claim 12 , further comprising determining the contextual overlay based on feedback from at least one of the user, a subject, and the interactive medical device acquired during the interventional procedure.
14. The method as recited in claim 12 , wherein the contextual overlay is configured to identify at least one step of the interventional procedure to be performed by the user by identifying one or more specific controls to be manipulated by the user and describing a manipulation of the one or more specific controls.