Immersive three-dimensional display for robotic surgery
An immersive display for use in a robotic surgical system includes a support arm, a housing mounted to the support arm and configured to engage with a face of the user, at least two eyepiece assemblies disposed in the housing and configured to provide a three-dimensional display, and at least one sensor, wherein the sensor enables operation of the robotic surgical system, and wherein the support arm is actuatable to move the housing for ergonomic positioning.
1. An immersive display for use in a robotic surgical system, comprising:
a support arm;
a housing mounted to the support arm and configured to engage with a face of a user, the housing further configured to receive information to be displayed;
two eyepiece assemblies disposed in the housing and configured to display images from an endoscopic camera as a three-dimensional (3D) display; and
a sensor that is a part of either the support arm or the housing, and is configured to monitor head gestures of the user;
wherein the support arm is actuatable to move the housing and in response to receiving an indication of a detected intentional forward head gesture of the user from the sensor, 1) the support arm moves the housing to track the intentional forward head gesture so as to maintain the engagement between the face of user and the housing and 2) the two eyepiece assemblies become configured to display zoomed-in images from the endoscopic camera according to the intentional forward head gesture,
and in response to receiving an indication of a detected intentional backward head gesture of the user form the sensor, 1) the support arm moves the housing to track the intentional backward head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies become configured to display zoomed-out images from the endoscopic camera according to the intentional backward head gesture.
2. The immersive display of claim 1 , wherein the support arm is articulated and comprises a plurality of joints.
3. The immersive display of claim 1 , further comprising at least one shield coupled to the housing, wherein the shield is movable between a first position in which the shield is configured to obscure at least a portion of a field of view of the user, and a second position in which the shield is configured to reveal the portion of the field of view of the user.
4. The immersive display of claim 1 , wherein the housing is coupled to a contoured face frame configured to engage with the face of the user.
5. The immersive display of claim 4 , wherein the support arm is coupled to a first portion of the housing and the face frame is coupled to a second portion of the housing that is movable relative to the first portion of the housing.
6. The immersive display of claim 1 , further comprising at least one auxiliary display coupled to the housing.
7. The immersive display of claim 1 , wherein at least one of the eyepiece assemblies is configured to display a left eye stereoscopic image and at least one of the eyepiece assemblies is configured to display a right eye stereoscopic image.
8. The immersive display of claim 1 , wherein the sensor is further configured to identify the user for authorization to operate the robotic surgical system.
9. The immersive display of claim 1 , further comprising at least one tracking device coupled to the housing to monitor position of the housing.
10. The immersive display of claim 1 , wherein the immersive display is configured to provide guidance for maintaining a correspondence between a relative spatial relationship of the eyepiece assemblies and user hand positions and a relative spatial relationship of the endoscopic camera and a surgical instrument.
11. The immersive display of claim 1 , wherein, in response to receiving an indication of a detected intentional lateral head gesture of the user from the sensor, 1) the support arm moves the housing to track the intentional lateral head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies become configured to display panning images from the endoscopic camera.
12. The immersive display of claim 1 , wherein the two eyepiece assemblies are configured to display a graphical user interface and the sensor is further configured to detect an intentional head gesture for navigation of the graphical user interface.
13. The immersive display of claim 1 , wherein the two eyepiece assemblies are configured to display a visual representation of at least one of a user hand position and a user foot position relative to at least one target position overlaid upon images from the endoscopic camera.
14. The immersive display of claim 1 , further comprising at least one external camera coupled to the housing, wherein the camera is configured to provide at least one image of an environment external to the housing.
15. The immersive display of claim 4 , wherein the housing has a cavity therein, the cavity containing one or more display screens in an optical path of the at least two eyepiece assemblies and wherein the face frame is configured to encompass the at least two eyepiece assemblies.
16. An immersive display system for use in a robotic surgical system, comprising:
a support arm;
a housing mounted to the support arm and configured to engage with a face of a user, the housing further configured to receive information to be displayed;
an actuator configured to actuate the support arm to move the housing;
at least two eyepiece assemblies disposed in the housing and configured to display images from an endoscopic camera as a three-dimensional (3D) display;
a sensor that is a part of either the support arm or the housing; and
a programmed processor that, in response to detecting an intentional forward head gestures of the user based on output from the sensor, signals 1) the actuator to move the housing to track the intentional forward head gestures so as to maintain the engagement between the face of user and the housing and 2) the two eyepiece assemblies to display zoomed-in images from the endoscopic camera according to the intentional forward head gesture,
and in response to detecting an intentional backward head gesture of the user based on output from the sensor, signals 1) the actuator to move the housing to track the intentional backward head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies to display zoomed-out images from the endoscopic camera according to the intentional backward head gesture.
17. The immersive display system of claim 16 , wherein the housing is coupled to a contoured face frame configured to engage with the face of the user.
18. The immersive display system of claim 17 , wherein the support arm is coupled to a first portion of the housing and the face frame is coupled to a second portion of the housing that is movable relative to the first portion of the housing.
19. The immersive display system of claim 16 , wherein at least one of the eyepiece assemblies is configured to display a left eye stereoscopic image and at least one of the eyepiece assemblies is configured to display a right eye stereoscopic image.
20. The immersive display system of claim 16 , further comprising at least one tracking device coupled to the housing to monitor position of the housing.
21. The immersive display system of claim 16 , wherein, in response to detecting an intentional lateral head gesture of the user based on output from the sensor, the programmed processor signals 1) the actuator to move the housing to track the intentional lateral head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies to display panning images from the endoscopic camera.
22. The immersive display of claim 1 , wherein, in response to receiving an indication of a detected intentional tilting head gesture of the user from the sensor, 1) the support arm moves the housing to track the intentional tiling head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies become configured to display tilting images from the endoscopic camera.
23. The immersive display of claim 16 , wherein, in response to detecting an intentional tilting head gesture of the user based on output from the sensor, the programmed processor signals 1) the actuator to move the housing to track the intentional tiling head gesture so as to maintain the engagement between the face of the user and the housing and 2) the two eyepiece assemblies to display tilting images from the endoscopic camera.
24. The immersive display of claim 1 , wherein the support arm moves and the two eyepiece assemblies become configured to display modified images from the endoscopic camera in response to receiving simultaneously 1) an indication of a detected intentional head gesture of the user from the sensor and 2) an indication of an activated clutch by the user.