IP Library › Granted Patent US 12,290,413
Granted Patent B2
US 12,290,413 · App. 18/488,022 · Granted May 6, 2025

Immersive three-dimensional display for robotic surgery

Inventors: Kent Anderson (Mountain View, CA); Joan Savall (Palo Alto, CA); Brent Nobles (Palo Alto, CA); Allegra Shum (San Francisco, CA); Pablo E. Garcia Kilroy (Menlo Park, CA); Karen Shakespear Koenig (San Jose, CA)
Assignee: Verb Surgical Inc.
A61B90/37A61B34/25A61B34/30A61B34/35A61B34/74A61B90/50G06F3/012G06F3/013G06T19/006H04N13/344A61B2017/00199A61B2017/00203A61B2017/00207A61B2017/00216A61B2017/00973A61B2090/365A61B2090/372H04N2213/001
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,290,413
App. No.
18/488,022
Granted
May 6, 2025
Kind
B2
Abstract

An immersive display for use in a robotic surgical system includes a support arm, a housing mounted to the support arm and configured to engage with a face of the user, at least two eyepiece assemblies disposed in the housing and configured to provide a three-dimensional display, and at least one sensor, wherein the sensor enables operation of the robotic surgical system, and wherein the support arm is actuatable to move the housing for ergonomic positioning.

Claims (43)

1. A robotic surgical system, comprising:

a support structure;

an immersive display mounted on the support structure, the immersive display comprising:

a housing comprising a display, wherein the housing is configured to engage with a face of a user, and

a proximity sensor that is part of the housing and is configured to sense movement of the face relative to the housing; and

a controller that is configured to restrict operation of the robotic surgical system based on the sensed movement.

2. The robotic surgical system of claim 1 , wherein the display is a stereoscopic display that comprises a first eyepiece assembly configured to display a left eye stereoscopic image and a second eyepiece assembly configured to display a right eye stereoscopic image.

3. The robotic surgical system of claim 1 , wherein the housing comprises a contoured face frame that is configured to receive the face to provide the user a line of sight with the display, wherein the proximity sensor is disposed on the contoured face frame.

4. The robotic surgical system of claim 1 , wherein the immersive display comprises three degrees of freedom with respect to the support structure.

5. The robotic surgical system of claim 4 , wherein the three degrees of freedom comprises:

a first degree of freedom to move the immersive display along a first axis;

a second degree of freedom to move the immersive display along a second axis that is orthogonal to the first axis; and

a third degree of freedom to tilt the immersive display about a third axis that is orthogonal to both first and second axes.

6. The robotic surgical system of claim 1 , wherein the controller is configured to restrict the operation when the sensed movement comprises the face moving away from the housing.

7. The robotic surgical system of claim 1 , wherein the display is configured to present images of an endoscope camera, wherein the controller is configured to cause another display to present the images responsive to the sensed movement.

8. The robotic surgical system of claim 1 further comprises:

a robotic arm; and

an input device for receiving user input to manipulate the robotic arm,

wherein the restricted operation comprises preventing the input device from causing the robotic arm to be manipulated responsive to receiving the user input.

9. The robotic surgical system of claim 8 further comprises:

a seat on which the user sits while the housing is engaged with the user's face to view the display; and

a console to which the support structure and the input device are coupled, wherein the console is separate from the robotic arm and the seat.

10. The robotic surgical system of claim 9 , wherein the console comprises the input device that includes at least one of a handheld device for receiving user input based on hand gestures by the user and a foot pedal for receiving user input based on foot gestures by the user.

11. An immersive display for use in a robotic surgical system that includes a robotic arm, comprising:

a support structure;

a housing mounted to the support structure and configured to receive a face of a user and to provide a display; and

a proximity sensor that is part of the housing and is configured to sense movement of the face relative to the housing,

wherein in response to receiving an indication of a movement of the face from the proximity sensor, operation of the robotic arm is restricted.

12. The immersive display of claim 11 , wherein the robotic surgical system comprises an input device for receiving user input to manipulate the robotic arm, wherein operation of the robotic arm is restricted by preventing the input device from causing the robotic arm to be manipulated responsive to receiving the user input.

13. The immersive display of claim 11 , wherein the support structure is mounted on a console that comprises an input device that includes at least one of a handheld device for receiving user input based on hand gestures by the user and a foot pedal for receiving user input based on foot gestures by the user.

14. The immersive display of claim 11 , wherein the housing comprises a contoured face frame that is configured to receive the face to provide the user a line of sight with the display, wherein the proximity sensor is disposed on the contoured face frame.

15. The immersive display of claim 11 , wherein the housing comprises three degrees of freedom with respect to the support structure that includes:

a first degree of freedom to move the immersive display along a first axis;

a second degree of freedom to move the immersive display along a second axis that is orthogonal to the first axis; and

a third degree of freedom to tilt the immersive display about a third axis that is orthogonal to both first and second axes.

16. A method comprising:

displaying at least one image from an endoscope camera used in a robotic surgical system on a display that is disposed in a housing;

determining, using a proximity sensor that is part of the housing, whether a face of a user is engaged with the housing; and

restricting operation of the robotic surgical system responsive to determining that the face of the user is not engaged with the housing.

17. The method of claim 16 , wherein the robotic surgical system comprises an input device configured to receive user input to manipulate a robotic component of the robotic surgical system, wherein restricting operation comprises preventing the input device from causing the robotic component to be manipulated responsive to the input device receiving the user input.

18. The method of claim 16 further comprising causing another display to display the at least one image from the endoscope camera responsive to determining that the face of the user is not engaged with the housing.

19. The method of claim 16 , wherein the housing comprises a contoured face frame that is configured to receive the face to provide the user a line of sight with the display, wherein the proximity sensor is disposed on the contoured face frame.

20. The method of claim 16 , wherein the operation of the robotic surgical system is restricted responsive to determining that the face is moving away from the housing based on sensor data of the proximity sensor.

Assignments (1)
MERGER Recorded Jan 23, 2026
From: VERB SURGICAL INC.
To: AURIS HEALTH, INC.
Reel/Frame 073571/0718 →
Continuity (5)
Continuation 17819843 · Aug 15, 2022
Continuation 17009644 · Sep 1, 2020
Continuation 15724185 · Oct 3, 2017
Provisional Application 62403655 · Oct 3, 2016
Related Publication 20240115346A1 · Apr 11, 2024
References Cited (79)
US 4574199A · Pryor · 1986 [cited by examiner]
US 5243665A · Maney · 1993 [cited by examiner]
US 5920394A · Gelbart · 1999 [cited by examiner]
US 8535336B2 · Trovato · 2013 [cited by examiner]
US 10646156B1 · Schnorr · 2020 [cited by examiner]
US 10883708B2 · Chien · 2021 [cited by examiner]
US 20030060927A1 · Gerbi et al. · 2003 [cited by applicant]
US 20030114962A1 · Niemeyer · 2003 [cited by applicant]
US 20050065658A1 · Green · 2005 [cited by applicant]
US 20070265495A1 · Vayser · 2007 [cited by applicant]
US 20080262312A1 · Carroll · 2008 [cited by examiner]
US 20090046146A1 · Hoyt · 2009 [cited by examiner]
US 20090088634A1 · Zhao · 2009 [cited by examiner]
US 20100079356A1 · Hoellwarth · 2010 [cited by applicant]
US 20100217991A1 · Choi · 2010 [cited by applicant]
US 20110118748A1 · Itkowitz · 2011 [cited by applicant]
US 20110118753A1 · Itkowitz et al. · 2011 [cited by applicant]
US 20140018960A1 · Itkowitz · 2014 [cited by applicant]
US 20140207541A1 · Nerayoff · 2014 [cited by examiner]
US 20140343404A1 · Razzaque et al. · 2014 [cited by applicant]
US 20150173846A1 · Schneider · 2015 [cited by examiner]
US 20150230697A1 · Phee et al. · 2015 [cited by applicant]
US 20160000606A1 · Spier · 2016 [cited by applicant]
US 20160037998A1 · Kawashima et al. · 2016 [cited by applicant]
US 20160196694A1 · Lindeman · 2016 [cited by applicant]
US 20160220324A1 · Tesar · 2016 [cited by applicant]
US 20190223964A1 · Navkar et al. · 2019 [cited by applicant]
US 20190293935A1 · Schneider · 2019 [cited by examiner]
US 20200138518A1 · Lang · 2020 [cited by examiner]
US 20210192759A1 · Lang · 2021 [cited by examiner]
CN 102186434A · 2011 [cited by applicant]
CN 104918572A · 2015 [cited by applicant]
CN 105511632A · 2016 [cited by applicant]
JP 08196541A · 1996 [cited by applicant]
JP 09005633A · 1997 [cited by applicant]
JP 10254381A · 1998 [cited by applicant]
JP 2003225198A · 2003 [cited by applicant]
JP 2003250812A · 2003 [cited by applicant]
JP 2004305367A · 2004 [cited by applicant]
JP 2013510671A · 2013 [cited by applicant]
JP 2014094039A · 2014 [cited by applicant]
JP 5737796B2 · 2015 [cited by applicant]
WO 2010021447A1 · 2010 [cited by applicant]
WO 2012075155A2 · 2012 [cited by applicant]
WO 2014155725A1 · 2014 [cited by applicant]
WO 2016133644A1 · 2016 [cited by applicant]
WO 2018057814A1 · 2018 [cited by applicant]
WO 2018067611A1 · 2018 [cited by applicant]
Advisory Action Office Action of the U.S. Patent Office dated Apr. 9, 2020 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Australian Full Examination Report dated Apr. 23, 2019 for related Australian Appln. No. 2017339943 3 Pages. [cited by applicant]
Chinese Office Action with English translation, dated Jul. 31, 2019, Chinese Application No. 201780003779.4. [cited by applicant]
Corrected Notice of Allowability of the U.S. Patent Office dated Aug. 26, 2020, for related U.S. Appl. No. 15/724,185. [cited by applicant]
Corrected Notice of Allowability of the U.S. Patent Office dated Jul. 8, 2020 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Decision of Rejection of the Chinese Patent Office dated Oct. 14, 2020 for related Chinese Patent Application No. 201780003779.4. [cited by applicant]
Decision to Grant a Patent of the Japanese Patent Office dated Jul. 17, 2020 for related Japanese Patent Application No. 2019-512313. [cited by applicant]
Examination Report No. 1 of the Australian Patent Office dated Apr. 23, 2019 for related Australian Patent Application No. 2017339943. [cited by applicant]
Examination Report of the Australian Patent Office dated May 3, 2021 for related Australian Patent Application No. 2020200343. [cited by applicant]
Examiner's Report of the Canadian Patent Office dated Mar. 9, 2020 for related Canadian Patent Application No. 3,035,258. [cited by applicant]
Extended European Search Report of the EP Patent Office dated Apr. 23, 2020 for related EP Patent Application No. 17859051.9. [cited by applicant]
Final Office Action of the U.S. Patent Office dated Jan. 27, 2020 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Final Office Action of the U.S. Patent Office dated Mar. 28, 2019 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Grant of Patent of the Korean Patent Office dated Mar. 10, 2021 for related Korean Patent Application No. 10-2019-7006568. [cited by applicant]
International Preliminary Report on Patentability of the PCT dated Apr. 18, 2019 for related PCT Patent Application No PCT/US2017/054995. [cited by applicant]
International Preliminary Report on Patentability, mailed Apr. 18, 2019, for Application No. PCT/US2017/054995. [cited by applicant]
International Search Report and Written Opinion of the PCT dated Nov. 30, 2017 for related PCT Patent Application No. PCT/US2017/054995. [cited by applicant]
Non-Final Office Action of the U.S. Patent Office dated Aug. 28, 2019 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Non-Final Office Action of the U.S. Patent Office dated Dec. 13, 2018 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Notice of Acceptance of the Australian Patent Office dated Oct. 3, 2019 for related Australian Patent Application No. 2017339943. [cited by applicant]
Notice of Allowance of the U.S. Patent Office dated May 15, 2020 for related U.S. Appl. No. 15/724,185. [cited by applicant]
Notice of Reasons for Refusal of the Japanese Patent Office dated Aug. 17, 2021 for related Japanese Patent Application No. 2020-141786. [cited by applicant]
Notification of Reasons for Refusal of the Japanese Patent Office dated Mar. 29, 2022 for related Japanese Patent Application No. 2020-141786. [cited by applicant]
Notification of Reasons for Refusal of tt,e Ja,..nese Patenl Office dated Mar. 29, 2022 fir related Japanese Patert Application No. 2020-141786. [cited by applicant]
Office Action of the Canadian Patent Office dated Dec. 15, 2020 for related Canadian Patent Application No. 3035258. [cited by applicant]
Office Action of the Chinese Patent Office dated Jul. 31, 2019 for related Chinese Patent Application No. 201780003779.4. [cited by applicant]
Office Action of the Chinese Patent Office dated Mar. 5, 2020 for related Chinese Patent Application No. 201780003779.4. [cited by applicant]
Office Action of the Japanese Patent Office dated Mar. 2, 2020 for related Japanese Patent Application No. 2019-512313. [cited by applicant]
Office Action of the Korean Patent Office dated Aug. 28, 2020 for related Korean Patent Application No. 10-2019-7006568. [cited by applicant]
Second Office Action of the China National Intellectual Property Administration dated Mar. 5, 2020, for related Chinese Patent Application No. 201780003779.4. [cited by applicant]
Written Opinion of the International Search Authority Mailed on Nov. 30, 2017 for WO Application No. PCT/US17/054995. [cited by applicant]