IP Library › Granted Patent US 12,702,485
Granted Patent B2
US 12,702,485 · App. 18/603,077 · Granted Aug 11, 2026

Feedback for surgical robotic system with virtual reality

Inventors: Bernhard Fuerst (Sunnyvale, CA); Eric Johnson (Pacific Grove, CA); Pablo Garcia Kilroy (Menlo Park, CA)
Assignee: Auris Health, Inc.
A61B34/10B25J13/06G06F3/04815A61B2034/104A61B2034/105
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,702,485
App. No.
18/603,077
Filed
Mar 12, 2024
Granted
Aug 11, 2026
Kind
B2
Art Unit
2614
USPC
345/419
Abstract

A virtual surgical robot being built from kinematic data is rendered to a display. A user input is received to effect a movement or a configuration of the virtual surgical robot. The kinematic data is modified based on evaluation of the movement or the configuration of the virtual surgical robot.

Claims (32)

1 . A method for development of a surgical robotic system, comprising:

rendering a virtual surgical robot within a virtual environment on a display, wherein the virtual surgical robot is built from kinematic data that defines at least a geometry of a corresponding surgical robot;

receiving, through a user input device configured to control the virtual surgical robot, a user input to cause the virtual surgical robot to perform a movement;

revising, based on the performed movement of the virtual surgical robot, a control algorithm used to control the corresponding surgical robot or a tool coupled to the corresponding surgical robot based on user inputs; and

using the revised control algorithm to move the virtual surgical robot within the virtual environment.

2 . The method of claim 1 , wherein the virtual surgical robot includes at least one of a robotic arm, a surgical robotic table, and a surgical tool attachment on the robotic arm.

3 . The method of claim 1 , wherein using the revised control algorithm to move the virtual surgical robot comprises causing the movement of the virtual surgical robot within the virtual environment according to the revised control algorithm.

4 . The method of claim 1 , wherein revising the control algorithm comprises:

evaluating the movement of the virtual surgical robot according to the control algorithm; and

adjusting the control algorithm based on the evaluation of the movement of the virtual surgical robot.

5 . The method of claim 4 further comprising repeating to move the virtual surgical robot using the revised control algorithm.

6 . The method of claim 4 , wherein the evaluation is based on reach or access of the virtual surgical robot responsive to the movement.

7 . The method of claim 4 , wherein the evaluation is based on access of virtual personnel within the virtual environment.

8 . The method of claim 1 , wherein revising the control algorithm comprises modifying at least one of transformations between movements of a user input device through which the user input is received, position commands to actuators of the corresponding surgical robot, gain parameters, damping parameters, and delays of the control algorithm.

9 . The method of claim 1 further comprising modifying, based on the movement of the virtual surgical robot, a workflow associated with a surgical robotic procedure.

10 . The method of claim 1 further comprising modifying, based on the movement of the virtual surgical robot, the kinematic data.

11 . The method of claim 10 further comprising evaluating the virtual surgical robot responsive to the movement to determine whether a surgical task using the virtual surgical robot is achievable, wherein modifying the kinematic data comprises adjusting, responsive to determining that the surgical task is not achieved, the geometry of the corresponding surgical robot.

12 . A system for development of a surgical robotic system, comprising at least one processor configured to:

render a virtual surgical robot within a virtual environment on a display, wherein the virtual surgical robot is built from kinematic data that defines at least a geometry of a corresponding surgical robot;

receive, through a user input device configured to control the virtual surgical robot, a user input to cause the virtual surgical robot to perform a movement;

revise, based on the performed movement of the virtual surgical robot, a control algorithm used to control the corresponding surgical robot or a tool coupled to the corresponding surgical robot based on user inputs; and

use the revised control algorithm to move the virtual surgical robot within the virtual environment.

13 . The system according to claim 12 , wherein the virtual surgical robot includes at least one of a robotic arm, a surgical robotic table, and a surgical tool attachment on the robotic arm.

14 . The system according to claim 12 , wherein using the revised control algorithm to move the virtual surgical robot comprises causing the movement of the virtual surgical robot within the virtual environment according to the revised control algorithm.

15 . The system according to claim 12 , wherein the at least one processor revises the control algorithm by:

evaluating the movement of the virtual surgical robot according to the control algorithm; and

adjusting the control algorithm based on the evaluation of the movement of the virtual surgical robot.

16 . The system according to claim 15 , wherein the evaluation is based on at least one of reach or access of the virtual surgical robot responsive to the movement, and access of virtual personnel within the virtual environment.

17 . The system according to claim 12 , wherein revising the control algorithm comprises modifying at least one of transformations between movements of a user input device through which the user input is received, position commands to actuators of the corresponding surgical robot, gain parameters, damping parameters, and delays of the control algorithm.

18 . The system according to claim 12 , wherein the at least one processor is further configured to modify, based on the movement of the virtual surgical robot, a workflow associated with a surgical robotic procedure.

19 . The system according to claim 12 , wherein the at least one processor is further configured to modify, based on the movement of the virtual surgical robot, the kinematic data.

20 . The system according to claim 19 , wherein the at least one processor is further configured to evaluate the virtual surgical robot responsive to the movement to determine whether a surgical task using the virtual surgical robot is achievable, wherein modifying the kinematic data comprises adjusting, responsive to determining that the surgical task is not achieved, the geometry of the corresponding surgical robot.

Assignments (1)
MERGER Recorded Jan 27, 2026
From: VERB SURGICAL INC.
To: AURIS HEALTH, INC.
Reel/Frame 073601/0736 →
Continuity (3)
Continuation 17486471 · Sep 27, 2021
Continuation 16870586 · May 8, 2020
Related Publication 20240245462A1 · Jul 25, 2024
References Cited (136)
US 4901575A · Bohannan et al. · 1990 [cited by applicant]
US 5448902A · Thoms et al. · 1995 [cited by applicant]
US 9643314B2 · Guerin et al. · 2017 [cited by applicant]
US 10712923B1 · Pathmanathan · 2020 [cited by applicant]
US 11166765B1 · Fuerst et al. · 2021 [cited by applicant]
US 11314846B1 · Colin et al. · 2022 [cited by applicant]
US 11896315B2 · Fuerst et al. · 2024 [cited by applicant]
US 11950850B2 · Fuerst et al. · 2024 [cited by applicant]
US 20020107674A1 · Bascle et al. · 2002 [cited by applicant]
US 20030109780A1 · Coste-Maniere et al. · 2003 [cited by applicant]
US 20040106916A1 · Quaid et al. · 2004 [cited by applicant]
US 20050004709A1 · Watanabe et al. · 2005 [cited by applicant]
US 20090177081A1 · Joskowicz et al. · 2009 [cited by applicant]
US 20090305210A1 · Guru et al. · 2009 [cited by applicant]
US 20100211897A1 · Cohen et al. · 2010 [cited by applicant]
US 20100234857A1 · Itkowitz et al. · 2010 [cited by applicant]
US 20110061011A1 · Hoguet · 2011 [cited by applicant]
US 20130245375A1 · Dimaio et al. · 2013 [cited by applicant]
US 20140018960A1 · Itkowitz · 2014 [cited by applicant]
US 20150032126A1 · Nowlin · 2015 [cited by examiner]
US 20160314710A1 · Jarc et al. · 2016 [cited by applicant]
US 20160332297A1 · Sugaya · 2016 [cited by applicant]
US 20170049517A1 · Felder et al. · 2017 [cited by applicant]
US 20170112576A1 · Coste-Maniere et al. · 2017 [cited by applicant]
US 20170215978A1 · Wallace et al. · 2017 [cited by applicant]
US 20170319282A1 · Jarc · 2017 [cited by examiner]
US 20170367771A1 · Tako et al. · 2017 [cited by applicant]
US 20180078319A1 · Nobles et al. · 2018 [cited by applicant]
US 20180098813A1 · Nesichi et al. · 2018 [cited by applicant]
US 20180116724A1 · Gmeiner et al. · 2018 [cited by applicant]
US 20180168750A1 · Staunton et al. · 2018 [cited by applicant]
US 20180233222A1 · Daley et al. · 2018 [cited by applicant]
US 20180263714A1 · Kostrzewski · 2018 [cited by examiner]
US 20180280099A1 · Cone et al. · 2018 [cited by applicant]
US 20180324539A1 · Lovitt · 2018 [cited by applicant]
US 20180338806A1 · Grubbs · 2018 [cited by applicant]
US 20180350145A1 · Byl et al. · 2018 [cited by applicant]
US 20190000578A1 · Yu et al. · 2019 [cited by applicant]
US 20190005838A1 · Yu et al. · 2019 [cited by applicant]
US 20190005848A1 · Garcia et al. · 2019 [cited by applicant]
US 20190008595A1 · Popovic et al. · 2019 [cited by applicant]
US 20190015167A1 · Draelos · 2019 [cited by examiner]
US 20190069962A1 · Tabandeh et al. · 2019 [cited by applicant]
US 20190083190A1 · Graves et al. · 2019 [cited by applicant]
US 20190279524A1 · Stoyanov et al. · 2019 [cited by applicant]
US 20190307510A1 · Rotenberg et al. · 2019 [cited by applicant]
US 20190318640A1 · Goel et al. · 2019 [cited by applicant]
US 20190325574A1 · Jin et al. · 2019 [cited by applicant]
US 20190327394A1 · Ramirez Luna et al. · 2019 [cited by applicant]
US 20190365499A1 · Nagao et al. · 2019 [cited by applicant]
US 20200038124A1 · Lin et al. · 2020 [cited by applicant]
US 20200060773A1 · Barral · 2020 [cited by examiner]
US 20200069373A1 · Yu · 2020 [cited by examiner]
US 20200069377A1 · Finley et al. · 2020 [cited by applicant]
US 20200074402A1 · Adato et al. · 2020 [cited by applicant]
US 20200078105A1 · Itkowitz et al. · 2020 [cited by applicant]
US 20200275976A1 · McKinnon · 2020 [cited by examiner]
US 20200297440A1 · Forstein · 2020 [cited by applicant]
US 20200337789A1 · Meglan · 2020 [cited by applicant]
US 20200405396A1 · Mcguan et al. · 2020 [cited by applicant]
US 20210121232A1 · Fuerst et al. · 2021 [cited by applicant]
US 20210192759A1 · Lang · 2021 [cited by applicant]
US 20210233429A1 · Barber et al. · 2021 [cited by applicant]
US 20210304637A1 · Garcia Kilroy et al. · 2021 [cited by applicant]
CA 3061333A1 · 2019 [cited by applicant]
DE 4242442A1 · 1994 [cited by applicant]
DE 4304170A1 · 1994 [cited by applicant]
DE 10340697A1 · 2004 [cited by applicant]
DE 102005034768A1 · 2007 [cited by applicant]
JP 2005022062A · 2005 [cited by applicant]
JP 2014102679A · 2014 [cited by applicant]
JP 2016221166A · 2016 [cited by applicant]
WO 2012001549A1 · 2012 [cited by applicant]
WO 2015095715A1 · 2015 [cited by applicant]
WO 2015121764A1 · 2015 [cited by applicant]
WO 2016149794A1 · 2016 [cited by applicant]
WO 2016194325A1 · 2016 [cited by applicant]
WO 2018195319A1 · 2018 [cited by applicant]
WO 2019006202A1 · 2019 [cited by applicant]
WO 2019051005A1 · 2019 [cited by applicant]
WO 2019089226A2 · 2019 [cited by applicant]
WO 2019139935A1 · 2019 [cited by applicant]
WO 2019148286A1 · 2019 [cited by applicant]
WO 2019164270A1 · 2019 [cited by applicant]
Advisory Action received for U.S. Appl. No. 17/486,471, mailed on Jul. 26, 2023, 4 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 17/486,471, mailed on May 10, 2023, 22 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/667,237 dated Mar. 31, 2021, 17 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/667,241 dated Apr. 27, 2021, 17 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/870,586, mailed on Mar. 31, 2021, 21 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/486,471, mailed on Aug. 31, 2023, 26 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/486,471, mailed on Oct. 12, 2022, 24 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/870,586, mailed on Jul. 20, 2021, 10 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/486,471, mailed on Dec. 12, 2023, 10 pages. [cited by applicant]
Advisory Action for U.S. Appl. No. 16/667,241 dated Nov. 2, 2021, 3 pages. [cited by applicant]
Advisory Action received for U.S. Appl. No. 16/667,237, mailed on Nov. 2, 2021, 3 pages. [cited by applicant]
Advisory Action received for U.S. Appl. No. 18/412,804, mailed on May 1, 2025, 3 pages. [cited by applicant]
Almusawi et al. (Simulation in virtual reality: Robotic training and surgical applications, Scientia Iranica, Sep. 2019) (Year: 2019). [cited by applicant]
Chen et al. (Image2scene: Transforming style of 3d room, ACM, 2015) (Year: 2015). [cited by applicant]
Decision to Grant a Patent received for Japanese Patent Application No. 2022-525369, mailed on Feb. 13, 2024, 5 pages (2 pages of English Translation and 3 pages of Original Document). [cited by applicant]
Final Office Action for U.S. Appl. No. 16/667,237 mailed Aug. 26, 2021, 15 pages. [cited by applicant]
Final Office Action for U.S. Appl. No. 16/667,241 mailed Aug. 26, 2021, 13 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/667,237, mailed on Mar. 16, 2022, 6 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 16/667,241, mailed on Mar. 16, 2022, 5 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 17/839,153, mailed on Jun. 22, 20237, 16 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 18/412,804, mailed on Feb. 21, 2025, 15 pages. [cited by applicant]
International Preliminary Report on Patentability for International Application No. PCT/US2019/064713 mailed May 12, 2022, 6 pages. [cited by applicant]
International Preliminary Report on Patentability for International Application No. PCT/US2020/031367 mailed Oct. 13, 2022, 6 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2019/064713 dated Jul. 21, 2020, 9 pages. [cited by applicant]
Lv et al. (Physically based real-time interactive assembly simulation of cable harness, Journal of Manufacturing Systems 43, 2017) (Year: 2017). [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/667,237 mailed Mar. 31, 2021, 17 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/667,237 mailed Nov. 22, 2021, 14 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/667,241 mailed Nov. 24, 2021, 15 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/840,144 mailed Aug. 4, 2022, 17 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/839,153, mailed on Feb. 28, 2023, 12 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/412,804, mailed on Aug. 28, 2024, 12 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/412,804, mailed on Jun. 16, 2025, 17 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/667,237, mailed on Apr. 27, 2022, 8 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/667,241, mailed on Apr. 26, 2022, 9 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/839,153, mailed on Oct. 3, 2023, 10 pages. [cited by applicant]
Notice of Final Rejection received for Korean Patent Application No. 10-2022-7018129, mailed on Apr. 23, 2025, 10 pages (5 pages of English Translation and 5 pages of Original Document). [cited by applicant]
Office Action received for Brazil Patent Application No. 112022007849-8, mailed on Apr. 24, 2025, 20 pages (10 pages of English Translation and 8 pages of Original Document). [cited by applicant]
Office Action received for Brazil Patent Application No. 112022007849-8, mailed on Nov. 30, 2023, 5 pages (1 page of English Translation and 4 pages of Original Document). [cited by applicant]
Office Action received for Japanese Patent Application No. 2022-525369, mailed on Nov. 7, 2023, 7 pages (4 pages of English Translation and 3 pages of Original Document). [cited by applicant]
Office Action received for Korean Patent Application No. 10-2022-7018129, mailed on Aug. 20, 2024, 8 pages. [cited by applicant]
Supplementary European Search Report and Search Opinion received for EP Application No. 19950367.3, mailed on Nov. 2, 2023, 11 pages. [cited by applicant]
Valentini (Interactive cable harnessing in augmented reality 2011) (Year: 2011). [cited by applicant]
Extended European Search Report received for EP Patent Application No. 20928854.7, mailed Mar. 21, 2024, 11 pages. [cited by applicant]
Examination Report received for EP Patent Application No. 20928854.7, mailed Jan. 24, 2025, 6 pages. [cited by applicant]
First Office Action received for Chinese Patent Application No. 202080099372.8, mailed Mar. 28, 2025, 21 pages (10 pages of Original Document and 11 pages of English Translation). [cited by applicant]
Notice of Allowance received for U.S. Patent Application No. 16/840, 144, mailed Feb. 6, 2023, 7 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 18/312,516, mailed Apr. 24, 2024, 8 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/780,350, mailed Jan. 14, 2026, 14 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/312,516, mailed Jan. 22, 2023, 15 pages. [cited by applicant]
International Search Report and The Written Opinion of The International Searching Authority received for PCT Patent Application No. PCT/US2020/031367, mailed Dec. 23, 2020, 13 pages. [cited by applicant]
Second Office Action received for Chinese Patent Application No. 201980101893.X, mailed Jan. 5, 2026, 21 pages (10 pages of Original Document and 11 pages of English Translation). [cited by applicant]
Tobias et al.; “Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods;” retrieved online: Doi 10.1109/JBHI.2014.2384134, IEEE Journal of Biomedical and Health Informa… [cited by applicant]