IP Library Granted Patent US 12,499,778
Granted Patent B2
US 12,499,778 · App. 18/503,674 · Granted Dec 16, 2025

Training users using indexed to motion pictures

Inventors: Anthony M. Jarc (Johns Creek, GA); Liheng Guo (Sunnyvale, CA)
Assignee: Intuitive Surgical Operations, Inc.
G09B5/02G06F3/011G06F3/0346G06F3/0482G09B19/24G16H40/63G16H70/20H04N5/76A61B34/37
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,499,778
App. No.
18/503,674
Granted
Dec 16, 2025
Kind
B2
Abstract

A system includes a sensor to sense a change in a position of an input control component and a tool; a storage medium embodies a collection of respective motion pictures, which show tool movement imparted by the control component in response to manual input provided by users, indexed based upon a metric; a storage medium embodies a set of instructions that cause a processor to perform operations comprising: determining a user's performance; recording a motion picture of the tool movement imparted by the master control component by the individual user; indexing the recorded motion picture to the collection of motion pictures; generating a user interface providing an interactive visual index; and displaying a selected motion picture on the display screen in response to user input at the interactive visual index.

Claims (32)

1 . A system comprising:

one or more processors, coupled to memory, and configured to:

identify a plurality of motion pictures, each motion picture of the plurality of motion pictures configured to show a movement of a tool of a surgical system controlled by user input received at the surgical system from a respective user, the plurality of motion pictures indexed by a respective level of performance of one or more users with respect to one or more metrics associated with the movement of the tool by the respective user;

display a visual index from which to select a motion picture among the plurality of motion pictures based at least on the one or more metrics, the visual index comprising a range of values of the one or more metrics;

receive a selection of a location on the visual index corresponding to a value of the range of values of the one or more metrics, the value corresponding to the respective level of performance; and

display a motion picture, from the plurality of motion pictures, indexed by the respective level of performance corresponding to the value of the range of values of one or more metrics at the selected location on the visual index.

2 . The system of claim 1 , wherein the one or metrics comprise a metric for an economy of motion of the respective user in movement of the tool, the economy of motion determined based at least on distance traveled by the tool.

3 . The system of claim 1 , wherein the one or more metrics comprise a metric for workspace range performance of the respective user based at least on volume of hand movements in using the tool.

4 . The system of claim 1 , wherein the one or more metrics comprise a metric for speed of movement of the tool by the respective user.

5 . The system of claim 1 , wherein the one or more processors are further configured to display the visual index in a user interface, wherein the visual index is interactive to select the location among a plurality of locations of the visual index.

6 . The system of claim 1 , wherein the visual index comprises a graphical representation of a relationship between a density of users at each respective level of performance.

7 . The system of claim 1 , wherein the one or more processors are further configured to display the level of performance of a user with respect to the visual index.

8 . The system of claim 1 , wherein the one or more processors are further configured to display an overall score of the one or more metrics among the one or more users.

9 . A computer-implemented method comprising:

identifying, by one or more processors, motion pictures configured to show a movement of a tool of a surgical system controlled by a respective user, each of the motion pictures indexed by a respective level of skill of a user with respect to a metric associated with the movement of the tool by the respective user;

generating, by the one or more processors, a visual index of a range of values of the metric;

receiving, by the one or more processors, an identification of a point of the visual index corresponding to the respective level of skill for a value of the metric in the range of values; and

showing, by the one or more processors, a display of a motion picture indexed by the respective level of skill corresponding to the identified value of the metric.

10 . The method of claim 9 , wherein the metric identifies an economy of motion of the respective user in movement of the tool, the economy of motion determined based at least on distance traveled by the tool.

11 . The method of claim 9 , wherein the metric identifies a camera movement frequency associated with the tool.

12 . The method of claim 9 , wherein the metric identifies a frequency of movement of the tool by the respective user.

13 . The method of claim 9 , wherein the one or more processors are further configured to display the visual index in a user interface, wherein the visual index is interactive to select the point among a plurality of points of the visual index.

14 . The method of claim 9 , wherein the visual index comprises a graphical representation of a relationship between a density of users at each respective level of skill.

15 . The method of claim 9 , wherein the one or more processors are further configured to display the level of skills of the user with respect to an overall score associated with the visual index.

16 . A non-transitory computer-readable medium comprising instructions configured to cause a computer system to perform operations comprising:

identifying motion pictures configured to show a movement of a tool of a surgical system controlled by a respective user, each of the motion pictures indexed by a respective level of performance of a user with respect to a manipulation metric associated with the movement of the tool by the respective user;

display a user interface providing a visual index of a range of values of the manipulation metric, each value in the range of values selectable by a visual indicator corresponding to the respective level of performance; and

show, responsive to selection of a value from the visual index by the visual indicator, a display of a motion picture indexed by the respective level of performance corresponding to the value of the manipulation metric.

17 . The non-transitory computer-readable medium of claim 16 wherein the manipulation metric comprises a metric associated with a motion of a wrist of the tool.

18 . The non-transitory computer-readable medium of claim 16 , wherein the manipulation metric identifies movement of the tool along one of a wrist roll axis, a wrist pitch axis or a wrist yaw axis.

19 . The non-transitory computer-readable medium of claim 16 , wherein the manipulation metric identifies wrist movement economy of motion of the tool.

20 . The non-transitory computer-readable medium of claim 16 , further comprising instructions configured to cause the computer system to display a user's level of performance for the manipulation metric.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2023
From: JARC, ANTHONY M.; GUO, LIHENG
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 065501/0096 →
Continuity (3)
Continuation 17293005
Provisional Application 62767872 · Nov 15, 2018
Related Publication 20240071243A1 · Feb 29, 2024
References Cited (15)
US 10791301B1 · Garcia et al. · 2020 [cited by applicant]
US 11847936B2 · Jarc et al. · 2023 [cited by applicant]
US 20080145830A1 · Huang et al. · 2008 [cited by applicant]
US 20140220527A1 · Li et al. · 2014 [cited by applicant]
US 20140287393A1 · Kumar et al. · 2014 [cited by applicant]
US 20170053543A1 · Agrawal et al. · 2017 [cited by applicant]
US 20170300752A1 · Biswas et al. · 2017 [cited by applicant]
US 20200333428A1 · Sun et al. · 2020 [cited by applicant]
US 20210407309A1 · Jarc et al. · 2021 [cited by applicant]
GB 2577714A · 2020 [cited by examiner]
WO WO2018089816A2 · 2018 [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2019/061879, mailed on May 27, 2021, 18 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2019/061879, mailed Feb. 13, 2020, 20 pages. [cited by applicant]
Jarc A., et al., “Viewpoint Matters: Objective Performance Metrics for Surgeon Endoscope Control during Robot-assisted Surgery,” Surgical Endoscopy, 2017, vol. 31 (3), pp. 1192-1202, 11 pages. [cited by applicant]
Vertut, J., and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]