Granted Patent B2

US 12,499,681 · App. 17/875,126 · Granted Dec 16, 2025

Inventors: Meysam Torabi (Union City, CA); Varun Goel (Santa Clara, CA); Jocelyn Elaine Barker (San Jose, CA); Rami Abukhalil (Santa Clara, CA); Richard W. Timm (Cincinnati, OH); Pablo E. Garcia Kilroy (Santa Clara, CA)

Assignee: Verb Surgical Inc.

G06V20/44G06V10/70G06V20/41G06V20/46G06V20/49G16H50/70G16H70/20G06V2201/034

Abstract

Claims (82)

1 . A computer-implemented method, comprising:

receiving a plurality of surgical videos of a surgical procedure involving an energy tool;

for each surgical video in the plurality of surgical videos,

detecting a set of activation events in the surgical video, wherein each activation event includes an identified starting timestamp and a duration, by:

applying a sequence of sampling windows to the surgical video to generate a sequence of windowed samples of the surgical video;

for each windowed sample in the sequence of windowed samples, applying a deep-learning model to a sequence of video frames within the windowed sample to generate an activation/non-activation inference and a confidence level associated with the activation/non-activation inference, thereby generating a sequence of activation/non-activation inferences and a sequence of associated confidence levels; and

identifying the set of activation events based on the sequence of activation/non-activation inferences and the sequence of associated confidence levels;

extracting a set of energy tool usage data based on the set of activation events; and

storing the set of energy tool usage data in a database indexed based on a set of energy tool usage metrics; and

in response to a user search request on a usage metric, presenting to the user the energy tool usage data that matches the requested usage metric from the database.

2 . The computer-implemented method of claim 1 , wherein the plurality of surgical videos is associated with a plurality of surgical cases performed by a plurality of surgeons.

3 . The computer-implemented method of claim 1 , wherein the plurality of surgical videos is associated with a plurality of surgical cases performed using a variety of energy tool types and models.

4 . The computer-implemented method of claim 1 , wherein storing the extracted set of energy tool usage data in the database includes:

separating the extracted set of energy tool usage data into a set of categories in the database corresponding to the set of energy tool usage metrics; and

further separating the extracted energy tool usage data within each category of the set of categories into a set of subcategories of the corresponding energy tool usage metric.

5 . The computer-implemented method of claim 4 , wherein the set of energy tool usage metrics includes one or more of:

the energy dosage;

the number of activations;

the energy tool on-screen presence duration; and

the number of activations per unit time.

6 . The computer-implemented method of claim 5 , wherein the set of subcategories corresponding to the energy dosage metric includes:

Low;

Normal; and

High.

7 . The computer-implemented method of claim 5 , wherein the set of subcategories corresponding to the number of activations metric includes:

Frequent;

Average; and

Infrequent.

8 . The computer-implemented method of claim 5 , wherein the set of subcategories corresponding to the energy tool on-screen presence duration metric includes:

Short;

Normal; and

Long.

9 . The computer-implemented method of claim 5 , wherein the set of subcategories corresponding to the number of activations per unit time metric includes:

Low;

Normal; and

High.

10 . A computer-implemented method, comprising:

receiving a plurality of surgical videos of a surgical procedure involving an energy tool;

for each surgical video in the plurality of surgical videos,

detecting a set of activation events in the surgical video, wherein each activation event includes an identified starting timestamp and a duration,

detecting a set of on-screen presence events of the energy tool for a corresponding surgical task in the surgical procedure involving the energy tool, wherein each on-screen presence event includes the identified starting timestamp and the duration of one or more of the set of activation events,

extracting a set of energy tool usage data by superimposing the set of activation events and the set of on-screen presence events to identify within the duration of each on-screen presence event a subset of the set of activation events as associated with the corresponding surgical task,

extracting a set of energy tool usage data associated with the corresponding surgical task based on the subset, and

storing the set of energy tool usage data in a database indexed based on a set of energy tool usage metrics; and

in response to a search request from a user on a requested energy tool usage metric, presenting to the user from the database the set of energy tool usage data that matches the requested energy tool usage metric.

11 . The computer-implemented method of claim 10 , wherein extracting the set of energy tool usage data associated with the corresponding surgical task includes determining an energy dosage delivered during the corresponding surgical task based on a total activation duration of the subset.

12 . The computer-implemented method of claim 11 , wherein extracting the set of energy tool usage data associated with the corresponding surgical task further includes:

computing a nominal number of activations of the corresponding surgical task for the plurality of surgical videos by averaging the number of activations of the corresponding surgical task for the plurality of surgical videos; and

computing a nominal energy dosage used in the corresponding surgical task for the plurality of surgical videos by averaging the energy dosages of the corresponding surgical task for the plurality of surgical videos.

13 . The computer-implemented method of claim 10 , wherein extracting the set of energy tool usage data associated with the corresponding surgical task further includes:

computing an activation momentum metric as the ratio of the total number of activation events within the duration of the detected on-screen presence event to the duration of the detected on-screen presence event.

14 . The computer-implemented method of claim 10 , wherein extracting the set of energy tool usage data associated with the corresponding surgical task further includes:

computing a combined duration of the set of detected on-screen presence events;

counting a total number of the detected activation events in the surgical video; and

computing a surgical momentum metric as the ratio of the total number of the detected activation events in the surgical video to the combined duration of the set of detected on-screen presence events.

15 . A system, comprising:

one or more processors; and

a memory coupled to the one or more processors;

wherein the memory stores a set of instructions that, when executed by the one or more processors, cause the system to:

receive a plurality of surgical videos of a surgical procedure involving an energy tool;

for each surgical video in the plurality of surgical videos,

detect a set of activation events in the surgical video, wherein each activation event includes an identified starting timestamp and a duration, by:

applying a sequence of sampling windows to the surgical video to generate a sequence of windowed samples of the surgical video,

for each windowed sample in the sequence of windowed samples, applying a deep-learning model to a sequence of video frames within the windowed sample to generate an activation/non-activation inference and a confidence level associated with the activation/non-activation inference, thereby generating a sequence of activation/non-activation inferences and a sequence of associated confidence levels, and

identifying the set of activation events based on the sequence of activation/non-activation inferences and the sequence of associated confidence levels;

extract a set of energy tool usage data based on the set of activation events; and

store the set of energy tool usage data in a database indexed based on a set of energy tool usage metrics; and

in response to a user search request on a usage metric, present the stored energy tool usage data that matches the requested usage metric from the database.

16 . The system of claim 15 , wherein the set of energy tool usage metrics includes one or more of:

the energy dosage;

the number of activations;

the energy tool on-screen presence duration; and

the number of activations per unit time.

17 . The system of claim 15 , wherein the set of energy tool usage metrics includes one or more of:

an energy dosage metric;

a number of activations;

an energy tool on-screen presence duration; and

a number of activations per unit time.

18 . The system of claim 17 , wherein a set of subcategories corresponding to the energy dosage metric includes:

low;

normal; and

high.

Assignments (2)

Continuity (1)

References Cited (12)