Granted Patent B2

US 12,505,700 · App. 18/231,675 · Granted Dec 23, 2025

Inventors: Robert P. Dooley (Dublin, CA); Dylan James Snow (Concord, CA); Michael Kuniavsky (San Francisco, CA); Ronak Kumar Bhatia (Sunnyvale, CA)

Assignee: ACCENTURE GLOBAL SOLUTIONS LIMITED

G06V40/20G06T7/20G06T7/70G06V10/761G06T2207/30196

Abstract

Claims (60)

1 . A computer-implemented method comprising:

receiving a set of time series image frames within a time period including a plurality of time points;

identifying a first entity included in the set of time series image frames, wherein the first entity is coupled with a plurality of first positions corresponding to the plurality of time points;

identifying a second entity included in the set of time series image frames, wherein the second entity is coupled with a plurality of second positions corresponding to the plurality of time points;

determining a position difference of the first entity between any two consecutive time points in the time period;

determining a position difference of the second entity between any two consecutive time points in the time period;

determining an interaction between the first entity and the second entity over the time period based on i) the position difference of the first entity over the time period, and ii) the position difference of the second entity over the time period, comprises:

determining a relative distance between the first entity and the second entity at each time point based on the plurality of first positions and the plurality of second positions;

determining a difference of the relative distance between two consecutive time points; and

determining the interaction using the difference of the relative distance over the time period;

determining whether metadata of the interaction satisfies a threshold; and

providing feedback on the interaction indicating whether the metadata of the interaction satisfies the threshold.

2 . The computer-implemented method of claim 1 , wherein the metadata of the interaction comprises one or more of a duration of the interaction, an angle of the interaction, a type of the interaction, a position change of the first entity over the time period caused by the interaction, and a position change of the second entity over the time period caused by the interaction.

3 . The computer-implemented method of claim 1 , wherein the position difference comprises relative position difference and absolute position difference.

4 . The computer-implemented method of claim 1 , wherein the first entity corresponds to a human, the second entity corresponds to an object operated by the human, and wherein the interaction corresponds to a particular activity performed by the human on the object.

5 . The computer-implemented method of claim 1 , further comprising:

determining a plurality of interactions between an entity corresponding to a human and other entities, the plurality of interactions correspond to a plurality of subtasks of a task, the interaction comprising a subtask of the plurality of subtasks.

6 . The computer-implemented method of claim 1 , wherein determining whether metadata of the interaction satisfies a threshold comprises:

executing a machine learning model using the set of time series image frames as input of the machine learning model, wherein the machine learning model is previously trained using training data set corresponding to standard operations of the interaction.

7 . The computer-implemented method of claim 1 , wherein each image frame of the set of time series image frames is associated with a time stamp that corresponds to a time point of the plurality of time points.

8 . One or more non-transitory computer-readable storage media coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving a set of time series image frames within a time period including a plurality of time points;

identifying a first entity included in the set of time series image frames, wherein the first entity is coupled with a plurality of first positions corresponding to the plurality of time points;

identifying a second entity included in the set of time series image frames, wherein the second entity is coupled with a plurality of second positions corresponding to the plurality of time points;

determining a position difference of the first entity between any two consecutive time points in the time period;

determining a position difference of the second entity between any two consecutive time points in the time period;

determining an interaction between the first entity and the second entity over the time period based on i) the position difference of the first entity over the time period, and ii) the position difference of the second entity over the time period, comprises:

determining a relative distance between the first entity and the second entity at each time point based on the plurality of first positions and the plurality of second positions;

determining a difference of the relative distance between two consecutive time points; and

determining the interaction using the difference of the relative distance over the time period;

determining whether metadata of the interaction satisfies a threshold; and

providing feedback on the interaction indicating whether the metadata of the interaction satisfies the threshold.

9 . The one or more non-transitory computer-readable storage media of claim 8 , wherein the metadata of the interaction comprises one or more of a duration of the interaction, an angle of the interaction, a type of the interaction, a position change of the first entity over the time period caused by the interaction, and a position change of the second entity over the time period caused by the interaction.

10 . The one or more non-transitory computer-readable storage media of claim 8 , wherein the position difference comprises relative position difference and absolute position difference.

11 . The one or more non-transitory computer-readable storage media of claim 8 , wherein the first entity corresponds to a human, the second entity corresponds to an object operated by the human, and wherein the interaction corresponds to a particular activity performed by the human on the object.

12 . The one or more non-transitory computer-readable storage media of claim 8 , wherein the operations further comprise:

determining a plurality of interactions between an entity corresponding to a human and other entities, the plurality of interactions correspond to a plurality of subtasks of a task, the interaction comprising a subtask of the plurality of subtasks.

13 . The one or more non-transitory computer-readable storage media of claim 8 , wherein determining whether metadata of the interaction satisfies a threshold comprises:

executing a machine learning model using the set of time series image frames as input of the machine learning model, wherein the machine learning model is previously trained using training data set corresponding to standard operations of the interaction.

14 . The one or more non-transitory computer-readable storage media of claim 8 , wherein each image frame of the set of time series image frames is associated with a time stamp that corresponds to a time point of the plurality of time points.

15 . A system, comprising:

one or more processors; and

a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving a set of time series image frames within a time period including a plurality of time points;

identifying a first entity included in the set of time series image frames, wherein the first entity is coupled with a plurality of first positions corresponding to the plurality of time points;

identifying a second entity included in the set of time series image frames, wherein the second entity is coupled with a plurality of second positions corresponding to the plurality of time points;

determining a position difference of the first entity between any two consecutive time points in the time period;

determining a position difference of the second entity between any two consecutive time points in the time period;

determining an interaction between the first entity and the second entity over the time period based on i) the position difference of the first entity over the time period, and ii) the position difference of the second entity over the time period, comprises:

determining a relative distance between the first entity and the second entity at each time point based on the plurality of first positions and the plurality of second positions;

determining a difference of the relative distance between two consecutive time points; and

determining the interaction using the difference of the relative distance over the time period;

determining whether metadata of the interaction satisfies a threshold; and

providing feedback on the interaction indicating whether the metadata of the interaction satisfies the threshold.

16 . The system of claim 15 , wherein the metadata of the interaction comprises one or more of a duration of the interaction, an angle of the interaction, a type of the interaction, a position change of the first entity over the time period caused by the interaction, and a position change of the second entity over the time period caused by the interaction.

17 . The system of claim 15 , wherein the position difference comprises relative position difference and absolute position difference.

18 . The system of claim 15 , wherein the first entity corresponds to a human, the second entity corresponds to an object operated by the human, and wherein the interaction corresponds to a particular activity performed by the human on the object.

19 . The system of claim 15 , wherein the operations further comprise:

determining a plurality of interactions between an entity corresponding to a human and other entities, the plurality of interactions correspond to a plurality of subtasks of a task, the interaction comprising a subtask of the plurality of subtasks.

20 . The system of claim 15 , wherein each image frame of the set of time series image frames is associated with a time stamp that corresponds to a time point of the plurality of time points.

Assignments (1)

Continuity (1)

References Cited (16)