IP Library › Granted Patent US 10,922,105
Granted Patent B1
US 10,922,105 · App. 16/806,600 · Granted Feb 16, 2021

Realtime generated assistance video

Inventors: Pramod Verma (Fairfax, VA); Vijender Koorella (Austin, TX); Seema Nagar (Bangalore, IN); Kuntal Dey (New Delhi, IN)
Assignee: International Business Machines Corporation
G06F9/453G06F9/4881G06F11/3438G06F3/0482G06F11/3476H04L67/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,922,105
App. No.
16/806,600
Granted
Feb 16, 2021
Kind
B1
Abstract

A user performing a first process on a first computing device is detected. The first process includes one or more discrete tasks. A process flow graph, that is generated before performance of the first task, related to the first process is retrieved based on detection of the first process. A user action that relates to the one or more discrete tasks of the first process is identified based on the first process and based on the process flow graph. A current state of the first process is determined based on the first process and based on the process flow graph. A first video is generated based on the current state of the first process and based on the user action. The first video depicts one or more future actions that may be performed by the user to successfully perform the first process on the first computing device.

Claims (92)

1. A computer-implemented method comprising:

detecting a user performing a first process on a first computing device, wherein the first process includes one or more discrete tasks;

retrieving, based on detection of the first process, a process flow graph related to the first process, the process flow graph having been generated before the performing of the first task;

identifying, based on the first process and based on the process flow graph, a user action that relates to the one or more discrete tasks of the first process;

determining, based on the process flow graph and based on identification of the user action, a current state of the first process; and

generating, based on the current state of the first process and based on the user action, a first video that depicts one or more future actions that may be performed by the user to successfully perform the first process on the first computing device.

2. The method of claim 1 , further comprising:

detecting that the user is unsuccessful in performing the first process; and

providing, based on detecting the user is unsuccessful, the first video to the user.

3. The method of claim 2 , wherein the detecting the user is unsuccessful comprises:

monitoring a first input rate of the first user at a first time period;

monitoring a second input rate of the first user at a second time period; and

determining the second input rate falls below a predefined threshold value.

4. The method of claim 2 , wherein the detecting the user is unsuccessful comprises:

receiving, by an automated assistant, a request from the user for help in performing the first process.

5. The method of claim 2 , wherein detecting the user is unsuccessful comprises:

monitoring a first input of the first user at a first time;

monitoring a second input of the first user at a second time;

comparing the first input to the second input; and

determining the first input is the same as the second input.

6. The method of claim 2 , wherein:

the detection of the user performing the first process occurs at a first time period, and

the detecting that the user is unsuccessful comprises:

monitoring input actions of the first user on the first computing device during the first time period;

detecting, based on the input actions, one or more keywords indicative of an information seeking behavior; and

determining, based on the one or more keywords, the information seeking behavior is related to the performance of the first process.

7. The method of claim 1 , further comprising:

generating, based on the current state of the first process and based on the user action, a second video that depicts one or more prior actions that were performed by the user to achieve the current state.

8. The method of claim 7 , further comprising:

detecting the user is unsuccessful in the performing of the first process;

providing, based on the detecting that the user is unsuccessful, the first video to the user; and

providing, based on the detecting that the user is unsuccessful, the second video to the user.

9. The method of claim 1 , wherein the identification of the user action comprises:

scanning a memory of the first computing device; and

detecting, based on the scanning, a first file that matches one or more parameters of the process flow graph.

10. The method of claim 1 , wherein identification of the user action comprises:

scanning an IO entity that is at least one of an input and an output of the first computing device; and

detecting, based on the scanning of the IO entity, a first input received by the first computing device that matches one or more parameters of the process flow graph.

11. The method of claim 1 , wherein identification of the user action comprises:

scanning an IO entity that is at least one of an input and an output of the first computing device; and

detecting, based on the scanning of the IO entity, a first output of the first computing device that matches one or more parameters of the process flow graph.

12. The method of claim 1 , wherein the generation of the first video comprises:

retrieving the process flow graph;

generating an instruction computing instance, the instruction computing instance comprising one or more files indicated by the process flow graph and one or more configuration settings indicated by the process flow graph;

performing, on the instruction computing instance, a subset of the one or more discrete tasks that occur before the current state;

initiating a screen-recording of the instruction computing instance;

performing a second subset of the one or more discrete tasks, the second subset related to successful performance of the first process from the current state; and

transmitting the screen-recording to the first computing device.

13. The method of claim 12 , wherein:

the first process is performed on a first computing instance on the first computing device;

the instruction computing instance is on a second computing device; and

the transmitting of the screen-recording comprises:

sending, through a network, the screen-recording to the first computing instance.

14. The method of claim 12 , wherein:

the first process is performed on a first computing instance on the first computing device;

the instruction computing instance is a second computing instance performed on the first computing device; and

the transmitting of the screen-recording comprises:

providing, by the second computing instance and to the first computing instance, one or more access permissions to the screen-recording.

15. A system, the system comprising:

a memory, the memory containing one or more instructions; and

a processor, the processor communicatively coupled to the memory, the processor, in response to reading the one or more instructions, configured to:

detect a user performing a first process on a first computing device, wherein the first process includes one or more discrete tasks;

retrieve, based on detection of the first process, a process flow graph related to the first process, the process flow graph generated before performance of the first task;

identify, based on the first process and based on the process flow graph, a user action that relates to the one or more discrete tasks of the first process;

determine, based on the process flow graph and based on identification of the user action, a current state of the first process;

generate, based on the current state of the first process and based on the user action, a first video that depicts one or more future actions that may be performed by the user to successful perform the first process on the first computing device.

16. The system of claim 15 , the processor further configured to:

generate, based on the current state of the first process and based on the user action, a second video that depicts one or more prior actions that were performed by the user to achieve the current state.

17. The system of claim 15 , wherein the processor is further configured to:

detect the user is unsuccessful in performing the first process;

provide, based on detecting the user is unsuccessful, the first video to the user; and

provide, based on detecting the user is unsuccessful, the second video to the user.

18. A computer program product, the computer program product comprising:

one or more computer readable storage media; and

program instructions collectively stored on the one or more computer readable storage media, the program instructions configured to:

detect a user performing a first process on a first computing device, wherein the first process includes one or more discrete tasks;

retrieve, based on detection of the first process, a process flow graph related to the first process, the process flow graph generated before performance of the first task;

identify, based on the first process and based on the process flow graph, a user action that relates to the one or more discrete tasks of the first process;

determine, based on the process flow graph and based on identification of the user action, a current state of the first process;

generate, based on the current state of the first process and based on the user action, a first video that depicts one or more future actions that may be performed by the user to successful perform the first process on the first computing device.

19. The computer program product of claim 18 , wherein generation of the first video comprises:

retrieving the process flow graph;

generating an instruction computing instance, the instruction computing instance including one or more files indicated by the process flow graph and one or more configuration settings indicated by the process flow graph;

performing, on the instruction computing instance, a subset of the one or more discrete tasks that occur before the current state;

initiating a screen-recording of the instruction computing instance;

performing a second subset of the one or more discrete tasks, the second subset related to successful performance of the first process from the current state; and

transmitting the screen-recording to the first computing device.

20. The computer program product of claim 19 , wherein:

the first process is performed on a first computing instance on the first computing device,

the instruction computing instance is a second computing instance performed on the first computing device; and

transmitting the screen-recording comprises:

providing, by the second computing instance and to the first computing instance, one or more access permissions to the screen-recording.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: KYNDRYL, INC.
Reel/Frame 057885/0644 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2020
From: VERMA, PRAMOD; KOORELLA, VIJENDER; NAGAR, SEEMA; DEY, KUNTAL
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 051982/0478 →
Cited By (7)
US 12,261,996 US 12,277,743 US 12,555,000 US 12,639,045 US 12,659,550 US 12,675,977 US 12,684,110