IP Library Granted Patent US 9,965,464
Granted Patent B2
US 9,965,464 · App. 14/958,489 · Granted May 8, 2018

Automatic process guidance

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,965,464
App. No.
14/958,489
Granted
May 8, 2018
Kind
B2
Abstract

User interactions with a computing system are sensed and recorded. The recording represents a process for controlling a computer system. Voice input are received and the computer system actions that are taken based upon the voice inputs and a task recording.

Claims (69)

1. A computing system, comprising:

a processor; and

memory storing instructions executable by the processor, wherein the instructions, when executed, configure the computing system to provide:

a linguistic processing system configured to:

receive an indication of a task initiating speech input; and

generate a recognition result indicative of a task identified in the task initiating speech input; and

a script engine configured to:

based on the recognition result, identify a script that correspond to the task and includes step elements configured to generate step outputs indicative of computing steps to be performed in completing the task; and

run the script to:

generate the step outputs, in an order identified by the script and based on indications of responsive user inputs, wherein the linguistic processing system is configured to generate a speech output indicative of each of the step outputs generated by the script engine; and

generate machine instructions that execute the computing steps in the computing system based on the indications of the responsive user inputs.

2. The computing system of claim 1 , wherein the machine instructions comprise an interaction with a computing system component.

3. The computing system of claim 2 , wherein the computing system comprises a first computing system, and the computing system component comprises a component in a second computing system, that is separate from the first computing system.

4. The computing system of claim 1 , wherein the linguistic processing system comprises:

a speech recognition component configured to:

receive the indication of the task initiating speech input,

receive the indications of the responsive user inputs, and

generate a textual output that corresponds to one or more of the task initiating speech input and the responsive user inputs.

5. The computing system of claim 4 , wherein the linguistic processing system comprises:

a natural language understanding component configured to:

receive the textual output from the speech recognition system, and

generate an interpretation corresponding to the textual output, and

provide the interpretation to the script engine, as the recognition result.

6. The computing system of claim 5 , wherein the linguistic processing system comprises:

a text-to-speech component configured to:

generate the speech outputs based on the step outputs generated by the script engine.

7. The computing system of claim 1 , wherein the speech outputs, corresponding to the script outputs, comprise audible prompts indicative of a next step in the task.

8. The computing system of claim 1 , wherein the indications of responsive user inputs comprise indications of responsive user speech inputs, and the script engine is configured to generate the step outputs based on the indications of the responsive user speech inputs.

9. The computing system of claim 1 , wherein the indications of responsive user inputs comprise indications of responsive user touch inputs received from a touch sensitive user interface device, and the script engine is configured to generate the step outputs based on the indications of the responsive user touch inputs.

10. A computer implemented method, comprising:

receiving an indication of a task initiating speech input;

generating a recognition result indicative of a task identified in the task initiating speech input;

based on the recognition result, identifying a script that corresponds to the task and includes step elements configured to generate step outputs indicative of computing steps to be performed in completing the task; and

running the script by:

generating the step outputs, in an order identified by the script and based on indications of responsive user inputs;

generating a speech output indicative of each of the step outputs; and

generating machine instructions that execute the computing steps in a computing system based on the indications of the responsive user inputs.

11. The computer implemented method of claim 10 , wherein the machine instructions comprise an interaction with a computing system component.

12. The computer implemented method of claim 11 , wherein the machine instructions are generated by a first computing system, and the computing system component comprises a component in a second computing system that is separate from the first computing system.

13. The computer implemented method of claim 11 , wherein generating the recognition result comprises:

receiving the indications of the task initiating speech input and the responsive user inputs; and

generating a textual output that corresponds to one or more of the task initiating speech input and the responsive user inputs.

14. The computer implemented method of claim 13 , wherein generating the recognition result comprises:

generating an interpretation corresponding to the textual output; and

outputting the interpretation as the recognition result.

15. The computer implemented method of claim 10 , wherein generating the speech outputs, corresponding to the script outputs, comprises:

generating audible prompts indicative of a next step in the task.

16. The computer implemented method of claim 10 , wherein the indications of responsive user inputs comprise indications of responsive user speech inputs, and wherein generating the step outputs comprises:

generating the step outputs based on the indications of the responsive user speech inputs.

17. The computer implemented method of claim 10 , wherein the indications of responsive user inputs comprise indications of responsive user touch inputs received from a touch sensitive user interface device, and wherein generating the step outputs comprises:

generating the step outputs based on the indications of the responsive user touch inputs.

18. A computing system, comprising:

a processor; and

memory storing instructions executable by the processor, wherein the instructions, when executed, configure the computing system to provide:

a linguistic processing system configured to:

receive an indication of a task initiating speech input; and

generate a recognition result indicative of a task identified in the task initiating speech input;

a script engine configured to:

based on the recognition result, identify a script that corresponds to the task and includes step elements configured to generate step outputs indicative of computing steps to be performed in completing the task;

run the script to:

generate the step outputs, in an order identified by the script, wherein

the linguistic processing system is configured to generate speech outputs that are based on the step outputs represent audible instructions indicative of the steps,

the script engine is configured to receive indications of responsive user inputs that are responsive to the step outputs, and generate an additional step output based on indications of the responsive user inputs, and

the linguistic processing system is configured to generate an additional speech output indicative of the additional step output generated by the script engine; and

an interaction system configured to generate machine instructions that execute the computing steps in the computing system based on the indications of the responsive user inputs.

19. The computing system of claim 18 , wherein the linguistic processing system comprises:

a speech recognition component configured to receive an indications of the task initiating speech input and the responsive user inputs, and to generate a textual output that corresponds to one or more of the task initiating speech input and the responsive user inputs;

a natural language understanding component configured to receive the textual output from the speech recognition system, generate an interpretation corresponding to the textual output, and provide the interpretation to the script engine, as the recognition result; and

a text-to-speech component configured to generate the speech outputs based on the step outputs generated by the script engine.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2015
From: BONDARENKO, MAKSYM; DONGRE, MANALI; KHABAZAN, MARYAM; SHABAZZ, SHARRIEF; HAVE, HANS; GHOSH, DIPANJAN
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 037204/0438 →