IP Library Granted Patent US 9,721,587
Granted Patent B2
US 9,721,587 · App. 13/749,392 · Granted Aug 1, 2017

Visual feedback for speech recognition system

Inventors: Christian Klein (Duvall, WA); Meg Niman (Seattle, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G10L21/10G06F3/0304G06F3/167G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,721,587
App. No.
13/749,392
Filed
Jan 24, 2013
Granted
Aug 1, 2017
Kind
B2
Art Unit
2659
USPC
704/235
Abstract

Embodiments are disclosed that relate to providing visual feedback in a speech recognition system. For example, one disclosed embodiment provides a method including displaying a graphical feedback indicator having a variable appearance dependent upon a state of the speech recognition system. The method further comprises receiving a speech input, modifying an appearance of the graphical feedback indicator in a first manner if the speech input is heard and understood by the system, and modifying the appearance of the graphical feedback indicator in a different manner than the first manner if the speech input is heard and not understood.

Claims (37)

1. On a computing device, a method of providing user feedback for a speech recognition system, the method comprising:

displaying a graphical feedback indicator with an appearance that indicates the speech recognition system is in a passive listening mode;

receiving a triggering input;

in response to the triggering input, modifying an appearance of the graphical feedback indicator to have an appearance that indicates the speech recognition system is in an active listening mode;

while in the active listening mode, receiving a speech input;

when the speech recognition system understands the speech input, modifying the appearance of the graphical feedback indicator to have an appearance that indicates that the speech input is understood by the system, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input; and

when the speech recognition system does not understand the speech input, then modifying the appearance of the graphical feedback indicator in a different manner to have an appearance that indicates the speech input is not understood by the system.

2. The method of claim 1 , further comprising determining the volume of the speech input and modifying a length of the volume indicator to indicate the volume.

3. The method of claim 1 , further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to display one or more words of the speech input in real-time as each word is recognized.

4. The method of claim 1 , further comprising identifying the user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate an identity of the user.

5. The method of claim 1 , wherein modifying the appearance of the graphical feedback indicator in the different manner comprises displaying a prompt for the user to provide additional user input.

6. The method of claim 1 , further comprising modifying the appearance of the graphical feedback indicator from the passive listening mode to a different, active listening mode by maintaining a shape of the graphical feedback indicator while modifying the appearance.

7. A method of providing feedback for a speech recognition system of a computing device, the method comprising:

displaying a graphical feedback indicator with an appearance indicating that the speech recognition system is in a passive listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;

receiving a triggering input;

in response to the triggering input, modifying the appearance of the graphical feedback indicator to have an appearance that indicates the speech recognition system is in an active listening mode;

while in the active listening mode, receiving a speech input;

when the speech input is heard, modifying the appearance of the graphical feedback indicator to indicate that the speech input is heard;

when the speech input is understood, modifying the appearance of the graphical feedback indicator to indicate that the speech input is understood, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input; and

when the speech input is not understood, modifying the appearance of the graphical feedback indicator to indicate that the speech is not understood.

8. The method of claim 7 , wherein the appearance that indicates that the speech recognition system is in the active listening mode comprises an appearance that indicates the speech recognition system is in the active listening mode in a local context.

9. The method of claim 7 , wherein the appearance that indicates that the speech recognition system is in the active listening mode comprises an appearance that indicates that the speech recognition system is in the active listening mode in a global context.

10. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to indicate one or more words of the speech input as the speech input is recognized.

11. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator to identify one or more words corresponding to a canonical form of a speech command corresponding to the speech input.

12. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator to indicate an uncertainty in a recognition of the speech input.

13. The method of claim 12 , wherein indicating the uncertainty in the recognition of the speech input includes displaying two or more possible speech recognition results for the speech input.

14. A computing system for performing speech recognition and providing feedback regarding the speech recognition, the computing system comprising:

a logic machine; and

a storage machine comprising instructions executable by the logic machine to:

output to a display device a graphical feedback indicator with an appearance indicating that a speech recognition system is in a passive listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;

receiving a triggering input;

in response to the triggering input, modifying an appearance of the graphical feedback indicator to have an appearance that indicates that the speech recognition system is in an active listening mode;

receive, from one or more microphones, a speech input;

when the speech input is heard, modify the appearance of the graphical feedback indicator to indicate that the speech input is heard; and

when the speech input is understood, modify the appearance of the graphical feedback indicator to indicate that the speech input is understood, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input.

15. The computing system of claim 14 , the instructions being further executable to modify the appearance of the graphical feedback indicator to indicate whether the speech input is to be applied in a global operating system context or a local application context.

16. The computing system of claim 14 , the instructions being further executable to determine an identity of the user, and to modify the appearance of the graphical feedback indicator to display the identity of the user.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2013
From: KLEIN, CHRISTIAN; NIMAN, MEG
To: MICROSOFT CORPORATION
Reel/Frame 029690/0846 →
Continuity (1)
Related Publication 20140207452A1 · Jul 24, 2014