Method of and system for real time feedback in an incremental speech input interface

1. A computer-implemented method for selecting and presenting content items based on user input, the method comprising:

providing access to a set of content items, the content items being associated with metadata that describes a corresponding content item;

receiving a first input intended by the user to identify at least one desired content item;

determining that a portion of the first input has a measure of importance that exceeds a threshold value;

providing feedback to the user identifying the portion of the first input;

receiving second input from the user that is subsequent to the first input;

determining a measure of similarity between the portion of the first input and the second input;

comparing the measure of similarity with a threshold;

upon a condition when the measure of similarity is determined to be above the threshold, determining that the portion of the first input is phonetically similar to the second input;

upon a condition when the measure of similarity is determined to not be above the threshold, determining that the portion of the first input is not phonetically similar to the second input;

based on determining that the portion of the first input is phonetically similar to the second input, determining an alternative query input by altering the first input based on the second input;

based on determining that the portion of the first input is not phonetically similar to the second input, determining the alternative query input by supplementing the first input with the second input;

selecting a subset of content items from the set of content items based on comparing the alternative query input and the metadata associated with the subset of content items; and

presenting the subset of content items to the user.

2. The method of claim 1 ,

wherein the determining that the portion of the first input has the measure of importance that exceeds the threshold value includes identifying one or more phrase boundaries in incremental input of the first input, and

wherein the identifying the one or more phrase boundaries is based at least in part on at least one of (a) an identified disfluency from the user in the first input, (b) grammar rules applied to the first input, (c) the importance measure of the portion of the first input, (d) at least one previous conversational interaction with the user, and (e) a user preference signature,

the user preference signature describing preferences of the user for at least one of (i) particular content items and (ii) particular metadata associated with the content items, wherein the portion of the first input is identified based on the user preference signature,

wherein the disfluency includes at least one of a pause in speech input, an auditory time filler in speech input, and a pause in typing input.

3. The method of claim 1 , wherein selecting the subset of content items is based further upon a disfluency identified in the first input, and based further upon prior conversational interactions that are determined to be related to the first input and the second input.

4. The method of claim 1 , wherein the providing the feedback includes at least one of:

requesting clarification on the identified portion of the first input,

suggesting a completion of the received first input, and

repeating the portion of the first input to the user, so as to notify the user that the portion of the first input is potentially incorrectly recognized.

5. The method of claim 4 , wherein the requesting clarification on the identified portion of the first input is based at least in part on a determination that a disfluency occurs after the user has provided the portion of the first input.

6. The method of claim 4 , wherein the suggesting the completion of the received first input is based at least in part on a determination that the disfluency occurs before the user would be expected to provide the portion of the first input.

7. The method of claim 1 , wherein the feedback provided to the user is chosen based on at least one of:

a duration of an identified disfluency in the first input,

a measure of confidence in correct speech-to-text recognition of the portion of the first input,

a count of ambiguities detected in the first input,

a count of error corrections needed to identify the portion of the first input,

a count of nodes in a graph data structure, wherein the count of the nodes in the graph data structure measures a path between a first node representing an item of interest from a previous conversational interaction and a second node representing the portion of the first input, and

a measure of relatedness of the portion of the first input to previous conversational interactions with the user.

8. The method of claim 1 , wherein the presenting the subset of content items includes presenting the subset of content items before receiving completed input from the user, upon a determination of a strong recognition match for the first input and upon a determination that a responsiveness measure of the selected subset of content items would be above a threshold.

9. A system for selecting and presenting content items based on user input, the system comprising:

computer readable instructions encoded on a non-transitory computer readable medium, the computer readable instructions causing the computer system to be configured to:

provide access to a set of content items, the content items being associated with metadata that describes a corresponding content item;

receive a first input intended by the user to identify at least one desired content item;

determine that a portion of the first input has a measure of importance that exceeds a threshold value;

provide feedback to the user identifying the portion of the first input;

receive second input from the user that is subsequent to the first input;

determine a measure of similarity between the portion of the first input and the second input;

compare the measure of similarity to with threshold;

upon a condition when the measure of similarity is determined to be above the threshold, determine that the portion of the first input is phonetically similar to the second input;

upon a condition when the measure of similarity is determined not to be above the threshold, determine that the portion of the first input is not phonetically similar to the second input;

based on determining that the portion of the first input is phonetically similar to the second input, determine an alternative query input by altering the first input based on the second input;

based on determining that the portion of the first input is not phonetically similar to the second input, determine the alternative query input by supplementing the first input with the second input;

select a subset of content items from the set of content items based on comparing the alternative query input and the metadata associated with the subset of content items; and

present the subset of content items to the user.

10. The system of claim 9 ,

wherein the determination that the portion of the first input has the measure of importance that exceeds the threshold value includes computer-readable instructions causing the system to be configured to identify one or more phrase boundaries in incremental input of the first input, and

wherein the identification of the one or more phrase boundaries is based at least in part on at least one of (a) an identified disfluency from the user in the first input, (b) grammar rules applied to the first input, (c) the importance measure of the portion of the first input, (d) at least one previous conversational interaction with the user, and (e) a user preference signature, and

wherein the user preference signature describes preferences of the user for at least one of (i) particular content items and (ii) particular metadata associated with the content items, wherein the portion of the first input is identified based on the user preference signature,

wherein the disfluency includes at least one of a pause in speech input, an auditory time filler in speech input, and a pause in typing input.

11. The system of claim 9 , wherein the selection of the subset of content items is based further upon a disfluency identified in the first input, and based further upon prior conversational interactions that are determined to be related to the first input and the second input.

12. The system of claim 9 , wherein the computer-readable instructions causing the system to be configured to provide the feedback includes at least one of:

computer-readable instructions causing the system to be configured to request clarification on the identified portion of the first input,

computer-readable instructions causing the system to be configured to suggest a completion of the received first input, and

computer-readable instructions causing the system to be configured to repeat the portion of the first input to the user, so as to notify the user that the portion of the first input is potentially incorrectly recognized.

13. The system of claim 12 , wherein the request for clarification on the identified portion of the first input is based at least in part on a determination that a disfluency occurs after the user has provided the portion of the first input.

14. The system of claim 12 , wherein the suggestion of the completion of the received first input is based at least in part on a determination that the disfluency occurs before the user would be expected to provide the portion of the first input.

15. The system of claim 9 , wherein the feedback provided to the user is chosen based on at least one of:

a duration of an identified disfluency in the first input,

a measure of confidence in correct speech-to-text recognition of the portion of the first input,

a count of ambiguities detected in the first input,

a count of error corrections needed to identify the portion of the first input,

a count of nodes in a graph data structure, wherein the count of the nodes in the graph data structure measures a path between a first node representing an item of interest from a previous conversational interaction and a second node representing the portion of the first input, and

a measure of relatedness of the portion of the first input to previous conversational interactions with the user.

16. The system of claim 9 , wherein the computer-readable instructions causing the system to be configured to present the subset of content items includes computer-readable instructions causing the system to be configured to present the subset of content items before receiving completed input from the user, upon a determination of a strong recognition match for the first input and upon a determination that a responsiveness measure of the selected subset of content items would be above a threshold.

17. The method of claim 1 , wherein determining the measure of similarity between the portion of the first input and the portion of the second input includes detecting similar characters in the portion of the first input and the portion of the second input.

18. The method of claim 1 , wherein determining the measure of similarity between the portion of the first input and the portion of the second input includes detecting similar phonetic variations in the portion of the first input and the portion of the second input.

19. The system of claim 9 , wherein the computer readable instructions cause the computer system, when determining the measure of similarity between the portion of the first input and the portion of the second input, to determine the measure of similarity between the portion of the first input and the portion of the second input by detecting similar characters in the portion of the first input and the portion of the second input.

20. The system of claim 9 , wherein the computer readable instructions cause the computer system, when determining the measure of similarity between the portion of the first input and the portion of the second input, to determine the measure of similarity between the portion of the first input and the portion of the second input by detecting similar phonetic variations in the portion of the first input and the portion of the second input.