System and method for analysis and adjustment of speech-enabled systems
View Patent ↗A method for analyzing and adjusting the performance of a speech-enabled application includes selecting a number of user utterances that were previously received by the speech-enabled application. The speech-enabled application receives such user utterances and associates each user utterance with an action-object based on one or more salient terms in the user utterance that are associated with the action-object. The method further includes associating one of a number of action-objects with each of the selected user utterances. Furthermore, for each action-object, the percentage of the utterances associated with the action-object that include at least one of the salient terms associated with the action-object is determined. If the percentage does not exceed a selected threshold, the method also includes adjusting the one or more salient terms associated with the action-object.
1. A computer implemented method for adjusting a call center application operable to associate user utterances with action-objects based on existing sets of salient terms, the method for adjusting comprising:
automatically adding, by the computer, a new salient term to an existing set of salient terms associated with an action-object when a percentage of user utterances failing to associate with the action-object, based upon the existing set of salient terms, exceeds a specified threshold; and
automatically removing, by the computer, a current association of a salient term with the action-object from the existing set of salient terms when a percentage of user utterances associating with the action-object, based upon the existing set of salient terms, exceeds a specified threshold,
wherein a size of the existing set of salient terms is maintained near an optimum value with regard to a match rate and a performance rate of the call center application.
2. The method of claim 1 , wherein adding the new salient term includes selecting the new salient term based on a count indicative of terms used in a selected set of user utterances.
3. The method of claim 2 , wherein adding the new salient term further includes removing an existing salient term.
4. The method of claim 3 , wherein removing the current association of the salient term is based on the count.
5. The method of claim 1 , wherein removing the current association of the salient term includes selecting the salient term based on a count indicative of terms used in a selected set of user utterances.
6. The method of claim 5 , wherein removing the current association of the salient term further includes adding a new salient term.
7. The method of claim 6 , wherein adding the new salient term is based on the count.
8. A non-transitory computer readable memory medium, including computer executable instructions for adjusting a call center application operable to associate a user utterance with an action-object based on a salient term in the user utterance, the instructions including instructions executable to:
determine that at least one of a match rate for associating user utterances with the action-object and a performance rate for the associating user utterances with the action-object is below a corresponding threshold; and
responsive to executing the instructions to determine, modify salient terms associated with the action-object;
wherein the instructions executable to modify comprise instructions executable to remove an association of the salient term with the action-object based on a frequency with which the salient term occurs in a selected set of user utterances, and
wherein a size of a set of salient terms matching the action-object is maintained near an optimum value with regard to the match rate and the performance rate.
9. The memory medium of claim 8 , wherein the instructions executable to modify include instructions executable to add a new salient term associated with the action-object.
10. The memory medium of claim 9 , wherein the instructions executable to add comprise instructions executable to select the new salient term based on a word count indicative of a frequency with which terms occur in a selected set of user utterances.
11. The memory medium of claim 8 , wherein the instructions executable to remove comprise instructions executable to remove the association of the salient term based on a word count indicative of the frequency.
12. The memory medium of claim 11 , wherein a word count corresponding to the new salient term is greater than a word count corresponding to the salient term.
13. A service center system comprising a computing apparatus, including a processor having access to a computer readable storage medium storing processor executable instructions, the computing apparatus being enabled to execute the instructions to:
associate a user utterance with an action-object based on a salient term in the user utterance; and
adjust at least one of a set of salient action terms and salient object terms associated with the action-object when a specified percentage of user utterances including the salient term associate with the action-object, and wherein the instructions to adjust include instructions to remove an association of the salient term with the action-object,
wherein a size of a set of salient terms associated with the action-object is maintained near an optimum value with regard to a match rate and a performance rate of the service center system.
14. The service center system of claim 13 , wherein the instructions to adjust include instructions to add a new salient term associated with the action-object, wherein the new salient term includes at least one of a new salient action term and a new salient object term.
15. The service center system of claim 14 , wherein the instructions to add include instructions to select the new salient term based on a word count indicative of a frequency with which terms occur in a selected set of user utterances.
16. The service center system of claim 15 , wherein the instructions to remove include instructions to remove the association of the salient term based on the word count.
17. The service center system of claim 16 , wherein a word count corresponding to the new salient term is greater than a word count corresponding to the salient term.