System and method for automated customer service with contingent live interaction
View Patent ↗A balance between customer satisfaction and cost to providing customer care can be achieved based on the use of online interaction classification techniques. Such techniques can use measurements such as a log likelihood ratio to determine if an interaction should be removed from automation.
1. A computerized method for determining when to transfer a user from an automated service to a live agent comprising:
a) training a set of classification models using a set of classified historical interactions to perform real-time classification of an interaction wherein said classified historical interactions comprise, at least, prompts provided by the interactive voice response system, transcriptions of statements by a caller derived from an automatic speech recognizer, meanings ascribed to statements made by the caller, and confidence scores for the transcriptions; and
b) during an automated interaction between the user and the automated service, using a computer to calculate a log likelihood ratio, using said classification models, to determine whether to transfer said user, from said automated interaction to a live interaction, by computing a log of a prediction that the interaction is good over a prediction that the interaction is bad;
wherein:
1) said log likelihood ratio is computed using the formula log(P(x|LM good )/P(x|LM bad ));
2) LM good is a first classification model trained using records of one or more previous interactions classified as good;
3) LM bad is a second classification model trained using records of one or more previous interactions classified as bad; and
4) x is a set of responses made by the user during the interaction.
2. A computerized method as claimed in claim 1 wherein the prediction that the interaction is good and the prediction that the interaction is bad are made without respect to the topic of the interaction.
3. A computerized method as claimed in claim 1 wherein:
a) said log likelihood ratio is compared against a threshold value to determine whether said interaction is bad; and
b) said threshold value may be dynamically reset based on external factors.
4. A computerized method as claimed in claim 3 wherein said threshold value may be dynamically reset based on a lifetime value of a relationship with said user.
5. A computerized method as claimed in claim 4 wherein said classification models are based on a boostexter classification.
6. A computerized method as claimed in claim 5 wherein said boostexter classification is derived using Bayes' rule.
7. A computerized method as claimed in claim 4 wherein said classification models are based on an N-gram language model.
8. A computerized method as claimed in claim 4 wherein said log likelihood ratio is re-calculated for each turn in said automated interaction and wherein said re-calculation takes place after each turn in real time during said automated interaction.
9. A non-transitory computer readable medium storing computer executable instructions to configure a computer to determine when to transfer a user from an automated service to a live agent by performing steps comprising:
a) predicting whether an interaction is good, based on a first classification model trained using records of one or more previous interactions classified as good, using P(x|LM good );
b) predicting whether the interaction is bad, based on a second classification model trained using records of one or more previous interactions classified as bad, using P(x|LM bad );
c) calculating a log likelihood ratio using log(P(x|LM good )/P(x|LM bad ));
d) comparing said log likelihood ratio to a threshold value, such that if said log likelihood ratio falls below said threshold value, instructions are executed to transfer said user from automation to said live agent;
wherein:
i) x is a set of responses made by the user during the interaction; and
ii) the one or more previous interactions classified as good and the one or more previous interactions classified as bad comprise, at least, prompts provided by an interactive voice response system, transcriptions of statements by a caller derived from an automatic speech recognizer, meanings ascribed to statements made by the caller, and confidence scores for the transcriptions.
10. The non-transitory computer readable medium as claimed in claim 9 wherein said classification model is based on a boostexter classification.
11. The non-transitory computer readable medium as claimed in claim 10 wherein said boostexter classification is derived using Bayes' rule.
12. The non-transitory computer readable medium as claimed in claim 9 wherein said classification model is based on an N-gram language model.
13. The non-transitory computer readable medium as claimed in claim 9 wherein said threshold value may be dynamically modified based on external factors.
14. The non-transitory computer readable medium as claimed in claim 13 wherein said threshold value may be dynamically reset based on a lifetime value of a relationship with said user.
15. The non-transitory computer readable medium as claimed in claim 14 wherein said instructions recalculate the log likelihood ratio for each turn in said interaction.
16. A computerized system for determining when to transfer a user from an automated service to a live agent comprising:
a) an interactive voice response system (IVR);
b) a monitoring module;
wherein
i) said user interacts with said IVR;
ii) said monitoring module evaluates, after each turn in said IVR, a probability that said user's interaction with the IVR is good and a probability that said user's interaction with the IVR is bad;
iii) said monitoring module signals an alarm to bring in a human agent if a log of the ratio of said good probability over said bad probability is below a predetermined threshold;
iv) said monitoring module evaluates the probability that the user's interaction with the IVR is good using P(x|LM good );
v) the monitoring module evaluates the probability that the user's interaction with the IVR is bad using P(x|LM bad );
vi) x is a set of responses made by the user during the interaction;
vii) LM good is a first classification model trained using records of one or more previous interactions classified as good;
viii) LM bad is a second classification model trained using records of one or more previous interactions classified as bad; and
ix) said one or more previous interactions classified as good and said one or more previous interactions classified as bad comprise, at least, prompts provided by the interactive voice response system, transcriptions of statements by a caller derived from an automatic speech recognizer, meanings ascribed to statements made by the caller, and confidence scores for the transcriptions.
17. A computerized system as claimed in claim 16 wherein:
a) x=r 1 , r 2 , . . . r n ; and
b) each r is a response made by the user in the user's interaction with the IVR.
18. A computerized system as claimed in claim 16 wherein said monitoring module evaluates said probabilities based on a boostexter classifier in an iterative algorithm.
19. A computerized system as claimed in claim 16 wherein said threshold may be dynamically reset based on a lifetime value of a relationship with said user.