Confidence classifier within context of intent classification

1. A method of applying a confidence classifier for intent classification in association with an automated chat bot, the method comprising:

processing, by a computing system, an utterance with an intent classifier to determine a probability distribution of possible intents associated with the utterance;

generating, by the computing system, a plurality of measures of peakedness of the probability distribution; and

applying, by the computing system, a trained confidence classifier to determine a single normalized probability of a most likely intent associated with the utterance based on the plurality of measures of peakedness of the probability distributions;

wherein generating the plurality of measures of peakedness of the probability distribution comprises:

sorting probability scores of the probability distribution in descending order;

selecting a subset of the greatest probabilities of the sorted probability scores as a probability set;

determining a plurality of ratios of consecutively ranked probabilities in the probability set;

determining a kurtosis score of the probabilities in the probability set;

determining an entropy of the probability distribution; and

normalizing the entropy of the probability distribution by dividing the entropy by a maximum possible entropy of the probability distribution to generate a normalized entropy.

2. The method of claim 1 , further comprising comparing, by the computing system, the normalized probability of the most likely intent associated with the utterance to a confidence threshold.

3. The method of claim 2 , further comprising:

selecting, by the computing system, the most likely intent as an intent associated with the utterance in response to determining that the normalized probability of the most likely intent associated with the utterance satisfies the confidence threshold; and

transmitting, by the computing system, a message to a user device in communication with the automated chat bot in response to selecting the most likely intent as the intent associated with the utterance, wherein the message is a response to the utterance.

4. The method of claim 1 , wherein generating the plurality of measures of peakedness of the probability distribution comprises applying a sigmoid function to each of the plurality of ratios, the kurtosis score, and the normalized entropy.

5. The method of claim 1 , wherein selecting the subset of the greatest probabilities of the sorted probability scores as the probability set comprises selecting five greatest probabilities of the sorted probabilities scores as the probability set.

6. The method of claim 1 , further comprising training, by the computing system, the confidence classifier by tuning parameters using stochastic gradient descent optimization.

7. A system for applying a confidence classifier for intent classification in association with an automated chat bot, the system comprising:

at least one processor; and

at least one memory comprising a plurality of instructions stored thereon that, in response to execution by the at least one processor, causes the system to:

process an utterance with an intent classifier to determine a probability distribution of possible intents associated with the utterance;

generate a plurality of measures of peakedness of the probability distribution; and

apply a trained confidence classifier to determine a single normalized probability of a most likely intent associated with the utterance based on the plurality of measures of peakedness of the probability distribution;

wherein to generate the plurality of measures of peakedness of the probability distribution comprises to:

rank probabilities of the probability distribution by greatest probability; and

determine a plurality of ratios of consecutively ranked probabilities in response to ranking the probabilities of the probability distribution.

8. The system of claim 7 , wherein the plurality of instructions further causes the system to compare the normalized probability of the most likely intent associated with the utterance to a confidence threshold.

9. The system of claim 8 , wherein the plurality of instructions further causes the system to:

select the most likely intent as an intent associated with the utterance in response to a determination that the normalized probability of the most likely intent associated with the utterance satisfies the confidence threshold; and

transmit a message to a user device in communication with the automated chat bot in response to selection of the most likely intent as the intent associated with the utterance, wherein the message is a response to the utterance.

10. The system of claim 7 , wherein the plurality of instructions further causes the system to train the confidence classifier by tuning parameters using stochastic gradient descent optimization.

11. A system for applying a confidence classifier for intent classification in association with an automated chat bot, the system comprising:

at least one processor; and

at least one memory comprising a plurality of instructions stored thereon that, in response to execution by the at least one processor, causes the system to:

process an utterance with an intent classifier to determine a probability distribution of possible intents associated with the utterance;

generate a plurality of measures of peakedness of the probability distribution; and

apply a trained confidence classifier to determine a single normalized probability of a most likely intent associated with the utterance based on the plurality of measures of peakedness of the probability distribution;

wherein to generate the plurality of measures of peakedness of the probability distribution comprises to:

sort probability scores of the probability distribution in descending order;

select a subset of the greatest probabilities of the sorted probability scores as a probability set;

determine a plurality of ratios of consecutively ranked probabilities in the probability set;

determine a kurtosis score of the probabilities in the probability set;

determine an entropy of the probability distribution; and

normalize the entropy of the probability distribution by dividing the entropy by a maximum possible entropy of the probability distribution to generate a normalized entropy.

12. The system of claim 11 , wherein generating the plurality of measures of peakedness of the probability distribution comprises applying a sigmoid function to each of the plurality of ratios, the kurtosis score, and the normalized entropy.

13. The system of claim 11 , wherein selecting the subset of the greatest probabilities of the sorted probability scores as the probability set comprises selecting five greatest probabilities of the sorted probabilities scores as the probability set.

14. The system of claim 11 , wherein the plurality of instructions further causes the system to compare the normalized probability of the most likely intent associated with the utterance to a confidence threshold.

15. The system of claim 14 , wherein the plurality of instructions further causes the system to:

select the most likely intent as an intent associated with the utterance in response to a determination that the normalized probability of the most likely intent associated with the utterance satisfies the confidence threshold; and

transmit a message to a user device in communication with the automated chat bot in response to selection of the most likely intent as the intent associated with the utterance, wherein the message is a response to the utterance.

16. The system of claim 11 , wherein the plurality of instructions further causes the system to train the confidence classifier by tuning parameters using stochastic gradient descent optimization.