Systems and methods for few-shot intent classifier models
Some embodiments of the current disclosure disclose methods and systems for training for training a natural language processing intent classification model to perform few-shot classification tasks. In some embodiments, a pair of an utterance and a first semantic label labeling the utterance may be generated and a neural network that is configured to perform natural language inference tasks may be utilized to determine the existence of an entailment relationship between the utterance and the semantic label. The semantic label may be predicted as the intent class of the utterance based on the entailment relationship and the pair may be used to train the natural language processing intent classification model to perform few-shot classification tasks.
1. A method for few-shot intent classification of an input natural language utterance, the method comprising:
receiving, via a communication interface, a training dataset containing a plurality of utterances and a plurality of pre-defined intent labels;
transforming the training dataset into a plurality of utterance-label pairs by pairing an utterance of the plurality of utterances with each of the plurality pre-defined intent labels, wherein each of the plurality of utterance-label pairs includes the utterance and a pre-defined intent label of the plurality of pre-defined intent labels;
generating a transformed entailment label for each of the plurality of utterance-label pairs, wherein the transformed entailment label indicates an entailment relationship between the utterance and the pre-defined intent label in an utterance-label pair;
inputting the plurality of utterance-label pairs to a classifier;
generating, using the classifier, an entailment probability distribution for each of the plurality of utterance-label pairs;
comparing the entailment probability distribution with the corresponding transformed entailment label for each of the plurality utterance-label pair;
computing a training objective based on the comparison of the entailment probability distribution with the corresponding transformed entailment label for each of the plurality of utterance-label pair; and
updating the classifier based on the training objective via backpropagation.
2. The method of claim 1 , wherein the training dataset contains less than a pre-defined number of utterances corresponding to the pre-defined intent label.
3. The method of claim 1 , wherein the entailment probability distribution indicates the probability a respective utterance and a respective pre-defined intent label in a respective input utterance-label pair match with each other.
4. The method of claim 1 , wherein the input utterance-label pair is formed by concatenating the utterance and the pre-defined intent label separated by a special token.
5. The method of claim 1 , wherein the transformed entailment label is selected from a group of a first label indicating the pre-defined intent label entails the first utterance, a second label indicating that the pre-defined intent label contradicts the utterance, and a third label indicating that the pre-defined intent label is irrelevant to the utterance.
6. The method of claim 1 , wherein the transformed entailment label is a binary label indicating whether the pre-defined intent label entails the utterance.
7. The method of claim 1 , wherein the transformed entailment label is generated in a form of an entailment probability distribution indicating a likelihood that the input utterance-label pair corresponds to the transformed entailment label.
8. The method of claim 7 , wherein the training objective is computed as a binary cross-entropy between the entailment probability distribution and a binary ground-truth label derived from the pre-defined intent label from the training dataset.
9. The method of claim 1 , further comprising:
receiving, via the communication interface, another utterance;
generating a plurality of utterance-label pairs by combining the other utterance with each of the pre-defined intent labels, respectively;
generating, for each utterance-label pair, by the updated classifier, a respective entailment probability distribution score; and
selecting an utterance-label pair having a highest entailment probability distribution score among the plurality of utterance-label pairs.
10. The method of claim 9 , further comprising:
determining whether the highest entailment probability distribution score is greater than a pre-defined threshold;
outputting another intent from the selected utterance-label pair in response to the other utterance when the highest entailment probability distribution score is greater than the pre-defined threshold; and
outputting an out-of-scope label in response to the other utterance when the highest entailment probability distribution score is no greater than the pre-defined threshold.
11. A system for few-shot intent classification of an input natural language utterance, the system comprising:
a communication interface receiving a training dataset containing a plurality of utterances and a plurality of pre-defined intent labels;
a memory storing a plurality of processor-executable instructions; and
a processor reading from the memory and executing the instructions to perform operations comprising:
transforming the training dataset into a plurality of utterance-label pairs by pairing an utterance of the plurality of utterances with each of the plurality pre-defined intent labels, wherein each of the plurality of utterance-label pairs includes the utterance and a pre-defined intent label of the plurality of pre-defined intent labels;
generating a transformed entailment label for each of the plurality of utterance-label pairs, wherein the transformed entailment label indicates an entailment relationship between the utterance and the pre-defined intent label in an utterance-label pair;
inputting the plurality of utterance-label pairs to a classifier;
generating, using the classifier, an entailment probability distribution for each of the plurality of utterance-label pairs;
comparing the entailment probability distribution with the corresponding transformed entailment label for each of the plurality utterance-label pair;
computing a training objective based on the comparison of the entailment probability distribution with the corresponding transformed entailment label for each of the plurality of utterance-label pair; and
updating the classifier based on the training objective via backpropagation.
12. The system of claim 11 , wherein the training dataset contains less than a pre-defined number of utterances corresponding to the pre-defined intent label.
13. The system of claim 11 , wherein the entailment probability distribution indicates the probability a respective utterance and a respective pre-defined intent label in a respective input utterance-label pair match with each other.
14. The system of claim 11 , wherein the input utterance-label pair is formed by concatenating the utterance and the pre-defined intent label separated by a special token.
15. The system of claim 11 , wherein the transformed entailment label is selected from a group of a first label indicating the pre-defined intent label entails the utterance, a second label indicating that the pre-defined intent label contradicts the utterance, and a third label indicating that the pre-defined intent label is irrelevant to the utterance.
16. The system of claim 11 , wherein the transformed entailment label is a binary label indicating whether the pre-defined intent label entails the utterance.
17. The system of claim 11 , wherein the transformed entailment label is generated in a form of an entailment probability distribution indicating a likelihood that the input utterance-label pair corresponds to the transformed entailment label.
18. The system of claim 17 , wherein the training objective is computed as a binary cross-entropy between the entailment probability distribution and a binary ground-truth label derived from the pre-defined intent label from the training dataset.
19. The system of claim 11 , further comprising:
receiving, via the communication interface, another utterance;
generating a plurality of utterance-label pairs by combining the other utterance with each of the pre-defined intent labels, respectively;
generating, for each utterance-label pair, by the updated classifier, a respective entailment probability distribution score; and
selecting an utterance-label pair having a highest entailment probability distribution score among the plurality of utterance-label pairs;
determining whether the highest entailment probability distribution score is greater than a pre-defined threshold;
outputting another intent from the selected utterance-label pair in response to the other utterance when the highest entailment probability distribution score is greater than the pre-defined threshold; and
outputting an out-of-scope label in response to the other utterance when the highest entailment probability distribution score is no greater than the pre-defined threshold.
20. A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for few-shot intent classification of an input natural language utterance, the instructions being executed by a processor to perform operations comprising:
receiving, via a communication interface, a training dataset containing a plurality of utterances and a plurality of pre-defined intent labels;
transforming the training dataset into a plurality of utterance-label pairs by pairing an utterance of the plurality of utterances with each of the plurality pre-defined intent labels, wherein each of the plurality of utterance-label pairs includes the utterance and a pre-defined intent label of the plurality of pre-defined intent labels;
generating a transformed entailment label for each of the plurality of utterance-label pairs, wherein the transformed entailment label indicates an entailment relationship between the utterance and the pre-defined intent label in an utterance-label pair;
inputting the plurality of utterance-label pairs to a classifier;
generating, using the classifier, an entailment probability distribution for each of the plurality of utterance-label pairs;
comparing the entailment probability distribution with the corresponding transformed entailment label for each of the plurality utterance-label pair;
computing a training objective based on the comparison of the entailment probability distribution with the corresponding generated transformed entailment label for each of utterance-label pair; and
updating the classifier based on the training objective via backpropagation.