Sub-model generation to improve classification accuracy
A method of classifying text input for use with a natural language understanding system can include determining classification information including a primary classification and one or more secondary classifications for a received text input using a statistical classification model (statistical model). A statistical classification sub-model (statistical sub-model) can be selectively built according to a model generation criterion applied to the classification information. The method further can include selecting the primary classification or the secondary classification for the text input as a final classification according to the statistical sub-model and outputting the final classification for the text input.
1. A method of classifying text input for use with a natural language understanding system, the method comprising:
via a processor, determining classification information comprising a primary classification and at least one secondary classification for a received text input using a statistical classification model (statistical model);
via the processor, selectively building a statistical classification sub-model (statistical sub-model) according to whether the classification information conforms to an accuracy requirement;
via the processor, selecting the primary classification or the at least one secondary classification for the text input as a final classification according to the statistical sub-model; and
via the processor, outputting the final classification for the text input.
2. The method of claim 1 , wherein selectively building a statistical sub-model further comprises:
via the processor, comparing a confidence score of the primary classification with a minimum threshold level; and
via the processor, building the statistical sub-model when the confidence score does not exceed the minimum threshold level.
3. The method of claim 1 , wherein selectively building a statistical sub-model further comprises:
via the processor, calculating a difference between a confidence score of the primary classification and a confidence score of the at least one secondary classification;
via the processor, comparing the difference with a difference threshold level; and
via the processor, building the statistical sub-model when the difference does not exceed the difference threshold level.
4. The method of claim 1 , wherein selectively building a statistical sub-model further comprises:
via the processor, determining that the primary classification and the at least one secondary classification match a predetermined set of classifications; and
via the processor, building the statistical sub-model when a match is determined.
5. The method of claim 1 , wherein the statistical model comprises a plurality of classes, wherein selectively building a statistical sub-model further comprises, via the processor, generating the statistical sub-model only for a subset of the plurality of classes of the statistical model.
6. The method of claim 1 , further comprising:
via the processor, selecting features associated with the primary classification and the at least one secondary classification from a plurality of features from training data used to create the statistical model; and
via the processor, building the statistical sub-model using the selected features.
7. The method of claim 1 , wherein selectively building a statistical sub-model further comprises:
via the processor, selecting training data associated with the primary classification and the at least one secondary classification from a corpus of training data used to create the statistical model; and
via the processor, building the statistical sub-model using the selected training data.
8. The method of claim 1 , further comprising storing the statistical sub-model for subsequent recall in processing further text input.
9. The method of claim 1 , further comprising, via the processor:
via the processor, determining a usage frequency of the statistical sub-model;
via the processor, comparing the usage frequency with a minimum frequency threshold level; and
via the processor, merging the primary classification and the at least one secondary classification into a single, merged class when the usage frequency exceeds the minimum frequency threshold level.
10. A method of improving classification accuracy of text input for use with a natural language understanding system, the method comprising:
via a processor, processing a plurality of text inputs using a statistical classification model (statistical model) and a statistical classification sub-model (statistical sub-model), wherein the statistical model comprises a plurality of classes and the statistical sub-model comprises a subset of the plurality of classes;
via the processor, determining a usage frequency of the statistical sub-model;
via the processor, comparing the usage frequency with a minimum frequency threshold level;
via the processor, merging the subset of the plurality of classes into a single, merged class within the statistical model when the usage frequency exceeds the minimum frequency threshold level; and
via the processor, outputting an updated statistical model specifying the merged class.
11. The method of claim 10 , wherein outputting further comprises:
via the processor, selecting training data corresponding to the subset of the plurality of classes from training data used to generate the statistical model;
via the processor, updating the training data by associating the selected training data with the merged class; and
via the processor, generating the updated statistical model from the updated training data.
12. The method of claim 10 , wherein outputting further comprises, via the processor, mapping each class of the subset of the plurality of classes to the merged class within the statistical model, wherein a text input belonging to any of the classes of the subset of the plurality of classes is classified to the merged class.
13. A computer program product comprising:
a computer-readable storage comprising computer-usable program code stored thereon that classifies text input for use with a natural language understanding system, the computer program product comprising:
computer-usable program code that determines classification information comprising a primary classification and at least one secondary classification for a received text input using a statistical classification model (statistical model);
computer-usable program code that selectively builds a statistical classification sub-model (statistical sub-model) according to whether the classification information conforms to an accuracy requirement;
computer-usable program code that selects the primary classification or the at least one secondary classification for the text input as a final classification according to the statistical sub-model; and
computer-usable program code that outputs the final classification for the text input, wherein
the computer-readable storage is not a transitory, propagating signal per se.
14. The computer program product of claim 13 , wherein the computer-usable program code that selectively builds a statistical sub-model further comprises:
computer-usable program code that compares a confidence score of the primary classification with a minimum threshold level; and
computer-usable program code that builds the statistical sub-model when the confidence score does not exceed the minimum threshold level.
15. The computer program product of claim 13 , wherein the computer-usable program code that selectively builds a statistical sub-model further comprises:
computer-usable program code that calculates a difference between a confidence score of the primary classification and a confidence score of the at least one secondary classification;
computer-usable program code that compares the difference with a difference threshold level; and
computer-usable program code that builds the statistical sub-model when the difference does not exceed the difference threshold level.
16. The computer program product of claim 13 , wherein the computer-usable program code that selectively builds a statistical sub-model further comprises:
computer-usable program code that determines that the primary classification and the at least one secondary classification match a predetermined set of classifications; and
computer-usable program code that builds the statistical sub-model when a match is determined.
17. The computer program product of claim 13 , wherein the statistical model comprises a plurality of classes, wherein the computer-usable program code that selectively builds a statistical sub-model further comprises computer-usable program code that generates the statistical sub-model only for a subset of the plurality of classes of the statistical model.
18. The computer program product of claim 13 , wherein the computer-readable storage further comprises:
computer-usable program code that selects features associated with the primary classification and the at least one secondary classification from a plurality of features from training data used to create the statistical model; and
computer-usable program code that builds the statistical sub-model using the selected features.
19. The computer program product of claim 13 , wherein the computer usable program code that selectively builds a statistical sub-model further comprises:
computer-usable program code that selects training data associated with the primary classification and the at least one secondary classification from a corpus of training data used to create the statistical model; and
computer-usable program code that builds the statistical sub-model using the selected training data.
20. The computer program product of claim 13 , wherein the computer-readable storage further comprises computer-usable program code that stores the generated statistical sub-model for subsequent recall in processing further text input.