Applied artificial intelligence technology for adaptively classifying sentences based on the concepts they express to improve natural language understanding
Disclosed herein is computer technology that provides adaptive mechanisms for learning concepts that are expressed by natural language sentences, and then applies this learning to appropriately classify new natural language sentences with the relevant concept that they express.
1. A method for applying artificial intelligence technology to natural language understanding, the method comprising:
processing a plurality of sentences via a processor using a classifier that is configured to distinguish between a plurality of concepts expressed by the plurality of sentences based on sentence composition and sentence structure, wherein the plurality of concepts comprise higher level meanings that are expressed by the sentences beyond literal meanings of words in the sentences, a first subset of the plurality of sentences being associated with respective concepts of the plurality of concepts based on the classifier;
identifying via the processor a second subset of the plurality of sentences that are unclassified by the classifier;
clustering the unclassified sentences into sentence clusters based on defined criteria and one or more characteristics shared among dependency parse trees corresponding to the unclassified sentences;
providing via the processor a user interface (UI) facilitating review of the unclassified sentences according to the sentence clusters;
receiving user input associating an unclassified sentence of the second subset of the plurality of sentences with a concept of the plurality of concepts; and
training a natural language generation (NLG) system based on the unclassified sentence to: interactively receive natural language user input,
interactively receive stylistic selection user input indicative of one or more of the plurality of sentences and/or plurality of concepts, and
interactively determine natural language output responsive to the natural language input and expressing the associated concept in a manner that stylistically resembles the one or more sentences and/or concepts with respect to sentence composition and sentence structure.
2. The method of claim 1 further comprising:
the processor associating an unclassified sentence with a concept in response to user input through the UI.
3. The method of claim 2 further comprising:
the processor training the classifier with the unclassified sentence and its associated concept.
4. The method of claim 1 further comprising:
the processor clustering the unclassified sentences into sentence clusters based on defined criteria; and
wherein the UI organizes the unclassified sentences for review by the user according to the sentence clusters.
5. The method of claim 4 wherein the UI provides a user-selectable list of indicators, each indicator corresponding to a different one of the sentence clusters, wherein user selection of an indicator from the list causes the UI to present an unclassified sentence belonging to the sentence cluster corresponding to the selected indicator to the user for the user to assign a concept classification thereto.
6. The method of claim 4 further comprising:
the processor associating an unclassified sentence within a sentence cluster with a concept in response to user input through the UI.
7. The method of claim 1 further comprising:
the processor conditioning the clustering for an identified root verb on a defined minimum number of the unclassified sentences sharing that identified root verb.
8. The method of claim 1 wherein the processor comprises a plurality of processors that perform one or more different steps of the method.
9. The method of claim 1 wherein the processor comprises one processor that performs the method.
10. The method of claim 1 wherein the sentences include a plurality of words, and wherein the sentence composition and sentence structure used by the classifier to distinguish between the concepts expressed by the sentences includes parts of speech classifications for the words as they are used within the sentences.
11. The method of claim 1 wherein the higher level meanings include relationships between a plurality of entities and/or attributes of entities that are expressed by the sentences.
12. The method of claim 11 wherein the concepts include a change concept.
13. The method of claim 11 wherein the concepts include a compare concept.
14. The method of claim 11 wherein the concepts include a driver concept.
15. The method of claim 11 wherein the concepts include a rank concept.
16. An applied artificial intelligence apparatus for natural language understanding, the apparatus comprising:
a processor configured to (1) process a plurality of sentences using a classifier that is configured to distinguish between a plurality of concepts expressed by the sentences based on sentence composition and sentence structure, wherein the concepts comprise higher level meanings that are expressed by the sentences beyond literal meanings of words in the sentences, (2) identify a plurality of sentences that are unclassified by the classifier as a result of the processing step, (3) cluster the unclassified sentences into sentence clusters based on defined criteria and one or more characteristics shared among dependency parse trees corresponding to the unclassified sentences, (4) provide a user interface facilitating review of the unclassified sentences according to the sentence clusters, (5) receive user input associating an unclassified sentence of the second subset of the plurality of sentences with a concept of the plurality of concepts, and (6) train a natural language generation (NLG) system based on the classified and unclassified sentences to interactively receive natural language user input, interactively receive stylistic selection user input indicative of one or more of the plurality of sentences and/or plurality of concepts, and interactively determine natural language output responsive to the natural language input and expressing the associated concept in a manner that stylistically resembles the one or more sentences and/or concepts with respect to sentence composition and sentence structure.
17. The apparatus of claim 16 wherein the processor is further configured to associate an unclassified sentence with a concept in response to user input through the UI.
18. The apparatus of claim 17 wherein the processor is further configured to train the classifier with the unclassified sentence and its associated concept.
19. The apparatus of claim 16 wherein the processor is further configured to cluster the unclassified sentences into sentence clusters based on defined criteria, and wherein the UI is configured to organize the unclassified sentences for review by the user according to the sentence clusters.
20. The apparatus of claim 19 wherein the UI provides a user-selectable list of indicators, each indicator corresponding to a different one of the sentence clusters, wherein user selection of an indicator from the list causes the UI to present an unclassified sentence belonging to the sentence cluster corresponding to the selected indicator to the user for the user to assign a concept classification thereto.
21. The apparatus of claim 19 wherein the processor is further configured to associate an unclassified sentence within a sentence cluster with a concept in response to user input through the UI.
22. The apparatus of claim 19 wherein the processor is further configured to condition the clustering for an identified root verb on a defined minimum number of the unclassified sentences sharing that identified root verb.
23. The apparatus of claim 16 wherein the processor comprises a plurality of processors.
24. The apparatus of claim 16 wherein the sentences include a plurality of words, and wherein the sentence composition and sentence structure used by the classifier to distinguish between the concepts expressed by the sentences includes parts of speech classifications for the words as they are used within the sentences.
25. The apparatus of claim 16 wherein the higher level meanings include relationships between a plurality of entities and/or attributes of entities that are expressed by the sentences.
26. The apparatus of claim 25 wherein the concepts include a change concept.
27. The apparatus of claim 25 wherein the concepts include a compare concept.
28. The apparatus of claim 25 wherein the concepts include a driver concept.
29. The apparatus of claim 25 wherein the concepts include a rank concept.
30. A computer program product for applying artificial intelligence technology to natural language understanding, the computer program product comprising:
a plurality of processor-executable instructions that are resident on a non-transitory computer-readable storage medium, wherein the instructions upon execution are configured to cause a processor to:
process a plurality of sentences using a classifier that is configured to distinguish between a plurality of concepts expressed by the sentences based on sentence composition and sentence structure, wherein the concepts comprise higher level meanings that are expressed by the sentences beyond literal meanings of words in the sentences, a first subset of the plurality of sentences being associated with respective concepts of the plurality of concepts based on the classifier;
cluster the unclassified sentences into sentence clusters based on defined criteria and one or more characteristics shared among dependency parse trees corresponding to the unclassified sentences;
identify a second subset of the plurality of sentences that are unclassified by the classifier as a result of the processing step; and
provide a user interface facilitating review of the unclassified sentences according to the sentence clusters;
receive user input associating an unclassified sentence of the second subset of the plurality of sentences with a concept of the plurality of concepts; and
train a natural language generation (NLG) system based on the classified and unclassified sentences to interactively receive natural language user input, interactively receive stylistic selection user input indicative of one or more of the plurality of sentences and/or plurality of concepts, and interactively determine natural language output responsive to the natural language input and expressing the associated concept in a manner that stylistically resembles the one or more sentences and/or concepts with respect to sentence composition and sentence structure.
31. The computer program product of claim 30 further comprising: the processor providing a user interface (UI) through which a user reviews the at least one unclassified sentence.
32. The computer program product of claim 31 further comprising:
the processor associating the unclassified sentence with a concept in response to user input through the UI.
33. The computer program product of claim 32 further comprising:
the processor retraining the trained classifier with the unclassified sentence and its associated concept.
34. The computer program product of claim 30 wherein the at least one unclassified sentence comprises a plurality of unclassified sentences, the method further comprising: the processor clustering the unclassified sentences into sentence clusters based on defined criteria; and the processor providing a user interface (UI) that organizes the unclassified sentences for review by a user according to the sentence clusters.
35. The computer program product of claim 34 further comprising:
the processor associating an unclassified sentence within a sentence cluster with a concept in response to user input through the UI.
36. The computer program product of claim 30 wherein the processor comprises a plurality of processors that perform one or more different steps of the method.
37. The computer program product of claim 30 wherein the processor comprises one processor that performs the method.
38. The computer program product of claim 30 wherein the sentences include a plurality of words, and wherein the sentence composition and sentence structure used by the classifier to distinguish between the concepts expressed by the sentences includes parts of speech classifications for the words as they are used within the sentences.
39. The computer program product of claim 30 wherein the sentences include a plurality of words, and wherein the sentence composition and sentence structure used by the classifier to distinguish between the concepts expressed by the sentences includes parts of speech classifications for the words as they are used within the sentences.