Applied artificial intelligence technology for using natural language processing to train a natural language generation system with respect to numeric style features
Disclosed herein is computer technology that applies natural language processing (NLP) techniques to training data to generate information used to train a natural language generation (NLG) system to produce output that stylistically resembles the training data. In this fashion, the NLG system can be readily trained with training data supplied by a user so that the NLG system is adapted to produce output that stylistically resembles such training data. In an example, an NLP system detects a plurality of linguistic features in the training data. These detected linguistic features are then aggregated into a specification data structure that is arranged for training the NLG system to produce natural language output that stylistically resembles the training data. Parameters in the specification data structure can be linked to objects in an ontology used by the NLG system to facilitate the training of the NLG system based on the detected linguistic features.
1. A natural language processing method comprising:
performing natural language processing (NLP) on training data to detect a plurality of linguistic features in the training data, wherein the training data comprises a plurality of words arranged in a natural language, and wherein the detected linguistic features include a numeric style feature;
generating a specification data structure based on the detected linguistic features, the specification data structure arranged for training a natural language generation (NLG) system to produce natural language output that stylistically resembles the training data;
training the NLG system based on the specification data structure to thereby configure the NLG system to produce natural language output that stylistically resembles the training data; and
the trained NLG system processing a data set to generate a natural language output that expresses an idea derived from the processed data set, wherein the generated natural language output includes numeric data that is expressed in accordance with the numeric style feature; and
wherein the performing, generating, training, and processing steps are performed by a processor.
2. The method of claim 1 wherein the specification data structure comprises a machine-readable representation of the detected linguistic features.
3. The method of claim 1 wherein the performing step comprises the processor performing pattern matching on the training data to detect the numeric style feature.
4. The method of claim 3 wherein the pattern matching comprises regular expression pattern matching.
5. The method of claim 1 wherein the numeric style feature comprises a decimal precision feature.
6. The method of claim 1 wherein the numeric style feature comprises a decimal separator feature.
7. The method of claim 1 wherein the numeric style feature comprises a digit grouping delimiter feature.
8. The method of claim 1 wherein the numeric style feature comprises a currency symbol feature.
9. The method of claim 1 further comprising:
modifying the specification data structure to selectively choose in response to user input which of the detected linguistic features are to be used for training the NLG system, wherein the processor performs the modifying step.
10. The method of claim 9 further comprising:
providing a user interface for presentation to a user, the user interface configured to summarize the detected linguistic features; and
receiving user input through the user interface, wherein the received user input includes commands that identify which of the detected linguistic features are to be used to train the NLG system, wherein the processor performs the receiving step.
11. The method of claim 1 further comprising:
receiving the training data as text sentence input from a user.
12. The method of claim 1 further comprising:
receiving the training data as a pre-existing document.
13. The method of claim 1 further comprising:
receiving the training data as speech input from a user.
14. The method of claim 1 wherein the training data comprises a corpus of documents.
15. The method of claim 1 wherein the training data comprises a plurality of sentences, the method further comprising performing the NLP on each of a plurality of the sentences to detect a plurality of linguistic features in the sentences.
16. The method of claim 1 wherein the processor comprises a plurality of processors.
17. The method of claim 16 wherein different processors perform the performing and generating steps.
18. The method of claim 1 wherein the same processor performs the performing and generating steps.
19. An apparatus for natural language processing, the apparatus comprising:
a processor configured to (1) perform natural language processing (NLP) on training data to detect a plurality of linguistic features in the training data, wherein the training data comprises a plurality of words arranged in a natural language, and wherein the detected linguistic features include a numeric style feature, (2) generate a specification data structure based on the detected linguistic features, the specification data structure arranged for training a natural language generation (NLG) system to produce natural language output that stylistically resembles the training data, and (3) train the NLG system based on the specification data structure to thereby configure the NLG system to produce natural language output that stylistically resembles the training data; and
the trained NLG system, wherein the trained NLG system is configured to process a data set to generate a natural language output that expresses an idea derived from the processed data set, wherein the generated natural language output includes numeric data that is expressed in accordance with the numeric style feature.
20. The apparatus of claim 19 wherein the specification data structure comprises a machine-readable representation of the detected linguistic features.
21. The apparatus of claim 19 wherein the processor is further configured to perform pattern matching on the training data to detect the numeric style feature.
22. The apparatus of claim 21 wherein the pattern matching comprises regular expression pattern matching.
23. The apparatus of claim 19 wherein the numeric style feature comprises a decimal precision feature.
24. The apparatus of claim 19 wherein the numeric style feature comprises a decimal separator feature.
25. The apparatus of claim 19 wherein the numeric style feature comprises a digit grouping delimiter feature.
26. The apparatus of claim 19 wherein the numeric style feature comprises a currency symbol feature.
27. The apparatus of claim 19 wherein the processor is further configured to modify the specification data structure to selectively choose in response to user input which of the detected linguistic features are to be used for training the NLG system.
28. The apparatus of claim 27 wherein the processor is further configured to:
provide a user interface for presentation to a user, the user interface configured to summarize the detected linguistic features; and
receive user input through the user interface, wherein the received user input includes commands that identify which of the detected linguistic features are to be used to train the NLG system.
29. The apparatus of claim 19 wherein the processor is further configured to receive the training data as text sentence input from a user.
30. The apparatus of claim 19 wherein the processor is further configured to receive the training data as a pre-existing document.
31. The apparatus of claim 19 wherein the processor is further configured to receive the training data as speech input from a user.
32. The apparatus of claim 19 wherein the training data comprises a corpus of documents.
33. The apparatus of claim 19 wherein the training data comprises a plurality of sentences, and wherein the processor is further configured to perform the NLP on each of a plurality of the sentences to detect a plurality of linguistic features in the sentences.
34. The apparatus of claim 19 wherein the processor comprises a plurality of processors.
35. The apparatus of claim 19 wherein the processor is included as part of the NLG system.
36. The apparatus of claim 19 wherein the processor is part of an NLP system, and wherein the NLG system includes a different processor.
37. A computer program product for natural language processing, the computer program product comprising:
a plurality of processor-executable instructions that are resident on a non-transitory computer readable storage medium, wherein the instructions are configured, upon execution by a processor, to cause the processor to (1) perform natural language processing (NLP) on training data to detect a plurality of linguistic features in the training data, wherein the training data comprises a plurality of words arranged in a natural language, and wherein the detected linguistic features include a numeric style feature, (2) generate a specification data structure based on the detected linguistic features, the specification data structure arranged for training a natural language generation (NLG) system to produce natural language output that stylistically resembles the training data, and (3) train the NLG system based on the specification data structure to thereby configure the NLG system to produce natural language output that stylistically resembles the training data, wherein the trained NLG system is configured to process a data set to generate a natural language output that expresses an idea derived from the processed data set, wherein the generated natural language output includes numeric data that is expressed in accordance with the numeric style feature.