Generating concepts from text reports

1. A method performed by one or more computers, the method comprising:

receiving a collection of text reports, wherein a first portion of the text reports are associated with respective concepts;

performing a clustering process for a plurality of different cluster sizes, including generating, for each cluster size of the plurality of different cluster sizes, a plurality of different clusters of text reports from a second portion of the text reports, the second portion including at least one text report that does not have an associated concept;

evaluating each of the plurality of different cluster sizes to select an optimal cluster size;

generating, from the collection of text reports, clusters using the selected optimal cluster size;

aggregating text associated with text reports in the first portion of text reports to generate respective first aggregated text for each cluster that is associated with a corresponding concept;

aggregating text associated with text reports belonging to each respective cluster to generate respective second aggregated text for each cluster of the clusters generated using the selected optimal cluster size;

maintaining a training dataset comprising the first and second aggregated text; and

generating a predictive model from the training dataset to generate a concept for an input text report.

2. The method of claim 1 , further comprising:

performing sentiment analysis on one or more text reports in the second portion of text reports,

wherein generating the clusters using the selected optimal cluster size comprises assigning text reports having similar sentiment to a same respective cluster.

3. The method of claim 1 , further comprising:

augmenting the training set with a corpus of publically available text.

4. The method of claim 1 , wherein the second portion of the text reports comprises text reports that are not associated with a concept.

5. The method of claim 1 , wherein the collection of the text reports are all associated with a same issue in an organization.

6. The method of claim 1 , further comprising:

receiving a test collection of text reports for which concepts are to be generated;

performing the clustering process for a second plurality of different cluster sizes and evaluating each of the second plurality of different cluster sizes to select a second optimal cluster size;

aggregating text associated with the test collection of text reports to generate test aggregated text for each cluster of the clusters generated using the second optimal cluster size; and

providing the test aggregated text to the predictive model to generate respective concepts for the test collection of text reports.

7. The method of claim 6 , further comprising:

augmenting the first portion of the text reports with text reports and generated concepts of the test collection of text reports; and

regenerating the model using augmented first portion of the text reports having the generated concepts for the test collection of test reports.

8. The method of claim 7 , further comprising repeatedly reaugmenting the first portion of the text reports and regenerating the model as additional test collections of text reports are received.

9. The method of claim 1 , wherein maintaining the training dataset comprises, for each aggregated text:

selecting, from the aggregated text, selected text that is most representative of the aggregated text.

10. The method of claim 9 , further comprising, for each aggregated text:

generating an undirected graph having a plurality of nodes and a plurality of edges that connect a respective pair of nodes, wherein each node corresponds to a text;

identifying, from the undirected graph, one or more subgraphs;

determining, for each identified subgraph, respective importance scores for nodes that are included in the subgraph; and

selecting, from each identified subgraph and based at least on the respective importance scores, one or more selected nodes.

11. The method of claim 10 , wherein determining respective importance scores for nodes that are included in the subgraph comprises:

determining a total number of nodes that are included in each identified subgraph.

12. The method of claim 10 , wherein determining respective importance scores for nodes that are included in the subgraph comprises:

determining respective PageRank scores for nodes that are included in each identified subgraph.

13. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving a collection of text reports, wherein a first portion of the text reports are associated with respective concepts;

performing a clustering process for a plurality of different cluster sizes, including generating, for each cluster size of the plurality of different cluster sizes, a plurality of different clusters of text reports from a second portion of the text reports, the second portion including at least one text report that does not have an associated concept;

evaluating each of the plurality of different cluster sizes to select an optimal cluster size;

generating, from the collection of text reports, clusters using the selected optimal cluster size;

aggregating text associated with text reports in the first portion of text reports to generate respective first aggregated text for each cluster that is associated with a corresponding concept;

aggregating text associated with text reports belonging to each respective cluster to generate respective second aggregated text for each cluster of the clusters generated using the selected optimal cluster size;

maintaining a training dataset comprising the first and second aggregated text; and

generating a predictive model from the training dataset to generate a concept for an input text report.

14. The system of claim 13 , wherein the operations further comprise:

performing sentiment analysis on one or more text reports in the second portion of text reports,

wherein generating the clusters using the selected optimal cluster size comprises assigning text reports having similar sentiment to a same respective cluster.

15. The system of claim 13 , wherein the operations further comprise:

receiving a test collection of text reports for which concepts are to be generated;

performing the clustering process for a second plurality of different cluster sizes and evaluating each of the second plurality of different cluster sizes to select a second optimal cluster size;

aggregating text associated with the test collection of text reports to generate test aggregated text for each cluster of the clusters generated using the second optimal cluster size; and

providing the test aggregated text to the predictive model to generate respective concepts for the test collection of text reports.

16. The system of claim 13 , wherein maintaining the training dataset comprises, for each aggregated text:

selecting, from the aggregated text, selected text that is most representative of the aggregated text.

17. The system of claim 16 , wherein the operations further comprise, for each aggregated text:

generating an undirected graph having a plurality of nodes and a plurality of edges that connect a respective pair of nodes, wherein each node corresponds to a text;

identifying, from the undirected graph, one or more subgraphs;

determining, for each identified subgraph, respective importance scores for nodes that are included in the subgraph; and

selecting, from each identified subgraph and based at least on the respective importance scores, one or more selected nodes.

18. A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

receiving a collection of text reports, wherein a first portion of the text reports are associated with respective concepts;

performing a clustering process for a plurality of different cluster sizes, including generating, for each cluster size of the plurality of different cluster sizes, a plurality of different clusters of text reports from a second portion of the text reports, the second portion including at least one text report that does not have an associated concept;

evaluating each of the plurality of different cluster sizes to select an optimal cluster size;

generating, from the collection of text reports, clusters using the selected optimal cluster size;

aggregating text associated with text reports in the first portion of text reports to generate respective first aggregated text for each cluster that is associated with a corresponding concept;

aggregating text associated with text reports belonging to each respective cluster to generate respective second aggregated text for each cluster of the clusters generated using the selected optimal cluster size;

maintaining a training dataset comprising the first and second aggregated text; and

generating a predictive model from the training dataset to generate a concept for an input text report.

19. The non-transitory computer storage medium of claim 18 , wherein the operations further comprise:

performing sentiment analysis on one or more text reports in the second portion of text reports,

wherein generating the clusters using the selected optimal cluster size comprises assigning text reports having similar sentiment to a same respective cluster.

20. The non-transitory computer storage medium of claim 18 , wherein the operations further comprise:

receiving a test collection of text reports for which concepts are to be generated;

performing the clustering process for a second plurality of different cluster sizes and evaluating each of the second plurality of different cluster sizes to select a second optimal cluster size;

aggregating text associated with the test collection of text reports to generate test aggregated text for each cluster of the clusters generated using the second optimal cluster size; and

providing the test aggregated text to the predictive model to generate respective concepts for the test collection of text reports.