Systems and methods for programatically classifying text using topic classification

1. A method of programmatically classifying text comprising:

receiving, from a non-transitory computer readable media, a block of text that includes at least one topic;

identifying the at least one topic associated with the block of text, the identifying comprising comparing each portion of a plurality of portions of the block of text to a topic listing of a plurality of topics;

identifying one or more categories for each of the identified topics, the identifying one or more categories comprising comparing each of the identified topics to a hierarchical category graph that includes a mapping of topics to categories;

determining from the identified one or more categories for each of the identified topics, a set of unique categories that is a subset of the identified one or more categories, wherein each unique category is associated with at least one portion of the plurality of portions of the block of text;

determining, by a processor, an actual category frequency for each of the unique categories in the set, the determining based on a number of times each of the topics in the block of text is associated with the unique category;

associating each of the unique categories in the set with M levels of parent categories of each of the unique categories and N levels of child categories for each of the unique categories;

determining category graph data for the block of text including each unique category, the M levels of parent categories for each of the unique categories, and the N levels of child categories for each of the unique categories;

determining a connected category score for each of the unique categories based on the category graph data, the connected category score indicating a number of unique categories connected to the parent categories in the M levels and connected to child categories in the N levels;

ranking the unique categories in the set based on the actual category frequency and the connected category score;

associating at least one of the unique categories of the set with the block of text based on the ranking; and

presenting a machine-generated summary of the block of text that includes at least the highest ranked unique category in the set.

2. The method of claim 1 further comprising:

identifying an expected category frequency for each of the unique categories in the set, the expected category frequency determined based on a frequency of each of the unique categories in the set for a model block of text having approximately a same length and the block of text;

determining a surprise score for each of the unique categories in the set based on a difference between the actual category frequency and the expected category frequency; and

updating the ranking of the unique categories in the set based on the surprise score.

3. The method of claim 2 , wherein determining the expected category frequency for each of the unique categories in the set includes determining the expected frequency for each of the unique categories in the set based on a length of the block of text and a number of topics associated with each of the unique categories in the set as defined by classification data, the classification data indicating associations between topics and categories of a text classification system.

4. The method of claim 2 further comprising:

determining classification data indicating an expected category frequency for each of the unique categories in the set of a text classification system; and

determining the expected frequency for each of the unique categories in the set based on the classification data.

5. The method of claim 1 further comprising:

determining a topic frequency for each of the topics based on a number of times each topic is identified in the block of text; and

associating the at least one unique category of the set with the block of text based on the topic frequency for each of the topics.

6. The method of claim 1 further comprising:

associating each of the unique categories in the set with a corresponding parent category;

determining an actual parent category frequency for the corresponding parent category of each of the unique categories in the set;

determining a category score for each of the unique categories in the set derived from an algorithmic relationship of the actual parent category frequency and the actual category frequency; and

associating each of the unique categories in the set with the block of text based on the category score for each of the unique categories in the set.

7. The method of claim 1 wherein determining the actual category frequency is further based on:

determining the actual category frequency for each of the unique categories of the set, an actual parent category frequency for each parent category of the M levels of parent categories, and an actual child category frequency for each child category of the N levels of child categories for the unique category; and

associating each of the unique categories of the set with the block of text based on the connected category score for each of the unique categories of the set and one or more other connected category scores for each of unique categories of the set.

8. The method of claim 1 , wherein the block of text is a portion of a text comprising a second block of text and further comprising:

receiving, from the non-transitory computer readable media, the second block of text; and

associating a first unique category of the set with the block of text based on a first category frequency of the first unique category in the block of text and a second category frequency for each of the unique categories of the set in the second block of text.

9. The method of claim 1 , wherein identifying the topics includes identifying a first topic by comparing a portion of the block of text with a topic list of topics and determining the portion of the block of text matches at least the first topic.

10. A system configured to programmatically classify text, comprising:

a processor configured to:

receive, from a non-transitory computer readable media, a block of text that includes at least one topic;

identify the at least one topic associated with the block of text, the identification comprising comparing each portion of a plurality of portions of the block of text to a topic listing of a plurality of topics;

identify one or more categories for each of the identified topics, the identified one or more categories comprising comparing each of the identified topics to a hierarchical category graph that includes a mapping of topics to categories;

determine from the identified one or more categories for each of the identified topics, a set of unique categories that is a subset of the identified one or more categories, wherein each unique category is associated with at least one portion of the plurality of portions of the block of text;

determine an actual category frequency for each of the unique categories in the set, the determination based on a number of times each of the topics in the block of text is associated with the unique category;

associate each of the unique categories in the set with M levels of parent categories of each of the unique categories and N levels of child categories for each of the unique categories;

determine category graph data for the block of text including each unique category, the M levels of parent categories for each of the unique categories, and the N levels of child categories for each of the unique categories;

determine a connected category score for each of the unique categories based on the category graph data, the connected category score indicating a number of unique categories connected to the parent categories in the M levels and connected to child categories in the N levels;

rank the unique categories in the set based on the actual category frequency and the connected category score;

associate at least one of the unique categories of the set with the block of text based on the ranking; and

presenting a machine-generated summary of the block of text that includes at least the highest ranked unique category in the set.

11. The system of claim 10 , wherein the processor is further configured to:

identify an expected category frequency for each of the unique categories in the set, the expected category frequency determined based on a frequency of each of the unique categories in the set for a model block of text having approximately a same length and the block of text;

determine a surprise score for each of the unique categories in the set based on a difference between of the actual category frequency and the expected category frequency; and

update the ranking of the unique categories in the set based on the surprise score.

12. The system of claim 11 , wherein the processor configured to determine the expected category frequency for each of the unique categories in the set includes the processor being configured to determine the expected frequency for each of the unique categories in the set based on a length of the block of text and a number of topics associated with each of the unique categories in the set as defined by classification data, the classification data indicating associations between topics and categories of a text classification system.

13. The system of claim 11 , wherein the processor is further configured to:

determine classification data indicating the expected category frequency for each of the unique categories in the set of a text classification system; and

determine the expected frequency for each of the unique categories in the set based on the classification data.

14. The system of claim 10 , wherein the processor is further configured to:

determine a topic frequency for each of the topics based on a number of times each topic is identified in the block of text; and

associate the at least one unique category of the set with the block of text based on the topic frequency for each of the topics.

15. The system of claim 10 , wherein the processor is further configured to:

associate each of the unique categories in the set with a corresponding parent category;

determine an actual parent category frequency for the corresponding parent category of each of the unique categories in the set;

determine a category score for each of the unique categories in the set derived from an algorithmic relationship of the actual parent category frequency and the actual category frequency; and

associate each of the unique categories in the set with the block of text based on the category score for each of the unique categories in the set.

16. The system of claim 10 , wherein the processor is further configured to determine the actual category frequency based on:

determination of the actual category frequency for each of the unique categories of the set, an actual parent category frequency for each parent category of the M levels of parent categories, and an actual child category frequency for each child category of the N levels of child categories for the unique category; and

associate each of the unique categories of the set with the block of text based on the connected category score for each of the unique categories of the set and one or more other connected category scores for each of unique categories of the set.

17. The system of claim 10 , wherein:

the block of text is a portion of a text comprising a first block of text and a second block of text; and

the processor is further configured to:

receive, from the non-transitory computer readable media, the second block of text; and

associate at least one unique category of the set with the block of text based on a first category frequency of the unique categories of the set in the first block of text and a second category frequency for each of the unique categories of the set in the second block of text.

18. The system of claim 10 , wherein the processor configured to identify the topic includes the processor being configured to identify a first topic by comparing a portion of the block of text with a topic list of topics and determining the portion of the block of text matches at least the first topic.

19. A computer program product comprising a non-transitory computer readable storage medium and computer program instructions stored therein, the computer program instructions comprising program instructions for:

receiving, from a non-transitory computer readable media, a block of text that includes at least one topic;

identifying the at least one topic associated with the block of text, the identifying comprising comparing each portion of a plurality of portions of the block of text to a topic listing of a plurality of topics;

identifying one or more categories for each of the identified topics, the identifying one or more categories comprising comparing each of the identified topics to a hierarchical category graph that includes a mapping of topics to categories;

determining from the identified one or more categories for each of the identified topics, a set of unique categories that is a subset of the identified one or more categories, wherein each unique category is associated with at least one portion of the plurality of portions of the block of text;

determining, by a processor, an actual category frequency for each of the unique categories in the set, the determining based on a number of times each of the topics in the block of text is associated with the unique category;

associating each of the unique categories in the set with M levels of parent categories of each of the unique categories and N levels of child categories for each of the unique categories;

determining category graph data for the block of text including each unique category, the M levels of parent categories for each of the unique categories, and the N levels of child categories for each of the unique categories;

determining a connected category score for each of the unique categories based on the category graph data, the connected category score indicating a number of unique categories connected to the parent categories in the M levels and connected to child categories in the N levels;

ranking the unique categories in the set based on the actual category frequency and the connected category score;

associating at least one of the unique categories of the set with the block of text based on the ranking; and

presenting a machine-generated summary of the block of text that includes at least the highest ranked unique category in the set.

20. The computer program product of claim 19 , wherein the computer program instructions comprising program instructions for:

identifying an expected category frequency for each of the unique categories in the set, the expected category frequency determined based on a frequency of each of the unique categories in the set for a model block of text having approximately a same length and the block of text;

determining a surprise score for each of the unique categories in the set based on a difference between the actual category frequency and the expected category frequency; and

updating the ranking of the unique categories in the set based on the surprise score.