System and method for generating a taxonomy from a plurality of documents
View Patent ↗A system and method for generating a taxonomy is provided in which the taxonomy is generated based on clusters of phrases and a topical library. The taxonomy permits a user of a text processing system to rapidly search through a database and find relevant documents since the classifications in the taxonomy are narrow enough to limit the number of documents classified in each of the classifications.
1. A computer-implemented method comprising:
inputting text;
extracting phrases from the text;
constructing clusters of the phrases defining connections between the phrases;
selecting leader phrases from the clusters that each have a pre-determined number of connections to other ones of the phrases; and
defining a taxonomy that classifies categories of information within the text, based on the leader phrases.
2. The method of claim 1 wherein defining the taxonomy comprises defining a multi-level taxonomy having a first level of categories and a second level of categories.
3. The method of claim 2 wherein defining the multi-level taxonomy comprises:
assigning the leader phrases to the first level; and
assigning the other ones of the phrases that are connected to the leader phrases to the second level, in association with their respective leader phrases.
4. The method of claim 3 wherein defining the multi-level taxonomy comprises defining a third level of categories, and wherein defining the third level of categories comprises:
determining sub-heading phrases that are not leader phrases and that are connected to the other ones of the phrases; and
assigning the sub-heading phrases to the third level.
5. The method of claim 2 wherein defining the multi-level taxonomy comprises:
assigning topics from a pre-determined topical library to the first level;
associating each of the leader phrases with at least one of the topics; and
assigning the leader phrases to the second level in association with their respective topics.
6. The method of claim 1 wherein constructing clusters of the phrases comprises constructing links between the phrases that are defined by the connections, such that a user of the taxonomy may follow the connections by selecting a desired one of the links.
7. The method of claim 1 comprising outputting the taxonomy as a tool for analyzing the text.
8. A system comprising:
a computer-based text processing system to generate a multi-level taxonomy that classifies content using linked phrases that have been extracted from the content, wherein a first level of the multi-level taxonomy includes a leader phrase selected from the phrases based on a number of links associated with the leader phrase; and
a user interface operable to output the multi-level taxonomy, such that a user proceeds from the first level to a second level of the multi-level taxonomy, wherein the second level includes connected phrases that are linked to the leader phrase.
9. The system of claim 8 comprising a topical library that includes topics associated with the content, wherein the text processing system is operable to access the topical library and include the topics within the multi-level taxonomy.
10. The system of claim 8 wherein the text processing system is operable to include a third level within the taxonomy, wherein the third level includes sub-phrases that are linked to the connected phrases.
11. The system of claim 8 wherein the text processing system comprises:
a parser operable to identify the phrases within the content; and
a clusterizer that is operable to determine the links between the phrases.
12. The system of claim 11 wherein the text processing system comprises a map generator that is operable to display the linked phrases.
13. The system of claim 8 wherein the text processing system is operable to receive the content from the user and output the multi-level taxonomy in response.
14. An apparatus comprising a computer-readable storage medium having instructions stored thereon, wherein a processor performs operations upon reading the instructions, the instructions including:
a first code segment for inputting documents, the documents including text;
a second code segment for extracting phrases from the text;
a third code segment for establishing connections between the phrases;
a fourth code segment for defining a leader phrase from among the phrases, based on a number of the connections associated with the leader phrase; and
a fifth code segment for defining a taxonomy in which the leader phrase is included at a first level.
15. The apparatus of claim 14 in which the fifth code segment includes secondary phrases directly connected to the leader phrase by the third code segment at a second level of the taxonomy.
16. The apparatus of claim 14 in which the fifth code segment includes tertiary phrases directly connected to the secondary phrases by the third code segment at a third level of the taxonomy.
17. The apparatus of claim 14 wherein the fifth code segment includes a sixth code segment for accessing topics from within a topical library that is associated with content of the documents, wherein the fifth code segment associates the leader phrase with one of the topics within the taxonomy.
18. The apparatus of claim 14 comprising a sixth code segment for outputting the taxonomy as a multi-level taxonomy, including the leader phrase at the first level and phrases connected to the leader phrase at a second level below the first level.
19. The apparatus of claim 18 including a seventh code segment for receiving a user selection of the leader phrase and for providing the second-level phrases in response thereto.
20. The apparatus of claim 19 wherein the seventh code segment includes an eighth code segment for outputting, in response to a further user selection of one of the second-level phrases, a document from among the documents that corresponds to the selected second-level phrase.