Computer-Implemented System And Method For Identifying Relevant Documents

1 . A computer-implemented system for identifying relevant documents, comprising:

a database to store a set of documents each associated with one or more concepts; and

a server comprising a central processing unit, memory, an input port to receive the document set, and an output port, wherein the central processing unit is configured to:

cluster at least a portion of the documents in the set based on the concepts; and

generate a matrix that provides a summary of those documents that are most relevant to one such concept by determining for each document a measure of similarity between a concept frequency occurrence and concept weights of each cluster and populating the matrix with the calculated measures of similarity; and

identify those documents associated with a threshold measure of similarity as the most relevant documents to one such concept.

2 . A system according to claim 1 , wherein the central processing unit removes those documents from the matrix with zero values for one or more concept frequency occurrences.

3 . A system according to claim 1 , wherein the central processing unit calculates the similarity according to the following equation:

d

cluster

=

∑

i

→

n



doc

term

i

·

cluster

term

i

where doc term represents the frequency of occurrence for a given term i in one such document and cluster term represents the weight of a given cluster for a given term i.

4 . A system according to claim 1 , wherein the central processing unit updates the concept weights of the clusters until the clusters settle.

5 . A system according to claim 1 , wherein the central processing unit removes duplicate documents from the identified documents that satisfy the threshold similarity.

6 . A system according to claim 1 , wherein the central processing unit identifies the documents for clustering by creating a histogram for each document that maps a relative frequency of occurrence for at least a portion of terms in that document, creating a corpus graph that maps the concepts for the documents in the set based on the terms, defining predefined thresholds for the corpus graph, and selecting those documents with terms that satisfy the thresholds for clustering.

7 . A system according to claim 6 , wherein the central processing unit removes from the set, those documents with terms that fail to satisfy the threshold.

8 . A system according to claim 6 , wherein the thresholds are set at 1% to 15% for shorter documents.

9 . A system according to claim 6 , wherein the central processing unit sets tighter thresholds for larger documents.

10 . A system according to claim 1 , wherein the central processing unit extracts terms from the documents in the set and generates a record for each term.

11 . A computer-implemented method for identifying relevant documents, comprising:

identifying a set of documents each associated with one or more concepts;

clustering at least a portion of the documents in the set based on the concepts;

generating a matrix that provides a summary of those documents that are most relevant to one such concept, comprising:

determining for each document a measure of similarity between a concept frequency occurrence and concept weights of each cluster; and

populating the matrix with the calculated measures of similarity; and

identifying those documents associated with a threshold measure of similarity as the most relevant documents to one such concept.

12 . A method according to claim 11 , further comprising:

removing those documents from the matrix with zero values for one or more concept frequency occurrences.

13 . A method according to claim 11 , further comprising:

calculating the similarity according to the following equation:

d

cluster

=

∑

i

→

n



doc

term

i

·

cluster

term

i

where doc term represents the frequency of occurrence for a given term i in one such document and cluster term represents the weight of a given cluster for a given term i.

14 . A method according to claim 11 , further comprising:

updating the concept weights of the clusters until the clusters settle.

15 . A method according to claim 11 , further comprising:

removing duplicate documents from the identified documents that satisfy the threshold similarity.

16 . A method according to claim 11 , further comprising:

identifying the documents for clustering, comprising:

creating a histogram for each document that maps a relative frequency of occurrence for at least a portion of terms in that document;

creating a corpus graph that maps the concepts for the documents in the set based on the terms;

defining predefined thresholds for the corpus graph;

selecting those documents with terms that satisfy the thresholds for clustering.

17 . A method according to claim 16 , further comprising:

removing from the set, those documents with terms that fail to satisfy the threshold.

18 . A method according to claim 16 , wherein the thresholds are set at 1% to 15% for shorter documents.

19 . A method according to claim 16 , further comprising:

setting tighter thresholds for larger documents.

20 . A method according to claim 11 , further comprising:

extracting terms from the documents in the set; and

generating a record for each term.