Computer-implemented system and method for generating a reference set via clustering

1. A computer-implemented method for generating a reference set via clustering, comprising the steps of:

obtaining a collection of unclassified documents;

grouping the unclassified documents into clusters;

selecting n-documents from each cluster, comprising:

building a hierarchical tree of the clusters; and

traversing the hierarchical tree to identify the n-documents, wherein one of the n-documents from each cluster is located closest to a center of that cluster;

combining the selected n-documents as reference set candidates

assigning a classification code to each of the reference set candidates; and

grouping two or more of the reference set candidates as a reference set of classified documents,

wherein the steps are performed by a suitably programmed computer.

2. A method according to claim 1 , further comprising:

applying a size threshold to the reference set candidates; and

selecting the reference set candidates for inclusion in the reference set when the size threshold is satisfied.

3. A method according to claim 1 , further comprising:

applying a size threshold to the reference set candidates; and

clustering the reference set candidates until the size threshold is satisfied.

4. A method according to claim 3 , wherein the reference set candidates are clustered via one of agglomerative and divisive clustering.

5. A method according to claim 1 , further comprising at least one of:

receiving a number of the n-documents from a user; and

determining the number of the n-documents.

6. A method according to claim 1 , further comprising:

selecting an additional n-document from each cluster that is furthest from the cluster center.

7. A method according to claim 1 , further comprising:

refining the reference set candidates, comprising at least one of:

changing clustering input parameters and reclustering the unclassified documents based on the clustering input parameters;

changing the unclassified document collection by filtering out a portion of the unclassified documents; and

selecting different n-documents from each of the clusters.

8. A method according to claim 1 , further comprising:

identifying features of the unclassified documents;

grouping the features into clusters;

identifying n-features from each cluster as reference set candidate features;

assigning a classification code to each of the reference set candidate features; and

grouping at least a portion of the documents associated with the classified reference set candidate features as a further reference set.

9. A method according to claim 1 , further comprising:

propagating the classification codes of the selected n-documents to a further set of unclassified documents.

10. A computer-implemented system for generating a reference set via clustering, comprising:

a collection module to obtain a collection of unclassified documents;

a clustering module to group the unclassified documents into clusters;

a candidate selection module to select n-documents from each cluster, comprising:

a tree module to build a hierarchical tree of the clusters; and

a traversal module to traverse the hierarchical tree to identify the n-documents, wherein one of the n-documents from each cluster is located closest to a center of that cluster;

a grouping module to combine the selected n-documents as reference set candidates;

a classification module to assign a classification code to each of the reference set candidates;

a reference set module to group two or more of the reference set candidates as a reference set of classified documents; and

a processor to execute the modules.

11. A system according to claim 10 , further comprising:

a size module to apply a size threshold to the reference set candidates and to select the reference set candidates for inclusion in the reference set when the size threshold is satisfied.

12. A system according to claim 10 , further comprising:

a size module to apply a size threshold to the reference set candidates and to cluster the reference set candidates until the size threshold is satisfied.

13. A system according to claim 12 , wherein the reference set candidates are clustered via one of agglomerative and divisive clustering.

14. A system according to claim 10 , further comprising at least one of:

an instruction receipt module to receive a number of the n-documents from a user; and

a document determination module to determine the number of the n-documents.

15. A system according to claim 10 , further comprising:

selecting an additional n-document from each cluster that is furthest from the cluster center.

16. A system according to claim 10 , further comprising:

a refining module to refine the reference set candidates, comprising at least one of:

a parameter module to change clustering input parameters and to recluster the unclassified documents based on the clustering input parameters;

a filter module to change the unclassified document collection by filtering out a portion of the unclassified documents; and

a document subset module to select different n-documents from each of the clusters.

17. A system according to claim 10 , further comprising:

a feature identification module to identify features of the unclassified documents;

a feature grouping module to group the features into clusters;

a candidate feature module to identify n-features from each cluster as reference set candidate features;

a feature classification module to assign a classification code to each of the reference set candidate features; and

a feature reference module to group at least a portion of the documents associated with the classified reference set candidate features as a further reference set.

18. A system according to claim 10 , further comprising:

a propagation module to propagate the classification codes of the selected n-documents to a further set of unclassified documents.

19. A computer-implemented method for generating a reference set via clustering, comprising the steps of:

obtaining a collection of unclassified documents;

grouping the unclassified documents into clusters;

selecting n-documents from each cluster and combining the selected n-documents as reference set candidates, wherein one of the n-documents from each cluster is located closest to a center of that cluster;

assigning a classification code to each of the reference set candidates; and

grouping two or more of the reference set candidates as a reference set of classified documents, comprising:

applying a size threshold to the reference set candidates; and

clustering the reference set candidates until the size threshold is satisfied,

wherein the steps are performed by a suitably programmed computer.

20. A method according to claim 19 , further comprising:

propagating the classification codes of the selected n-documents to a further set of unclassified documents.