Representative document selection

1. A system, comprising:

one or more memory units; and

one or more processing units operable to:

access a set of documents;

for each particular document of the set of documents, determine a distance from the particular document to a nearest document in a set of representative documents;

select a subset of documents using an algorithm for choosing initial seed values and the determined distances to the nearest representative document, the subset of documents comprising a predetermined number of documents from the set of documents;

repeat for each particular document of the subset of documents:

add the particular document of the subset of documents to the set of representative documents to create a new set of representative documents;

remove the particular document of the subset of documents from the set of documents to create a new set of documents; and

calculate a sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents; and

select as a new representative document the particular document of the subset of documents that resulted in the lowest sum of distances.

2. The system of claim 1 , the one or more processing units further operable to:

generate a first term-document matrix (TDM) comprising terms from the set of documents; and

generate a second TDM comprising terms from the subset of documents;

wherein:

determining the distances to a nearest representative document identified in the set of representative documents comprises utilizing the first TDM; and

calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the second TDM.

3. The system of claim 1 , wherein the distance from each particular document in the set of documents to a nearest document in the set of representative documents comprises a Euclidean distance squared.

4. The system of claim 1 , wherein the algorithm for choosing initial seed values comprises k-means++ probabilistic seed selection.

5. The system of claim 1 , wherein determining the distance from each particular document of the set of documents to a nearest document in the set of representative documents comprises utilizing the following equation:

min

r

∈

R

⁢



d

⁢

-

r



2

;

wherein R is the set of representative documents, d is a document in the set of documents, and r is a document in the set of representative documents.

6. The system of claim 1 , wherein selecting the subset of documents comprises:

calculating a total using the following equation:

Total

=

∑

d

∈

D

⁢

min

r

∈

R

⁢



d

-

r



2

;

calculating a probability for each document in the set of documents according to the following equation:

min

r

∈

R

⁢



d

-

r



2

Total

;

and

randomly selecting a predetermined number of the documents in the set of documents based on the calculated probabilities;

wherein D is the set of documents, R is the set of representative documents, d is a document in the set of documents, and r is a document in the set of representative documents.

7. The system of claim 1 , wherein calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the following equation:

TOTALc

=

∑

d

⁢

⁢

c

∈

Dc

⁢

min

rc

∈

Rc

⁢



d

⁢

⁢

c

-

rc



2

;

wherein Dc is the new set of documents, Rc is the new set of representative documents, dc is a document in the new set of documents, and rc is a document in the new set of representative documents.

8. A computer-implemented method, comprising:

accessing, by one or more computing systems, a set of documents;

for each particular document of the set of documents, determining, by the one or more computing systems, a distance from the particular document to a nearest document in a set of representative documents;

selecting, by the one or more computing systems using an algorithm for choosing initial seed values and the determined distances to the nearest representative document, a subset of documents, the subset of documents comprising a predetermined number of documents from the set of documents;

repeating, by the one or more computing systems, for each particular document of the subset of documents:

adding the particular document of the subset of documents to the set of representative documents to create a new set of representative documents;

removing the particular document of the subset of documents from the set of documents to create a new set of documents; and

calculating a sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents; and

selecting, by the one or more computing systems, the particular document of the subset of documents that resulted in the lowest sum of distances as a new representative document.

9. The computer-implemented method 8 further comprising:

generating, by the one or more computing systems, a first term-document matrix (TDM) comprising terms from the set of documents; and

generating, by the one or more computing systems, a second TDM comprising terms from the subset of documents;

wherein:

determining the distances to a nearest representative document identified in the set of representative documents comprises utilizing the first TDM; and

calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the second TDM.

10. The computer-implemented method 8, wherein the distance from each particular document in the set of documents to a nearest document in the set of representative documents comprises a Euclidean distance squared.

11. The computer-implemented method 8, wherein the algorithm for choosing initial seed values comprises k-means++ probabilistic seed selection.

12. The computer-implemented method 8, wherein determining the distance from each particular document of the set of documents to a nearest document in the set of representative documents comprises utilizing the following equation:

min

r

∈

R

⁢



d

⁢

-

r



2

;

wherein R is the set of representative documents, d is a document in the set of documents, and r is a document in the set of representative documents.

13. The computer-implemented method 8, wherein selecting the subset of documents comprises:

calculating a total using the following equation:

Total

=

∑

d

∈

D

⁢

min

r

∈

R

⁢



d

-

r



2

;

calculating a probability for each document in the set of documents according to the following equation:

min

r

∈

R

⁢



d

-

r



2

Total

;

and

randomly selecting a predetermined number of the documents in the set of documents using the calculated probabilities;

wherein D is the set of documents, R is the set of representative documents, d is a document in the set of documents, and r is a document in the set of representative documents.

14. The computer-implemented method 8, wherein calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the following equation:

TOTALc

=

∑

d

⁢

⁢

c

∈

Dc

⁢

min

rc

∈

Rc

⁢



d

⁢

⁢

c

-

rc



2

;

wherein Dc is the new set of documents, Rc is the new set of representative documents, dc is a document in the new set of documents, and rc is a document in the new set of representative documents.

15. A non-transitory computer-readable medium comprising software, the software when executed by one or more processing units operable to perform operations comprising:

accessing a set of documents;

repeating for each particular document of the set of documents:

adding the particular document to a set of representative documents to create a new set of representative documents;

removing the particular document from the set of documents to create a new set of documents; and

calculating a sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents; and

selecting the particular document of the set of documents that resulted in the lowest sum of distances as a new representative document.

16. The non-transitory computer-readable medium of claim 15 , the operations further comprising generating a term-document matrix (TDM) comprising terms from the set of documents, wherein calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the generated TDM.

17. The non-transitory computer-readable medium of claim 15 , wherein the distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises a Euclidean distance squared.

18. The non-transitory computer-readable medium of claim 15 , wherein determining the distance from each particular document of the set of documents to a nearest document in the set of representative documents comprises utilizing the following equation:

min

r

∈

R

⁢



d

⁢

-

r



2

;

wherein R is the set of representative documents, d is a document in the set of documents, and r is a document in the set of representative documents.

19. The non-transitory computer-readable medium of claim 15 , wherein calculating the sum of distances from each document of the new set of documents to a nearest document in the new set of representative documents comprises utilizing the following equation:

TOTALc

=

∑

d

⁢

⁢

c

∈

Dc

⁢

min

rc

∈

Rc

⁢



d

⁢

⁢

c

-

rc



2

;

wherein Dc is the new set of documents, Rc is the new set of representative documents, dc is a document in the new set of documents, and rc is a document in the new set of representative documents.

20. The non-transitory computer-readable medium of claim 15 , the operations further comprising presenting the new representative document to a user.