Text document categorization using rules and document fingerprints

1. A method comprising:

obtaining a plurality of text documents;

classifying the plurality of text documents into one or more document categories based on a plurality of classification rules, wherein each of the one or more document categories include one or more first text documents of the plurality of text documents,

identifying a second text document that is not classifiable to the one or more document categories based on the plurality of classification rules;

after the plurality of text documents are classified into the one or more document categories based on the classification rules, generating one or more document fingerprints for respective first text documents in the one or more document categories;

generating a second document fingerprint for the second text document; and

classifying the second text document into one of the one or more document categories based on the fingerprint for the second text document and the one or more document fingerprints for the respective first text documents.

2. The method of claim 1 , further comprising obtaining the plurality of classification rules from a rule repository.

3. The method of claim 1 , further comprising generating an index table correlating the one or more document fingerprints to the respective first text documents and to one or more respective document categories.

4. The method of claim 3 , wherein a first document fingerprint of the one or more document fingerprints is generated by:

obtaining a first sequence of a respective first text document;

generating a first hash based on the first sequence;

obtaining a second sequence of the respective first text document, the second sequence being shifted from the first sequence;

generating a second hash based on the second sequence; and

generating the first document fingerprint for the respective first text document based on the first hash and the second hash.

5. The method of claim 4 , wherein the first sequence and the second sequence are shifted from each other by a predetermined length of characters.

6. The method of claim 4 , further comprising:

using a same hash function to generate the first hash and the second hash.

7. The method of claim 4 , further comprising:

calculating a similarity value from the second document fingerprint and the first document fingerprint;

determining whether the similarity value is greater than a predetermined threshold; and

in response to determining that the similarity value is greater than the predetermined threshold, classifying the second text document into the one of the one or more document categories.

8. An apparatus comprising:

one or more processors; and

a memory configured to store instructions executable by the one or more processors,

wherein the one or more processors are configured to execute the instructions to perform operations including:

obtaining a plurality of text documents;

classifying the plurality of text documents into one or more document categories based on a plurality of classification rules, wherein each of the one or more document categories include one or more first text documents of the plurality of text documents,

identifying a second text document that is not classifiable to the one or more document categories based on the plurality of classification rules;

after the plurality of text documents are classified into the one or more document categories based on the classification rules, generating one or more document fingerprints for respective first text documents in the one or more document categories;

generating a fingerprint for the second text document; and

classifying the second text document into one of the one or more document categories based on the fingerprint for the second text document and the one or more document fingerprints for the respective first text documents.

9. The apparatus of claim 8 , wherein the operations further comprise:

obtaining the plurality of classification rules from a rule repository.

10. The apparatus of claim 8 , wherein the operations further comprise:

generating an index table correlating the one or more document fingerprints to the respective first text documents and to one or more respective document categories.

11. The apparatus of claim 10 , wherein a first document fingerprint of the one or more document fingerprints is generated by:

obtaining a first sequence of a respective first text document;

generating a first hash based on the first sequence;

obtaining a second sequence of the respective first text document, the second sequence being shifted from the first sequence;

generating a second hash based on the second sequence; and

generating the first document fingerprint for the respective first text document based on the first hash and the second hash.

12. The apparatus of claim 11 , wherein the first sequence and the second sequence are shifted from each other by a predetermined length of characters.

13. The apparatus of claim 11 , wherein the operations further comprise:

using a same hash function to generate the first hash and the second hash.

14. The apparatus of claim 11 , wherein the operations further comprise:

calculating a similarity value from the second document fingerprint and the first document fingerprint;

determining whether the similarity value is greater than a predetermined threshold; and

in response to determining that the similarity value is greater than the predetermined threshold, classifying the second text document into the one of the one or more document categories.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:

obtaining a plurality of text documents;

classifying the plurality of text documents into one or more document categories based on a plurality of classification rules, wherein each of the one or more document categories include one or more first text documents of the plurality of text documents,

identifying a second text document that is not classifiable to the one or more document categories based on the plurality of classification rules;

after the plurality of text documents are classified into the one or more document categories based on the classification rules, generating one or more document fingerprints for respective first text documents in the one or more document categories;

generating a second document fingerprint for the second text document; and

classifying the second text document into one of the one or more document categories based on the fingerprint for the second text document and the one or more document fingerprints for the respective first text documents.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

obtaining the plurality of classification rules from a rule repository.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

generating an index table correlating the one or more document fingerprints to the respective first text documents and to one or more respective document categories.

18. The non-transitory computer-readable storage medium of claim 17 , wherein a first document fingerprint of the one or more document fingerprints is generated by:

obtaining a first sequence of a respective first text document;

generating a first hash based on the first sequence;

obtaining a second sequence of the respective first text document, the second sequence being shifted from the first sequence;

generating a second hash based on the second sequence; and

generating the first document fingerprint for the respective first text document based on the first hash and the second hash.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

using a same hash function to generate the first hash and the second hash.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

calculating a similarity value from the second document fingerprint and the first document fingerprint;

determining whether the similarity value is greater than a predetermined threshold; and

in response to determining that the similarity value is greater than the predetermined threshold, classifying the second text document into the one of the one or more document categories.