Document clustering and reconstruction
A scanner scans a group of documents. For example, the documents can be a group of invoices. The documents are received and processed. Objects (e.g., a text object, such as a word) and their locations are identified in each of the documents. Occurrences of similar objects in the identified locations between the documents are determined. A document sorting algorithm is applied to generate a score for each of the documents. The score for each of the documents is generated based on a number of occurrences of similar objects between the documents. The generated score of each of the documents is used to identify a template document. The template document is then used to cluster the documents.
1. A method comprising:
scanning, by an electronic scanner, a plurality of documents to produce electronic representations of the plurality of scanned documents;
receiving, by a microprocessor, the electronic representations of the plurality of scanned documents;
for each of the electronic representations of the plurality of scanned documents, identifying, by the microprocessor, a plurality of objects and physical locations of each of the plurality of objects within the electronic representations of the plurality of scanned documents;
determining, by the microprocessor, occurrences of objects in same identified physical locations of each of the plurality of objects between the electronic representations of the plurality of scanned documents, wherein the objects comprise at least one of: a same text, a same letter, a same word, a same picture, a same logo, a same phrase, a same number, a same capitalization, a same case, and a same punctuation mark;
applying, by the microprocessor, a document sorting algorithm to generate a score for each of the electronic representations of the plurality of scanned documents, wherein the score for each of the electronic representations of the plurality of scanned documents is generated based on a number of occurrences of objects in the same identified physical locations between the plurality of scanned documents; and
comparing, by the microprocessor, the generated score of each of the electronic representations of the plurality of scanned documents to identify a template document, wherein the template document is one of the plurality of scanned documents.
2. The method of claim 1 , further comprising:
determining an amount of certainty for an occurrence of objects in a common object document location between the electronic representations of the plurality of scanned documents;
identifying the common object document location based on a minimum certainty threshold value;
determining that the template document contains a scanned error for an individual object in the common object document location in the template document; and
in response to determining that the template document contains the scanned error for the individual object in the common object document location in the template document, generating an updated electronic template document, by replacing the individual object in the common object document location in the template document with a second object, wherein the second object is from the common object location in a second one of the electronic representations of the plurality of scanned documents that has been determined to be correct.
3. The method of claim 2 , wherein the scanned error for the individual object in the template document is determined based on a number of occurrences of the objects in the common document location in the electronic representations of the plurality of scanned documents.
4. The method of claim 2 , wherein each of the electronic representations of the plurality of scanned documents comprises at least two separate objects that are at different common locations between the electronic representations of the plurality of scanned documents and wherein the amount of certainty is determined based on one of:
the two separate objects; or
a single one of the two separate objects.
5. The method of claim 1 , wherein the objects comprises at least one of: a text object in an unknown language, an object that is part of a computer programming language, a phrase, a number, a punctuation mark, a text object, a graphical object, a logo, and a picture.
6. The method of claim 1 , wherein the common locations are determined based on at one or more of a distance, a relative distance, a relative angle, a character distance, a word distance, and line distance.
7. The method of claim 1 , wherein the objects are text objects and wherein the electronic representations of the plurality of scanned documents are received from the electronic scanner.
8. The method of claim 1 , wherein the document sorting algorithm generates the score for each of the electronic representations of the plurality of scanned documents by at least one of:
summing the number of occurrences of objects between the electronic representations of the plurality of scanned documents; and
multiplying the number of occurrences of objects between the electronic representations of the plurality of scanned documents.
9. The method of claim 1 , wherein the template document is used to cluster the electronic representations of the plurality of scanned documents based on the template document.
10. The method of claim 1 , wherein determining occurrences of the objects in the identified physical locations of the plurality of objects between the electronic representations of the plurality of scanned documents further comprises recalculating the identified physical locations based on a misalignment of the identified physical locations due to a use of at least one of a different font and a different font size used by a scanner when electronically scanning one or more of the electronic representations of the plurality of scanned documents.
11. A system comprising:
an electronic scanner configured to scan a plurality of documents to produce the electronic representations of the plurality of scanned documents;
a document processor comprising a microprocessor configured to receive the electronic representations of the plurality of scanned documents, for each of the electronic representations of the plurality of scanned documents, identify a plurality of objects and physical locations of each of the plurality of objects, determining occurrences of objects in same identified physical locations of each of the plurality of objects between the electronic representations of the plurality of scanned documents within the electronic representations of the plurality of scanned documents, wherein the objects comprise at least one of: a same text, a same letter, a same word, a same picture, a same logo, a same phrase, a same number, a same capitalization, a same case, and a same punctuation mark, and apply a document sorting algorithm to generate a score for each of the electronic representations of the plurality of scanned documents, wherein the score for each of the electronic representations of the plurality of scanned documents is generated based on a number of occurrences of the objects in the same identified physical locations between the electronic representations of the plurality of scanned documents; and
a document classifier configured to compare the generated score of each of the electronic representations of the plurality of scanned documents to identify a template document, wherein the template document is one of the plurality of scanned documents.
12. The system of claim 11 , wherein the document processor is further configured to determine an amount of certainty for an occurrence of objects in a common object document location between the electronic representations of the plurality of scanned documents, identify the common object document location based on a minimum certainty threshold value, determine that the template document contains a scanned error for an individual object in the common object document location in the template document, and generate an updated electronic template document by replacing the individual object in the common object document location in the template document with a second object in response to determining that the template document contains the scanned error for the individual object in the common object document location in the template document, wherein the second object is from the common object location in a second one of the electronic representations of the plurality of scanned documents that has been determined to be correct.
13. The system of claim 12 , wherein the scanned error for the individual object in the template document is determined based on a number of occurrences of the objects in the common document location in the electronic representations of the plurality of scanned documents.
14. The system of claim 12 , wherein each of the electronic representations of the plurality of scanned documents comprises at least two separate objects that are at different common locations between the electronic representations of the plurality of scanned documents and wherein the amount of certainty is determined based on one of:
the two separate objects; or
a single one of the two separate objects.
15. The system of claim 11 , wherein the objects comprises at least one of: a text object in an unknown language, an object that is part of a computer programming language, a phrase, a number, a punctuation mark, a text object, a graphical object, a logo, and a picture.
16. The method of claim 11 , further comprising a scanner that generates the electronic representations of the plurality of scanned documents and wherein the objects are text objects.
17. The system of claim 11 , wherein the document sorting algorithm generates the score for each of the electronic representations of the plurality of scanned documents by at least one of:
summing the number of occurrences of objects between the electronic representations of the plurality of scanned documents; and
multiplying the number of occurrences of objects between the electronic representations of the plurality of scanned documents.
18. The system of claim 11 , wherein the template document is used to cluster the electronic representations of the plurality of scanned documents based on the template document.
19. The system of claim 11 , wherein document processor is further configured to recalculate the identified physical locations based on a misalignment of the identified physical locations due to a use of at least one of a different font and a different font size used by an electronic scanner when electronically scanning one or more of the electronic representations of the plurality of scanned documents.
20. A system comprising:
an electronic scanner configured to scan a plurality of documents to produce electronic representations of the plurality of scanned documents;
a document processor comprising a microprocessor that is configured to receive the electronic representations of the plurality of scanned documents,
for each of the electronic representations of the plurality of scanned documents, identify a plurality of objects and locations of each of the plurality of objects, determine occurrences of objects in same identified physical locations of each of the plurality of objects between the electronic representations of the plurality of scanned documents, wherein the objects comprise at least one of: a same text, a same letter, a same word, a same picture, a same logo, a same phrase, a same number, a same capitalization, a same case, and a same punctuation mark, apply a document sorting algorithm to generate a score for each of the electronic representations of the plurality of scanned documents, wherein the score for each of the electronic representations of the plurality of scanned documents is generated based on a number of occurrences of the objects in the same identified physical locations between the electronic representations of the plurality of scanned documents, determine an amount of certainty for an occurrence of the objects in a common object document location between the electronic representations of the plurality of scanned documents, identify the common object document location based on a minimum certainty threshold value, determine that a template document contains a scanned error for an individual object in the common object document location in the template document, and generate an updated electronic template document by replacing the individual object in the common object document location in the template document with a second object in response to determining that the template document contains the scanned error for the individual object in the common object document location in the template document, wherein the second object is from the common object location in a second one of the electronic representations of the plurality of scanned documents that has been determined to be correct; and
a document classifier configured to compare the generated score of each of the electronic representations of the plurality of scanned documents to identify the template document, wherein the template document is one of the plurality of scanned documents and cluster the electronic representations of the plurality of scanned documents based on the template document.