Systems and methods for segmentation of report corpus using visual signatures

1. A computer-implemented method, comprising:

converting a document to a grayscale image;

removing noise from the grayscale image by eroding isolated pixels;

determining connected regions in the grayscale image;

identifying a region of the grayscale image having a square shape;

computing an area of the region;

if the area is larger than a threshold, determining that the document contains a form; and

comparing the form with a second form to determine if the second form is similar to the form, wherein comparing the form comprises:

dividing the form into a plurality of grids;

summing pixels in each grid of the plurality of grids;

dividing the second form into a second plurality of second grids;

summing pixels in each second grid of the plurality of second grids; and

comparing the form further comprises computing a pairwise difference between a grid of the plurality of grids with a second grid of the second plurality of grids.

2. The computer-implemented method of claim 1 , wherein comparing the form further comprises determining the second form is similar to the form if the pairwise difference is less than a second threshold.

3. The computer-implemented method of claim 2 , wherein comparing the form further comprises determining the second form is different from the form if the pairwise difference is greater than the second threshold.

4. The computer-implemented method of claim 3 , further comprising clustering a plurality of forms using a silhouette score into a first cluster and a second cluster.

5. The computer-implemented method of claim 4 , further comprising using a first data extraction process specific to a first structure of the first cluster of forms and using a second data extraction process specific to a second structure of the second cluster of forms.

6. The computer-implemented method of claim 5 , further comprising providing a unique identifier to each form.

7. A system, comprising:

a plurality of document databases containing forms and non-forms;

a network interconnecting the plurality of document databases;

a document analysis server that converts a document from a document database to a grayscale image, removes noise from the grayscale image by eroding isolated pixels, determines connected regions in the grayscale image; identifies a region of the grayscale image having a square shape, computes an area of the region, if the area is larger than a threshold, determines that the document contains a form; compares the form with a second form to determine if the second form is similar to the form, wherein comparing the form comprises dividing the form into a plurality of grids and summing pixels in each grid of the plurality of grids; divides the second form into a second plurality of second grids and sums pixels in each second grid of the plurality of second grids; and computes a pairwise difference between a grid of the plurality of grids with a second grid of the second plurality of grids.

8. The system of claim 7 , wherein the document analysis server determines whether the second form is similar to the form if the pairwise difference is less than a second threshold.

9. The system of claim 8 , wherein the document analysis server determines the second form is different from the form if the pairwise difference is greater than the second threshold.

10. The system of claim 9 , wherein the document analysis server clusters a plurality of forms using a silhouette score into a first cluster and a second cluster.

11. The system of claim 10 , wherein the document analysis server uses a first data extraction process specific to a first structure of the first cluster of forms and uses a second data extraction process specific to a second structure of the second cluster of forms.

12. The system of claim 11 , wherein the document analysis server provides a unique identifier to each form.