Data extraction pipeline

1. A method for extracting information from an image of a document comprising:

acquiring an image comprising image data relating to at least a part of the document;

manipulating the image data to obtain a document image;

performing optical character recognition of the image data to produce recognized data;

attempting non-textual classification of the document image by machine recognition of non-textual characters to obtain classification data;

attempting textual classification of the document image by machine recognition of textual characters to obtain classification data; and

using the classification data and recognized data to extract information from the image;

wherein attempting textual classification of the document image by machine recognition of textual characters to obtain classification data comprises keyword classification of the document image to obtain classification data, wherein keyword classification comprises:

obtaining the recognized data;

retrieving a list of template data;

obtaining a weighting matrix using the list of template data;

classifying the document to a determined document type using the list of template data and the weighting matrix; and

if a threshold level of confidence in the classification is achieved, obtaining classification data by semantic labelling of the document image.

2. The method of claim 1 , wherein attempting non-textual classification of the document image by machine recognition of non-textual characters to obtain classification data comprises attempting machine readable code classification of the document image to obtain classification data,

wherein machine readable code classification comprises:

searching for a machine readable code segment in the document image; and

if at least one machine readable code is found, obtaining classification data from the at least one machine readable code segment.

3. The method of claim 1 , wherein semantic labelling of the document image comprises:

retrieving standard data of the determined document type, the standard data comprising keywords, keyword positions, and expected data patterns;

forming a homography that maps the standard data to the recognized data;

identifying additional classification data in the recognized data using the homography;

filtering the additional classification data by pattern matching using the expected data patterns; and

obtaining classification data comprising the filtered additional classification data.

4. The method of claim 3 , wherein the step of filtering the additional classification data by pattern matching using the expected data patterns comprises:

comparing the additional classification data by pattern matching the data entries of the additional classification data to the expected data patterns to form a data comparison result for each of the data entries;

flagging the data entries of the additional classification data that have a comparison result that exceeds at least one predefined threshold; and

correcting or removing the flagged data entries.

5. The method of claim 1 , wherein attempting textual classification of the document image by machine recognition of textual characters to obtain classification data is performed if non-textual classification has failed.

6. The method of claim 1 , wherein attempting textual classification of the document image by machine recognition of textual characters to obtain classification data comprises attempting at least one of:

keyword classification of the document image to obtain classification data;

convolutional neural network classification of the document image to obtain classification data; and

visual classification of the document image to obtain classification data.

7. The method of claim 1 , wherein if classification data has been obtained and a document type determined, or derived from the classification data, the classification data is validated by a process comprising comparing the classification data with standard data of a document of the determined document type.

8. The method of claim 1 , wherein the computer-implemented method further comprises consistency checking the classification data formed by non-textual classification, and wherein the consistency checking comprises:

optical character recognition of the image data to produce recognized data; and

consistency checking the classification data using the recognized data.

9. The method of claim 1 , wherein the step of attempting non-textual classification or the step of attempting textual classification comprises performing the associated classification and recording a result of associated classification.

10. The method of claim 9 , wherein the recorded result is marked as a fail if the associated classification did not complete.

11. The method of claim 9 , wherein the recorded result is marked as a fail if the obtained classification data does not comprise an expected data pattern or structure associated with a document type that was derived during the associated classification.

12. The method of claim 1 , wherein the weighting matrix is a term frequency-inverse document frequency weighting matrix characterizing the difference in term frequency of each extracted keyword relative to a corpus.

13. A method for extracting information from an image of a document comprising:

acquiring an image comprising image data relating to at least a part of the document;

manipulating the image data to obtain a document image;

performing optical character recognition of the image data to produce recognized data;

attempting non-textual classification of the document image by machine recognition of non-textual characters to obtain classification data;

attempting textual classification of the document image by machine recognition of textual characters to obtain classification data; and

using the classification data and recognized data to extract information from the image;

wherein attempting textual classification of the document image by machine recognition of textual characters to obtain classification data comprises:

attempting keyword classification of the document image to obtain classification data;

if keyword classification has failed, attempting convolutional neural network classification of the document image to obtain classification data; and

if convolutional neural network classification has failed, attempting visual classification of the document image to obtain classification data.

14. A method for extracting information from an image of a document comprising:

acquiring an image comprising image data relating to at least a part of a document;

classifying the image data as comprising a determined document type;

using knowledge of the determined document type to extract image data from the document image;

segmenting the extracted image data to obtain segmentation data;

performing optical character recognition on the segmentation data to obtain recognized data;

post-processing the recognized data to obtain classification data; and

using the classification data and recognized data to extract information from the image;

wherein performing optical character recognition on the segmented data comprises: cropping the image data using the segmentation data, recognizing text in the cropped image data, and obtaining recognized data, the recognized data comprising the position of the cropped image data and the recognized text;

wherein recognizing text in the cropped image data comprises using prior knowledge about the determined document type and using a recognizing network comprising a deep convolutional neural network with long short-term memory network layers on top.

15. The method of claim 14 , wherein classifying the image data as comprising a determined document type comprises using a deep convolutional neural network to assign the document image as comprising a determined document type.

16. The method of claim 14 , wherein segmenting the extracted image data to obtain segmentation data comprises:

searching the extracted image data to find at least one text field;

associating a label with each of the at least one text field;

obtaining segmentation data from the extracted image data, the segmentation data comprising the position of each of the at least one text field and the associated label; and

returning the segmentation data.

17. The method of any claim 14 , wherein segmenting the extracted image data to obtain segmentation data comprises using a deep convolutional neural network.