Method and system for extracting data from documents and automatically modifying data item of the extracted data based on guidance retrieved from feedback file
Improved techniques for extraction of data from documents, namely, from images of documents, so as to better enable software automation. The software automation can, for example, be provided by software robots of RPA systems. The improved techniques can provide automated feedback-based modification of data extracted from an image of a document through use of previously used validation guidance provided by a user in validating extracted data from a same or similar document. In one embodiment, the automated feedback-based modification can locate an appropriate feedback file through use of document fingerprints and document clusters. Then, guidance from the appropriate feedback file can be used to automatically modify at least a portion of the data extracted data from the image based on the guidance retrieved from the feedback file. Advantageously, the improved techniques can reduce the need for user participation in validation of data extracted from images of documents, and yield greater and more accurate automated data extraction.
1 . A computer-implemented process automation system for extracting data from one or more documents, comprising:
a data extraction module configured to use artificial intelligence or machine learning to extract data from an image of a document; and
an automated feedback-based modification module configured to process the extracted data to modify at least one data item within the extracted data, the at least one data item of the extracted data being modified in an automated manner based on a feedback file suitable for use with the document, the automated feedback-based modification module configured to at least:
determine a document fingerprint for the document;
determine the feedback file that is suitable for use with the document based on the document fingerprint;
retrieve guidance from the feedback file for the at least one data item of the extracted data being modified; and
automatically modify the at least one data item of the extracted data being modified based on the guidance retrieved from the feedback file.
2 . A computer-implemented process automation system as recited in claim 1 , wherein the automatically modifying of the at least one data item of the extracted data operates to correct, alter or replace of at least a portion of the data extracted.
3 . A computer-implemented process automation system as recited in claim 1 , wherein the system comprises:
a user validation module configured to process the extracted data to manually validate at least another data item within the extracted data that needs validation.
4 . A computer-implemented process automation system as recited in claim 3 , wherein the amount of user validation, via the user validation module, needed for the extracted data is reduced by operation of the automated feedback-based modification module.
5 . A computer-implemented process automation system as recited in claim 3 , wherein the user validation module receives user input to validate the at least another data item.
6 . A computer-implemented process automation system as recited in claim 5 , wherein the system comprises:
a feedback file creation/modification module configured to at least create a new feedback file or update the existing feedback file based on the received user input.
7 . A computer-implemented process automation system as recited in claim 1 , wherein the automated feedback-based modification module includes or couples to a feedback management module, the feedback management module being configured to locate the feedback file that is suitable for use with the document based on the document fingerprint.
8 . A computer-implemented process automation system as recited in claim 7 ,
wherein the feedback file is associated with a document cluster, and
wherein the feedback management module includes or couples to a document cluster search module that compares the document fingerprint with one or more cluster fingerprints associate with one or more document clusters.
9 . A computer-implemented process automation system as recited in claim 1 , wherein the guidance received from the feedback file includes or is derived from descriptive data for prior user validations stored in the feedback file.
10 . A computer-implemented process automation system as recited in claim 9 , wherein the descriptive data for prior user validations includes at least object name, and positional coordinates for where its value is located within a document.
11 . A computer-implemented process automation system as recited in claim 10 , wherein the descriptive data for prior user validations further includes one or more of: block type, key value orientation, data type, graphic anchor, key value anchor, total line segments, total vertical lines, and extraction error type.
12 . A computer-implemented process automation system as recited in claim 1 , wherein the automated feedback-based modification module is configured to retrieve the guidance by at least:
identifying a corresponding object with the identified feedback file, the corresponding object corresponding to the at least one data item of the extracted data being evaluated; and
retrieving feedback data pertaining to the corresponding object from the identified feedback file.
13 . A computer-implemented process automation system as recited in claim 12 , wherein the automated feedback-based modification module is configured to modify the at least one data item of the extracted data by at least:
locating the identified object at a particular location within the document using at least a portion of the retrieved feedback data; and
performing subsequent data extraction processing at the particular location to extract a value pertaining to the identified object in the document.
14 . A computer-implemented method as recited in claim 13 , wherein the automated feedback-based modification module is configured to modify the at least one data item of the extracted data by at least:
replacing the at least one data item of the extracted data with the value, the at least one data item of the extracted data being obtained from original data extraction processing.
15 . A computer-implemented method as recited in claim 13 , wherein the retrieved feedback data includes at least positional data based on where the corresponding object was previously found in the document, the positional data identifying the particular location.
16 . A computer-implemented method for enhancing extracted data from an image of a document, the method comprising:
determining a document fingerprint for the document;
identifying a feedback file that is suitable for use with the document based on the document fingerprint;
retrieving guidance from the identified feedback file pertaining to at least one data item of the extracted data being evaluated; and
enhancing the at least one data item of the extracted data being evaluated based on the guidance retrieved from the identified feedback file.
17 . A computer-implemented method as recited in claim 16 ,
wherein the at least one data item of the extracted data being evaluated includes at least one identified object that is noted as not being successfully extracted from the document, and
wherein the guidance retrieved from the identified feedback file pertaining to at least one data item of the extracted data being evaluated pertains to the same object as the identified object.
18 . A computer-implemented method as recited in claim 16 , wherein the retrieving guidance comprises:
identifying a corresponding object with the identified feedback file, the corresponding object corresponding to the at least one data item of the extracted data being evaluated; and
retrieving feedback data pertaining to the corresponding object from the identified feedback file.
19 . A computer-implemented method as recited in claim 18 , wherein the enhancing of the at least one data item of the extracted data comprises:
locating the identified object at a particular location within the document using at least a portion of the retrieved feedback data; and
performing subsequent data extraction processing at the particular location to extract a value pertaining to the identified object in the document.
20 . A computer-implemented method as recited in claim 19 , wherein the enhancing of the at least one data item of the extracted data comprises:
replacing the at least one data item of the extracted data with the value, the at least one data item of the extracted data being obtained from original data extraction processing.
21 . A computer-implemented method as recited in claim 19 , wherein the retrieved feedback data includes at least positional data based on where the corresponding object was previously found in the document, the positional data identifying the particular location.
22 . A computer-implemented method as recited in claim 16 , wherein the method comprises:
user validating at least another data item within the extracted data that needs validation after the automatically enhancement of the at least one data item of the extracted data has been performed in an automated manner.
23 . A computer-implemented method as recited in claim 22 , wherein the user validating receives user input to validate the at least another data item, and
wherein the method comprises:
following the user validating, updating the feedback file based on the received user input.
24 . A computer-implemented method as recited in claim 16 , wherein the determining the document fingerprint for the document comprises:
determining objects within the extracted data for at least a portion of the document;
determining, from the extracted data, positional information for the determined objects within the document; and
forming the document fingerprint for the document based on at least the determined objects and the determined positional information.
25 . A computer-implemented method as recited in claim 24 , wherein the identifying of the feedback file that is suitable for use with the document based on the document fingerprint comprises:
searching of a plurality of document clusters for a document cluster that matches the document fingerprint for the document, each of the document clusters having a feedback file; and
retrieving the feedback file associated with the matching document cluster, the retrieved feedback file being used as the identified feedback file that is suitable for user with the document.
26 . A computer-implemented method as recited in claim 25 , wherein the method comprises:
forming a new document cluster if the searching of the plurality of document clusters in unable to find a document cluster that matches the document fingerprint.
27 . A computer-implemented method as recited in claim 26 , wherein the new document cluster is assigned a cluster fingerprint based on the document fingerprint for the document.
28 . Computer-implemented method as recited in claim 25 ,
wherein each of the plurality of document clusters have a cluster fingerprint, and wherein the searching of the plurality of document clusters for a document cluster that matches the document fingerprint for the document, comprises:
comparing the document fingerprint to one or more of the cluster fingerprints corresponding to one or more of the document clusters; and
determining whether the document fingerprint matches any of the one or more of the cluster fingerprints based on the comparing.
29 . Computer-implemented method as recited in claim 28 , wherein the document fingerprint and the one or more cluster fingerprints include at least (i) a plurality of keys and (ii) positional references for each of the keys.
30 . A computer-implemented method as recited in claim 28 ,
wherein each of the document fingerprint and the one or more cluster fingerprints include at least a plurality of objects, and
wherein the determining whether the document fingerprint matches any of the one or more cluster fingerprints is based on at least a predetermined percentage of the objects matching exactly or within a threshold amount.
31 . A computer-implemented method as recited in claim 30 , wherein the objects pertain to at least one data item of the extracted data.
32 . A computer-implemented method as recited in claim 30 ,
wherein the objects pertain to keys and/or the positional references, and
wherein the determining whether the document fingerprint matches any of the one or more cluster fingerprints is based on at least a predetermined percentage of the keys and/r the positional references matching exactly or within a threshold amount.
33 . A computer-implemented method as recited in claim 16 , wherein retrieving the guidance from the feedback file pertaining to at least one data item of the extracted data being evaluated comprises:
parsing the feedback file for the guidance based on attributes of at least one data item of the extracted data being evaluated.
34 . A computer-implemented method as recited in claim 33 , wherein the enhancing the extracted data based on the guidance retrieved from the identified feedback file, comprises:
correcting at least the portion of the extracted data based on the guidance being retrieved from the parsing of the identified feedback file.
35 . A computer-implemented method as recited in claim 16 , wherein the enhancing of the at least one data item of the extracted data being evaluated, comprises:
modifying the at least one data item of the extracted data based on the guidance retrieved from the identified feedback file to correct, alter or replace at least a portion of the data extracted.
36 . A computer-implemented method as recited in claim 16 , wherein retrieving the guidance from the feedback file pertaining to at least one data item of the extracted data being evaluated comprises:
identifying the at least one data item from the extracted data that is to be modified;
determining a type and position of the identified data item; and
parsing the feedback file for the guidance from the feedback file pertaining to the at least one data item based on the type and position of the identified data item.