Dynamic optimization of key value pair extractors for document data extraction
Disclosed embodiments provide techniques for monitoring and evaluating the effectiveness of key value pairs (KVPs) used in a document processing system. In embodiments, KVPs are obtained from multiple extractors of a document processing system. A score is computed for the KVPs by computing an effectiveness metric for each KVP from the multiple KVPs. In response to the computed score being below a predetermined threshold, a model retraining process is performed to generate a new set of KVP extractors, and provide the new set of KVPs to the document processing system.
1. A computer-implemented method for optimized document processing, comprising:
obtaining a plurality of key value pairs (KVPs) from a plurality of KVP extractors of a document processing system;
computing a score for the plurality of KVPs by computing an effectiveness metric for each KVP from the plurality of KVPs;
wherein the effectiveness metric factors an importance metric associated with the document processing system;
in response to the computed score being below a predetermined threshold, performing a model retraining process to generate a new set of KVP extractors;
providing the new set of KVP extractors to the document processing system; and
dynamically adjusting the effectiveness metric based on the retraining via the document processing system.
2. The method of claim 1 , wherein the effectiveness metrics are based on a level of user correction applied to the plurality of KVPs.
3. The method of claim 1 , further comprising performing an extractor analysis, based on the effectiveness metrics.
4. The method of claim 3 , wherein the extractor analysis utilizes at least one classifier selected from the group consisting of: Forest Classifier, Support Vector Machine (SVM) classifier, logistic classifier, and gradient boost classifier.
5. The method of claim 4 , wherein the gradient boost classifier includes at least one classifier selected from the group consisting of XGB classifier, and LightGBM classifier.
6. The method of claim 1 , wherein the model retraining process includes:
generating a plurality of extractor combinations; and
ranking each extractor combination of the plurality of extractor combinations based on the importance metric.
7. The method of claim 6 , wherein the importance metric is computed using at least one function selected from the group consisting of: confusion matrix, F1 score, Receiver Operator Characteristic curve, and precision-recall curve.
8. The method of claim 1 , further comprising providing a KVP ranking model to the document processing system.
9. An electronic computation device comprising:
a processor;
a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to:
obtain a plurality of key value pairs (KVPs) from a plurality of KVP extractors of a document processing system;
compute a score for the plurality of KVPs by computing an effectiveness metric for each KVP from the plurality of KVPs;
wherein the effectiveness metric factors an importance metric associated with the document processing system;
in response to the computed score being below a predetermined threshold, perform a model retraining process to generate a new set of KVP extractors;
provide the new set of KVP extractors to the document processing system; and
dynamically adjusting the effectiveness metric based on the retraining via the document processing system.
10. The electronic computation device of claim 9 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to compute the effectiveness metrics based on a level of user correction applied to the plurality of KVPs.
11. The electronic computation device of claim 9 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to perform an extractor analysis, based on the effectiveness metrics.
12. The electronic computation device of claim 11 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to compute the extractor analysis utilizing at least one classifier from the group consisting of: Forest Classifier, Support Vector Machine (SVM) classifier, logistic classifier, and gradient boost classifier.
13. The electronic computation device of claim 12 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to compute the extractor analysis utilizing a gradient boost classifier that includes at least one of XGB classifier, and LightGBM classifier.
14. The electronic computation device of claim 9 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to:
generate a plurality of extractor combinations; and
rank each extractor combination of the plurality of extractor combinations based on the importance metric.
15. The electronic computation device of claim 14 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to compute the importance metric using at least one of, a confusion matrix, F1 score, Receiver Operator Characteristic curve, and precision-recall curve.
16. The electronic computation device of claim 9 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to provide a KVP ranking model to the document processing system.
17. A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to:
obtain a plurality of key value pairs (KVPs) from a plurality of KVP extractors of a document processing system;
compute a score for the plurality of KVPs by computing an effectiveness metric for each KVP from the plurality of KVPs;
wherein the effectiveness metric factors an importance metric associated with the document processing system;
in response to the computed score being below a predetermined threshold, perform a model retraining process to generate a new set of KVP extractors; and
provide the new set of KVP extractors to the document processing system; and
dynamically adjust the effectiveness metric based on the retraining via the document processing system.
18. The computer program product of claim 17 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to compute the effectiveness metrics are based on a level of user correction applied to the plurality of KVPs.
19. The computer program product of claim 17 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to:
generate a plurality of extractor combinations; and
rank each extractor combination of the plurality of extractor combinations based on the importance metric.
20. The computer program product of claim 19 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to compute the importance metric using at least one function selected from the group consisting of: confusion matrix, F1 score, Receiver Operator Characteristic curve, and precision-recall curve.