Machine learning based system and method for automatically extracting and correcting financial information from documents

1 . A machine-learning based (ML-based) method for automatically extracting and correcting one or more financial information from one or more documents, the ML-based method comprising:

obtaining, by one or more hardware processors, the one or more documents from one or more data sources;

pre-processing, by the one or more hardware processors, the one or more documents to generate pre-processed data associated with one or more contents;

classifying, by the one or more hardware processors, the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using a machine learning (ML) model,

wherein classifying the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using the ML model, comprises:

converting, by the one or more hardware processors, the pre-processed data into unstructured data having a single unstructured string based on at least one of lines, tabs, spaces within the one or more documents, to provide alignment information associated with the one or more documents to the ML model; and

generating, by the one or more hardware processors, the one or more system prompts to classify the unstructured data as one or more potential key-value pairs using Natural Language Processing (NLP) techniques, wherein the generation of the one or more system prompts includes one of:

performing, by the one or more hardware processors, alignment of the one or more documents based on the unstructured data using the ML model being pre-trained on a plurality of documents, to mitigate one or more alignment errors from the one or more documents; and

obtaining, by the one or more hardware processors, one or more localization information from one or more users, wherein the one or more system prompts utilize the one or more localization information for classifying the unstructured data as one or more potential key-value pairs;

extracting, by the one or more hardware processors, the one or more potential key-value pairs corresponding to the one or more financial information, using the ML model;

correcting, by the one or more hardware processors, the one or more potential key-value pairs to obtain one or more corrected key-value pairs corresponding to the one or more financial information based on one or more custom prompts, using the ML model; and

providing, by the one or more hardware processors, the one or more corrected key-value pairs corresponding to the one or more financial information as an output, to one or more end users on one or more user interfaces associated with one or more electronic devices associated with the one or more end users.

2 . The ML-based method of claim 1 , further comprising automatically generating, by the one or more hardware processors, the one or more custom prompts using an auto prompt generation engine (APGE) by:

obtaining, by the one or more hardware processors, historical data associated with one or more historical documents, wherein the historical data comprise at least one of: historical state end and file definition language (FDL) data for one or more entities present in the one or more historical documents;

determining, by the one or more hardware processors, one or more occurrences in the one or more historical documents to segregate the one or more historical documents, wherein the one or more occurrences comprise at least one of: one or more user edits comprising addition or subtraction of at least one of: prefix and suffix, mis-capture of information in the one or more historical documents during Optical Character Recognition (OCR), and missing of information in the one or more historical documents;

generating, by the one or more hardware processors, the one or more custom prompts by identifying one or more patterns associated with the one or more occurrences in the one or more historical documents, wherein the one or more custom prompts enable the ML model to correct the one or more potential key-value pairs into the one or more corrected key-value pairs by performing the at least one of: the addition and the subtraction of the at least one of: the prefix and the suffix to at least one of: one or more potential keys and one or more potential values, within the one or more potential key-value pairs; and

storing, by the one or more hardware processors, the one or more custom prompts along with one or more generic prompts in one or more prompt repositories.

3 . The ML-based method of claim 2 , further comprising training, by the one or more hardware processors, the ML model by:

obtaining, by the one or more hardware processors, the one or more custom prompts from the one or more prompt repositories, as the input at the ML model, to learn the one or more patterns associated with the one or more occurrences in the one or more historical documents;

encoding, by the one or more hardware processors, the one or more patterns associated with the one or more occurrences;

labelling, by the one or more hardware processors, the one or more patterns associated with the one or more occurrences, in one or more formats to obtain a labelled information;

encoding, by the one or more hardware processors, the labelled information by assigning each labelled information with one or more numeric values; and

storing, by the one or more hardware processors, the encoded labelled information, in a lookup table for training and inferencing by the ML model.

4 . The ML-based method of claim 1 , further comprising training, by the one or more hardware processors, the ML model using one or more hyperparameters,

wherein the one or more hyperparameters comprise at least one of: n estimators indicating a number of trees in a forest model, optimum features indicating optimum number of features considered for splitting a node, optimum depth indicating optimum number of levels in each decision tree, minimum samples split indicating minimum number of data points placed in the node prior to split of the node, minimum samples leaf indicating minimum number of the data points allowed in a leaf node, and bootstrap indicating a method for sampling the data points, and

wherein the one or more hyperparameters are automatically adjusted to determine one or more optimized hyperparameters in a subset value of each of the one or more hyperparameters.

5 . The ML-based method of claim 1 , wherein pre-processing the one or more documents comprises extracting, by the one or more hardware processors, data from the one or more electronic documents, using a document scraper model, wherein the extracted data comprise at least one of: one or more words, one or more phrases, one or more numbers, and one or more characters with metadata.

6 . The ML-based method of claim 5 , wherein pre-processing the one or more documents comprises grouping, by the one or more hardware processors, the data based on at least one of: parts of speech of at least one of: the one or more words and the one or more phrases, relationship between at least one of: the one or more words and the one or more phrases, and meaning of at least one of: the one or more words and the one or more phrases, using one or more custom rules.

7 . The ML-based method of claim 6 , wherein pre-processing the data associated with the one or more electronic documents further comprises at least one of:

replacing, by the one or more hardware processors, each character in the one or more contents with one or more corresponding Unicode string values, based on one or more custom noise removal rules;

removing, by the one or more hardware processors, one or more punctuations before the one or more phrases in the one or more contents, based on the one or more custom noise removal rules; and

removing, by the one or more hardware processors, one or more characters that are recurred, from the one or more contents, based on the one or more custom noise removal rules.

8 . The ML-based method of claim 1 , further comprising:

assessing, by the one or more hardware processors, an accuracy of extracted one or more financial information by comparing the extracted one or more financial information with one or more reference financial information;

generating, by the one or more hardware processors, one or more evaluation reports providing one or more insights into the accuracy of the extracted one or more financial information;

automatically generating, by the one or more hardware processors, one or more feedback in at least one of: real-time and periodic, on the one or more evaluation reports to provide one or more suggestions on the extraction process; and

monitoring, by the one or more hardware processors, performance of the extraction process over time to determine one or more changes in accuracy metrics and error patterns.

9 . A machine learning based (ML-based) system for automatically extracting and correcting one or more financial information from one or more documents, the ML-based system comprising:

one or more hardware processors;

a memory coupled to the one or more hardware processors, wherein the memory comprises a plurality of subsystems in form of programmable instructions executable by the one or more hardware processors, and wherein the plurality of subsystems comprises:

a document obtaining subsystem configured to obtain the one or more documents from one or more data sources;

a document pre-processing subsystem configured to pre-process the one or more documents to generate pre-processed data associated with one or more contents;

a content classifying subsystem configured to classify the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using a machine learning (ML) model,

wherein in classifying the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using the ML model, the content classifying subsystem is configured to:

convert the pre-processed data into unstructured data having a single unstructured string based on at least one of lines, tabs, spaces within the one or more documents, to provide alignment information associated with the one or more documents to the ML model; and

generate the one or more system prompts to classify the unstructured data as one or more potential key-value pairs using Natural Language Processing (NLP) techniques, wherein the generation of the one or more system prompts includes one of:

 performing alignment of the one or more documents based on the unstructured data using the ML model being pre-trained on a plurality of documents, to mitigate one or more alignment errors from the one or more documents; and

 obtaining one or more localization information from one or more users, wherein the one or more system prompts utilize the one or more localization information for classifying the unstructured data as one or more potential key-value pairs;

a key-value extracting subsystem configured to extract the one or more potential key-value pairs corresponding to the one or more financial information, using the ML model;

a key-value correcting subsystem configured to correct the one or more potential key-value pairs to obtain one or more corrected key-value pairs corresponding to the one or more financial information based on one or more custom prompts, using the ML model; and

an output subsystem configured to provide the one or more corrected key-value pairs corresponding to the one or more financial information as an output, to one or more end users on one or more user interfaces associated with one or more electronic devices associated with the one or more end users.

10 . The ML-based system of claim 9 , further comprising a prompt generating subsystem configured to automatically generate the one or more custom prompts using an auto prompt generation engine (APGE), wherein in generating the one or more custom prompts using an auto prompt generation engine (APGE), the prompt generating subsystem is configured to:

obtain historical data associated with one or more historical documents, wherein the historical data comprise at least one of: historical state end and file definition language (FDL) data for one or more entities present in the one or more historical documents;

determine one or more occurrences in the one or more historical documents to segregate the one or more historical documents, wherein the one or more occurrences comprise at least one of: one or more user edits comprising addition or subtraction of at least one of: prefix and suffix, mis-capture of information in the one or more historical documents during Optical Character Recognition (OCR), and missing of information in the one or more historical documents;

generate the one or more custom prompts by identifying one or more patterns associated with the one or more occurrences in the one or more historical documents, wherein the one or more custom prompts enable the ML model to correct the one or more potential key-value pairs into the one or more corrected key-value pairs by performing the at least one of: the addition and the subtraction of the at least one of: the prefix and the suffix to at least one of: one or more potential keys and one or more potential values, within the one or more potential key-value pairs; and

store the one or more custom prompts along with one or more generic prompts in one or more prompt repositories.

11 . The ML-based system of claim 10 , further comprising a training subsystem configured to train the ML model, wherein in training the ML model, the training subsystem is configured to:

obtain the one or more custom prompts from the one or more prompt repositories, as the input at the ML model, to learn the one or more patterns associated with the one or more occurrences in the one or more historical documents;

encode the one or more patterns associated with the one or more occurrences;

label the one or more patterns associated with the one or more occurrences, in one or more formats to obtain a labelled information;

encode the labelled information by assigning each labelled information with one or more numeric values; and

store the encoded labelled information, in a lookup table for training and inferencing by the ML model.

12 . The ML-based system of claim 9 , wherein the training subsystem is further configured to train the ML model using one or more hyperparameters,

wherein the one or more hyperparameters comprise at least one of: n estimators indicating a number of trees in a forest model, optimum features indicating optimum number of features considered for splitting a node, optimum depth indicating optimum number of levels in each decision tree, minimum samples split indicating minimum number of data points placed in the node prior to split of the node, minimum samples leaf indicating minimum number of the data points allowed in a leaf node, and bootstrap indicating a method for sampling the data points, and

wherein the one or more hyperparameters are automatically adjusted to determine one or more optimized hyperparameters in a subset value of each of the one or more hyperparameters.

13 . The ML-based system of claim 9 , wherein in pre-processing the one or more documents, the document pre-processing subsystem is configured to extract data from the one or more electronic documents, using a document scraper model, wherein the extracted data comprise at least one of: one or more words, one or more phrases, one or more numbers, and one or more characters with metadata.

14 . The ML-based system of claim 13 , wherein in pre-processing the one or more documents, the document pre-processing subsystem is further configured to group the data based on at least one of: parts of speech of at least one of: the one or more words and the one or more phrases, relationship between at least one of: the one or more words and the one or more phrases, and meaning of at least one of: the one or more words and the one or more phrases, using one or more custom rules.

15 . The ML-based system of claim 14 , wherein in pre-processing the data associated with the one or more electronic documents, the document pre-processing subsystem is further configured to at least one of:

replace each character in the one or more contents with one or more corresponding Unicode string values, based on one or more custom noise removal rules;

remove one or more punctuations before the one or more phrases in the one or more contents, based on the one or more custom noise removal rules; and

remove one or more characters that are recurred, from the one or more contents, based on the one or more custom noise removal rules.

16 . The ML-based system of claim 9 , further comprising an accuracy assessment subsystem configured to:

assess an accuracy of extracted one or more financial information by comparing the extracted one or more financial information with one or more reference financial information;

generate one or more evaluation reports providing one or more insights into the accuracy of the extracted one or more financial information;

automatically generate one or more feedback in at least one of: real-time and periodic, on the one or more evaluation reports to provide one or more suggestions on the extraction process; and

monitor performance of the extraction process over time to determine one or more changes in accuracy metrics and error patterns.

17 . A non-transitory computer-readable storage medium having instructions stored therein that when executed by one or more hardware processors, cause the one or more hardware processors to execute operations of:

obtaining the one or more documents from one or more data sources;

pre-processing the one or more documents to generate pre-processed data associated with one or more contents;

classifying the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using a machine learning (ML) model,

wherein classifying the one or more contents as one or more potential key-value pairs corresponding to the one or more financial information based on one or more system prompts, using the ML model, comprises:

converting the pre-processed data into unstructured data having a single unstructured string based on at least one of lines, tabs, spaces within the one or more documents, to provide alignment information associated with the one or more documents to the ML model; and

generating the one or more system prompts to classify the unstructured data as one or more potential key-value pairs using Natural Language Processing (NLP) techniques, wherein the generation of the one or more system prompts includes one of:

performing alignment of the one or more documents based on the unstructured data using the ML model being pre-trained on a plurality of documents, to mitigate one or more alignment errors from the one or more documents; and

obtaining one or more localization information from one or more users, wherein the one or more system prompts utilize the one or more localization information for classifying the unstructured data as one or more potential key-value pairs;

extracting the one or more potential key-value pairs corresponding to the one or more financial information, using the ML model;

correcting the one or more potential key-value pairs to obtain one or more corrected key-value pairs corresponding to the one or more financial information based on one or more custom prompts, using the ML model; and

providing the one or more corrected key-value pairs corresponding to the one or more financial information as an output, to one or more end users on one or more user interfaces associated with one or more electronic devices associated with the one or more end users.