AUTOMATED DOCUMENT PROCESSING
A system is described whereby a classifier can label pages of a document. Preliminary classification can be performed on a page-by-page basis based on the content and form of each individual page. The system can then perform a sequence-based classification based on the preliminary classification of preceding and following pages. This approach can use a hidden Markov model and can result in more accurate page labels in a document
1 . A method comprising:
calculating a map of state change probabilities between a plurality of page types, the state change probabilities indicating at least the probability that a first page type will precede a second page type;
receiving a document package comprising a plurality of page items;
determining, using a classifier, page type probability vectors for each of the plurality of page items; and
calculating predicted page types for each of the plurality of page items based on the respective page type probability vectors and the map of state change probabilities.
2 . The method of claim 1 , further comprising:
determining a first chain of predicted page types for at least a subset of the plurality of page items;
calculating a first score for the first chain based on the page type probability vectors and the map of state change probabilities;
determining a second chain of predicted page types for the at least a subset of the plurality of page items;
calculating a second score for the second chain based on the page type probability vectors and the map of state change probabilities; and
determine that the first chain of predicted page types is more likely than the second chain based on the first score and the second score.
3 . The method of claim 1 , further comprising:
identifying one or more document type field regions of a particular page item of the plurality of page items based on a respective predicted page type for the particular page item;
obtaining data from the one or more document type field regions;
validating the data based on at least one validation rule for the respective predicted page type; and
storing the data in a database.
4 . The method of claim 1 , wherein the classifier calculates page type probability vectors using a convolutional neural network and/or optical character recognition of the respective page items.
5 . The method of claim 1 , wherein the map of state change probabilities includes the probability that a third page type will follow the second page type.
6 . A system, comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the system to:
calculate a map of state change probabilities between a plurality of page types, the state change probabilities indicating at least the probability that a first page type will precede a second page type;
receive a document package comprising a plurality of page items;
determine, using a classifier, page type probability vectors for each of the plurality of page items; and
calculate predicted page types for each of the plurality of page items based on the respective page type probability vectors and the map of state change probabilities.
7 . The system of claim 6 , wherein the instructions when executed further cause the system to:
determine a first chain of predicted page types for at least a subset of the plurality of page items;
calculate a first score for the first chain based on the page type probability vectors and the map of state change probabilities;
determine a second chain of predicted page types for the at least a subset of the plurality of page items;
calculate a second score for the second chain based on the page type probability vectors and the map of state change probabilities; and
determine that the first chain of predicted page types is more likely than the second chain based on the first score and the second score.
8 . The system of claim 6 , wherein the instructions when executed further cause the system to:
identify one or more document type field regions of a particular page item of the plurality of page items based on a respective predicted page type for the particular page item;
obtaine data from the one or more document type field regions;
validate the data based on at least one validation rule for the respective predicted page type; and
store the data in a database.
9 . The system of claim 6 , wherein the classifier calculates page type probability vectors using a convolutional neural network and/or optical character recognition of the respective page items.
10 . The system of claim 6 , wherein the map of state change probabilities includes the probability that a third page type will follow the second page type.