System and Method for Processing Insurance Cards

1 . A system for processing a plurality of images of insurance cards to extract entities, comprising at least one computer-readable medium storing instructions, which, when executed:

process the images of the insurance cards using OCR to identify characters on the insurance cards and relative spatial orientation of said characters to determine a plurality of tokens and a spatial orientation of said tokens, the tokens representing possible combinations of identified characters on the insurance card;

determine coordinates for each token on the insurance card based on the spatial orientation of the tokens, the tokens and coordinates representing an OCR output;

generate a fully connected neural network including a node for each token based on the images of the insurance cards and the OCR output;

scoring each node with a member ID score for the likelihood that said node corresponds to a member ID on the insurance card; and

identifying, on each insurance card, a member ID for said insurance card based on the node with the highest member ID score.

2 . The system of claim 1 , wherein generating the fully connected neural network includes modeling the OCR output for each insurance card by using vector representations of each token.

3 . The system of claim 1 , wherein:

generating the fully connected neural network includes generating a graph based on the OCR output for each insurance card, with each token for said insurance card taken as a node of the graph and edges being declared when Euclidean distance is below a given threshold, wherein a node feature matrix is constructed based on the graph; and

scoring each node is based, at least in part, on the node feature matrix.

4 . A system for processing a plurality of insurance cards, comprising:

a camera configured to capture a plurality of images, the images including one image corresponding to each one of the insurance cards;

at least one computer configured, for each insurance card, to:

process the image of the insurance card using OCR to identify characters on the insurance card and relative spatial orientation of said characters to determine a plurality of tokens and a spatial orientation of said tokens, the tokens representing possible combinations of identified characters on the insurance card;

determine coordinates for each token on the insurance card based on the spatial orientation of the tokens;

execute a first processing step based on a first recurrent neural network (RNN), or RNN variant, to model the OCR output for each insurance card, using vector representations of each token to obtain a logit for each token;

execute a second processing step based on a graph neural network (GNN), or GNN variant, including generating embeddings based on an RNN output from the first RNN, the embeddings being vector representations of the tokens, and using the embeddings and the OCR output to generate a graph, with each token as a node, to construct a node feature matrix;

execute a third processing step using a hybrid convolutional neural network (CNN), the hybrid CNN processing the image of each insurance card with a CNN to generate an image representation of each insurance card and combining each image representation with a hidden output from the first RNN;

execute a fourth processing step using a second RNN, or RNN variant, the second RNN modeling the OCR output using a fixed length vector from the image of the insurance card; and

extract at least one entity from each insurance card based on the processing steps by assigning a score to the tokens based on a likelihood that the token corresponds to an expected characteristic, the expected characteristics including at least a member ID.

5 . The system of claim 4 , wherein, when executing the second processing step, edges define connections between nodes on the graph when a Euclidean distance between said nodes exceeds a predetermined threshold.

6 . The system of claim 4 , wherein:

each processing step generates at least one logit for each token correlating said token to with one of a plurality of expected characteristics; and

during the step of extracting the at least one entity, the score for each entity is assigned to each token based on the logits correlating said token to one of the expected characteristics.

7 . The system of claim 4 , wherein the at least one computer is further configured to train the system during the processing steps by executing the processing steps on insurance cards comprising:

a first group of insurance cards representing a validation set; and

a second group of insurance cards representing a training set.

8 . The system of claim 4 , wherein;

the at least one computer further includes a database of predetermined payer labels;

and, for each insurance card, the system determines a payer associated with said insurance card by processing the image of said insurance card and ranking a likelihood of each payer based on the database of predetermined payer labels.

9 . The system of claim 8 , wherein the system determines the payer with the CNN during execution of the third processing step.

10 . The system of claim 4 , wherein the system is further configured to generate a database of information for a plurality of members each associated with one of the insurance cards, the database registering one member for each insurance card and including at least a name and member ID for each member based on the entities extracted for said insurance card.

11 . The system of claim 4 , wherein the expected characteristics further include one or more of the following:

a name; and

an insurance company.

12 . The system of claim 4 , wherein, during execution of the second processing step, hybrid backpropagation is used to train the GNN and RNN collaboratively.

13 . The system of claim 4 , wherein, during execution of the third processing step, the CNN and first RNN are joined and the parameters of the CNN and first RNN are updated simultaneously to optimize train the hybrid CNN.