Self-attentive key-value extraction
Systems and methods are provided for automated identification of key-value pairs in documents. A document including readable text is received. The document is processed to determine, from the readable text, a plurality of tokens. Pairs of vectors corresponding to the plurality of tokens are determined, each pair of vectors comprising a query vector and a key vector. Attention scores are determined for the plurality of tokens by using the pairs of vectors. The attention scores are normalized to generate normalized attention scores. Connected tokens are identified in the plurality of tokens using the normalized attention scores.
1 . A computer-implemented method comprising:
receiving, by one or more processors of a server system, a document comprising readable text;
processing, by the one or more processors, the document to determine, from the readable text, a plurality of tokens;
determining, by the one or more processors, pairs of vectors corresponding to the plurality of tokens, each pair of vectors comprising a query vector and a key vector;
determining, by the one or more processors, attention scores for the plurality of tokens by using the pairs of vectors;
normalizing, by the one or more processors, the attention scores to generate normalized attention scores;
identifying, by the one or more processors, connected tokens in the plurality of tokens using the normalized attention scores; and
causing, by the one or more processors, the connected tokens to be processed by an application configured to perform scanned document processing by providing the connected tokens as input to the application via the server system.
2 . The computer-implemented method of claim 1 , wherein the document is formatted as an image.
3 . The computer-implemented method of claim 2 , further comprising:
processing the image, using an optical character recognition application, to generate the readable text.
4 . The computer-implemented method of claim 3 , wherein the readable text comprises keys and values.
5 . The computer-implemented method of claim 1 , wherein the vectors comprise key vectors and query vectors.
6 . The computer-implemented method of claim 1 , wherein the normalized attention scores are generated using 1) a first predictive model trained to predict links between words belonging to a same instance and configured to generate instance scores as output, and 2) a second predictive model trained to predict links between words belonging to a key-value pair and configured to generate key-value scores as output.
7 . A non-transitory computer-readable storage medium comprising programming code, which when executed by at least one data processor of a server system, causes operations comprising:
receiving a document comprising readable text;
processing the document to determine, from the readable text, a plurality of tokens;
determining pairs of vectors corresponding to the plurality of tokens, each pair of vectors comprising a query vector and a key vector;
determining attention scores for the plurality of tokens by using the pairs of vectors;
normalizing the attention scores to generate normalized attention scores;
identifying connected tokens in the plurality of tokens using the normalized attention scores; and
causing the connected tokens to be processed by an application configured to perform scanned document processing by providing the connected tokens as input to the application via the server system.
8 . The non-transitory computer-readable storage medium of claim 7 , wherein the document is formatted as an image.
9 . The non-transitory computer-readable storage medium of claim 8 , wherein the operations further comprise:
processing the image, using an optical character recognition application, to generate the readable text.
10 . The non-transitory computer-readable storage medium of claim 9 , wherein the readable text comprises keys and values.
11 . The non-transitory computer-readable storage medium of claim 7 , wherein the vectors comprise key vectors and query vectors.
12 . The non-transitory computer-readable storage medium of claim 7 , wherein the normalized attention scores are generated using 1) a first predictive model trained to predict links between words belonging to a same instance and configured to generate instance scores as output, and 2) a second predictive model trained to predict links between words belonging to a key-value pair and configured to generate key-value scores as output.
13 . A system comprising:
at least one data processor of a server system; and
at least one memory storing instructions, which when executed by the at least one data processor, cause operations comprising:
receiving a document comprising readable text;
processing the document to determine, from the readable text, a plurality of tokens;
determining pairs of vectors corresponding to the plurality of tokens, each pair of vectors comprising a query vector and a key vector;
determining attention scores for the plurality of tokens by using the pairs of vectors;
normalizing the attention scores to generate normalized attention scores;
identifying connected tokens in the plurality of tokens using the normalized attention scores; and
causing the connected tokens to be processed by an application configured to perform scanned document processing by providing the connected tokens as input to the application via the server system.
14 . The system of claim 13 , wherein the document is formatted as an image.
15 . The system of claim 14 , wherein the operations further comprise:
processing the image, using an optical character recognition application, to generate the readable text, wherein the readable text comprises keys and values.
16 . The system of claim 13 , wherein the vectors comprise key vectors and query vectors.
17 . The system of claim 13 , wherein the normalized attention scores are generated using 1) a first predictive model trained to predict links between words belonging to a same instance and configured to generate instance scores as output, and 2) a second predictive model trained to predict links between words belonging to a key value pair and configured to generate key-value scores as output.