Techniques of information extraction for selection marks
A method may include detecting one or more selection boxes and one or more text lines in a primary document. The method may include determining respective vectors associated with the selection box and adjacent text lines to the selection box in a plurality of directions. The method may include determining a set of respective vectors associated with a unique selection box. The method may include determining a variance between respective vectors in the set of respective vectors and identifying a particular direction corresponding to a minimal variance between the respective vectors in the set of respective vectors as compared to a variance of other sets of respective vectors. The method may include generating a key-value pair based on the set of respective vectors characterized by the minimal variance. The method may include generating a document model, including the key-value pair, and extracting data according to the document model.
1 . A method, comprising:
detecting, by a computing system, one or more selection boxes in a primary document;
detecting, by the computing system, one or more text lines in the primary document;
for each selection box of the one or more selection boxes:
determining, by the computing system, respective vectors associated with the selection box and adjacent text lines to the selection box in each of a plurality of directions;
for each direction of the plurality of directions:
determining, by the computing system, a set of respective vectors, each respective vector of the set of respective vectors associated with a unique selection box of the one or more selections boxes;
for each set of respective vectors:
determining, by the computing system, a variance between the respective vectors in the set of respective vectors;
identifying, by the computing system, a particular direction corresponding to the set of respective vectors characterized by a minimal variance between the respective vectors in the set of respective vectors as compared to a variance of every other set of respective vectors;
generating, by the computing system, a key-value pair based at least in part on the adjacent text line in the particular direction corresponding to the set of respective vectors characterized by the minimal variance;
generating, by the computing system, a document model of the primary document including the key-value pair; and
extracting, by the computing system, data according to the document model.
2 . The method of claim 1 , wherein extracting data according to the document model further comprises:
accessing, by the computing system, the document model of the primary document;
receiving, by the computing system, a user-filled document based at least in part on the primary document;
determining, by the computing system, that a selection box of the one or more selection boxes has been marked; and
in response to determining that the selection box has been marked:
extracting, by the computing system and from the user-filled document, information associated with the key-value pair associated with the marked selection box.
3 . The method of claim 1 , further comprising:
determining, by the computing system, one or more classes based at least in part on annotations of the primary document, the annotations marking a subset of the one or more selection marks and adjacent text lines in the particular direction; and
assigning, by the computing system, the class as the key of the key-value pair, and each of the subset of selection marks and adjacent text lines as potential values.
4 . The method of claim 1 , wherein the one or more selection marks are extracted to a particular class.
5 . The method of claim 1 , wherein identifying, by the computing system, the particular direction corresponding to the set of respective vectors characterized by the minimal variance is performed using a neural-net model.
6 . The method of claim 1 , wherein identifying, by the computing system, the particular direction corresponding to the set of respective vectors characterized by the minimal variance is performed using a rules-based model.
7 . The method of claim 1 , further comprising providing the document model to a second computing system to process a user-filled document based at least in part on the primary document.
8 . The method of claim 1 , wherein each of the vectors comprise a distance.
9 . The method of claim 1 , wherein the key of the key value pair is the adjacent text line in the particular direction of an associated selection box of the one or more selection boxes and the value of the key value pair is associated with a state of the associated selection box.
10 . The method of claim 9 , wherein the state of the associated selection box comprises a selection mark.
11 . The method of claim 1 , wherein the adjacent text line in the particular direction corresponding to the set of respective vectors characterized by the minimal variance comprises typed and/or handwritten text.
12 . The method of claim 1 , wherein the plurality of directions comprises four directions.
13 . A computing system, comprising:
one or more processors; and
a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the system to perform operations to:
detect, by a computing system, one or more selection boxes in a primary document;
detect, by the computing system, one or more text lines in the primary document;
for each selection box of the one or more selection boxes:
determine, by the computing system, respective vectors associated with the selection box and adjacent text lines to the selection box in each of a plurality of directions, each of the respective vectors comprising a respective distance;
for each direction of the plurality of directions:
determine, by the computing system, a set of respective vectors, each respective vector of the set of respective vectors associated with a unique selection box of the one or more selections boxes;
for each set of respective vectors:
determine, by the computing system, a variance between the respective vectors in the set of respective vectors;
identifying, by the computing system, a particular direction corresponding to the set of respective vectors characterized by a minimal variance between the respective vectors in the set of respective vectors as compared to a variance of every other set of respective vectors;
generate, by the computing system, a key-value pair based at least in part on the adjacent text line in the particular direction corresponding to the set of respective vectors characterized by the minimal variance;
generate, by the computing system, a document model of the primary document including the key-value pair; and
extract, by the computing system, data according to the document model.
14 . The computing system of claim 13 , the operations further causing the computing system to:
access, by the computing system, the document model of the primary document;
receive, by the computing system, a user-filled document based at least in part on the primary document;
determine, by the computing system, that a selection box of the one or more selection boxes has been marked; and
in response to determining that the selection box has been marked:
extract, by the computing system and from the user-filled document, information associated with the key-value pair associated with the marked selection box.
15 . The computing system of claim 13 , the operations further causing the computing system to:
determining, by the computing system, one or more classes based at least in part on annotations of the primary document, the annotations marking a subset of the selection marks and adjacent text lines in the particular direction; and
assigning, by the computing system, the class as the key of the key-value pair, and each of the subset of selection marks and adjacent text lines as potential values.
16 . The computing system of claim 13 , wherein identifying, by the computing system, the particular direction corresponding to the set of respective vectors characterized by the minimal variance is performed using a neural-net model.
17 . The computing system of claim 13 , wherein identifying, by the computing system, the particular direction corresponding to the set of respective vectors characterized by the minimal variance is performed using a rules-based model.
18 . The computing system of claim 13 , further comprising providing the document model to a second computing system to process a user-filled document based at least in part on the primary document.
19 . The computing system of claim 13 , wherein the key of the key value pair is the adjacent text line in the particular direction of an associated selection box of the one or more selection boxes and the value of the key value pair is associated with a state of the associated selection box.
20 . A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:
detecting, by a computing system, one or more selection boxes in a primary document;
detecting, by the computing system, one or more text lines in the primary document;
for each selection box of the one or more selection boxes:
determining, by the computing system, respective vectors associated with the selection box and adjacent text lines to the selection box in each of a plurality of directions, each of the respective vectors comprising a respective distance;
for each direction of the plurality of directions:
determining, by the computing system, a set of respective vectors, each respective vector of the set of respective vectors associated with a unique selection box of the one or more selections boxes;
for each set of respective vectors:
determining, by the computing system, a variance between the respective vectors in the set of respective vectors;
identifying, by the computing system, a particular direction corresponding to the set of respective vectors characterized by a minimal variance between the respective vectors in the set of respective vectors as compared to a variance of every other set of respective vectors;
generating, by the computing system, a key-value pair based at least in part on the adjacent text line in the particular direction corresponding to the set of respective vectors characterized by the minimal variance;
generating, by the computing system, a document model of the primary document including the key-value pair; and
extracting, by the computing system, data according to the document model.