Granted Patent B2

US 12,494,076 · App. 17/994,977 · Granted Dec 9, 2025

Inventors: Sohyeong Kim (Berlin, DE); Xiang Yu (Berlin, DE)

Assignee: SAP SE

G06V30/414G06V30/19007

Abstract

Claims (50)

1 . A computer-implemented method for extracting document information, comprising:

receiving, by communication interfaces of one or more computing devices over a wired or wireless communication path, optical character recognition (OCR) information of a document, wherein the OCR information corresponds to one or more word boxes;

determining, by the one or more computing devices, beginning, inside, and outside (BIO) tags and labels of the one or more word boxes based on the OCR information;

grouping, by the one or more computing devices, a first word box and a second word box based on BIO tags of the first and the second word boxes, wherein the first word box corresponds to a beginning tag and the second word box corresponds to an inside tag;

determining that a distance between a first center point of the first word box and a second center point of the second word box is smaller than a predetermined value;

in response to determining that the distance is smaller than the predetermined value, merging, by the one or more computing devices, the first and the second word boxes into a combined word box based on a label of the first word box matching a label of the second word box; and

outputting, by the one or more computing devices, the combined word box and the label of the first word box.

2 . The computer-implemented method of claim 1 , wherein the determining the BIO tags and the labels of the one or more word boxes further comprises:

retrieving one or more features of the one or more word boxes based on the OCR information;

feeding the one or more features to a transformer encoder to obtain encoding results;

feeding the encoding results to a BIO tag decoder to obtain the BIO tags of the one or more word boxes; and

feeding the encoding results to a label decoder to obtain the labels of the one or more word boxes.

3 . The computer-implemented method of claim 2 , wherein the transformer encoder comprises an attention-based transformer model.

4 . The computer-implemented method of claim 2 , wherein the encoding results are numerical vectors.

5 . The computer-implemented method of claim 2 , wherein the one or more features comprise words, characters, box coordinates, OCR line number, and/or OCR word number.

6 . The computer-implemented method of claim 1 , wherein the BIO tags comprise beginning tags, inside tags, and outside tags.

7 . A system, comprising:

a communication interface;

a memory; and

at least one processor coupled to the memory and configured to:

receive, using the communication interface, optical character recognition (OCR) information of a document over a wired or wireless communication path, wherein the OCR information corresponds to one or more word boxes;

determine beginning, inside, and outside (BIO) tags and labels of the one or more word boxes based on the OCR information;

group a first word box and a second word box based on BIO tags of the first and the second word boxes, wherein the first word box corresponds to a beginning tag and the second word box corresponds to an inside tag;

determine that a distance between a first center point of the first word box and a second center point of the second word box is smaller than a predetermined value;

in response to determining that the distance is smaller than the predetermined value, merge the first and the second word boxes into a combined word box based on a label of the first word box matching a label of the second word box; and

output the combined word box and the label of the first word box.

8 . The system of claim 7 , wherein to determine the BIO tags and the labels of the one or more word boxes, the at least one processor is configured to:

retrieve one or more features of the one or more word boxes based on the OCR information;

feed the one or more features to a transformer encoder to obtain encoding results;

feed the encoding results to a BIO tag decoder to obtain the BIO tags of the one or more word boxes; and

feed the encoding results to a label decoder to obtain the labels of the one or more word boxes.

9 . The system of claim 8 , wherein the transformer encoder comprises an attention-based transformer model.

10 . The system of claim 8 , wherein the encoding results are numerical vectors.

11 . The system of claim 8 , wherein the one or more features comprise words, characters, box coordinates, OCR line number, and/or OCR word number.

12 . The system of claim 7 , wherein the BIO tags comprise beginning tags, inside tags, and outside tags.

13 . A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations to extract document information, the operations comprising:

receiving, using a communication interface of the at least one computing device, optical character recognition (OCR) information of a document over a wired or wireless communication path, wherein the OCR information corresponds to one or more word boxes;

determining beginning, inside, and outside (BIO) tags and labels of the one or more word boxes based on the OCR information;

grouping a first word box and a second word box based on BIO tags of the first and the second word boxes, wherein the first word box corresponds to a beginning tag and the second word box corresponds to an inside tag;

determining that a distance between a first center point of the first word box and a second center point of the second word box is smaller than a predetermined value;

in response to determining that the distance is smaller than the predetermined value, merging the first and the second word boxes into a combined word box based on a label of the first word box matching a label of the second word box; and

outputting the combined word box and the label of the first word box.

14 . The device of claim 13 , wherein the determining the BIO tags and the labels of the one or more word boxes further comprises:

retrieving one or more features of the one or more word boxes based on the OCR information;

feeding the one or more features to a transformer encoder to obtain encoding results;

feeding the encoding results to a BIO tag decoder to obtain the BIO tags of the one or more word boxes; and

feeding the encoding results to a label decoder to obtain the labels of the one or more word boxes.

15 . The device of claim 14 , wherein the transformer encoder comprises an attention-based transformer model.

16 . The device of claim 14 , wherein the encoding results are numerical vectors.

17 . The device of claim 14 , wherein the one or more features comprise words, characters, box coordinates, OCR line number, and/or OCR word number.

Assignments (1)

Continuity (1)

References Cited (13)