IP Library › Granted Patent US 12,230,046
Granted Patent B2
US 12,230,046 · App. 18/492,910 · Granted Feb 18, 2025

Searchable table extraction

Inventors: Shivam Agrahari (Bansi, IN); Dakshayani Singaraju (Tirupati, IN); Veresh Jain (Bangalore, IN)
Assignee: Oracle Financial Services Software Limited
G06V30/412G06F16/8373G06F40/103G06F40/143G06V10/82G06V30/16G06V30/19147G06V30/413
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,230,046
App. No.
18/492,910
Filed
Oct 24, 2023
Granted
Feb 18, 2025
Kind
B2
Examiner
FABER, DAVID
Art Unit
2172
USPC
715/227
Abstract

Tables including cells can be extracted from an input document image, as objects and processed to be output in an XML format. The tables can be filtered based on one or more keywords, to reduce a number of the tables. The filtered tables that include the one or more keywords can be obtained. A query can be executed on the filtered tables, the query including one or more keys composed based on cell level information associated with the filtered tables. One or more cells among the cells can be identified based on the one or more keys. As a response to the query, the target content that corresponds to the one or more cells can be provided.

Claims (89)

1. A computer-implemented method comprising:

extracting, by a table extraction model from an input document image, a plurality of tables as a plurality of objects, the plurality of tables including a plurality of cells, respectively;

outputting, by the table extraction model, the plurality of tables in an extensible Markup Language (XML) format and outputting cell level information of the plurality of cells;

filtering the plurality of tables based on a plurality of keywords and a condition comprising one from among AND and OR, to reduce a number of the plurality of tables, the filtering further comprising:

for each of the plurality of keywords, traversing each of the plurality of tables along a row direction and a column direction using one or more sub-keywords, each of the one or more sub-keywords being composed of one or more words of the keyword,

based on the traversing:

for each of the plurality of tables, determining whether a table comprises a first keyword from the plurality of keywords and determining whether the table comprises a second keyword from the plurality of keywords,

for the condition being AND, based on the table containing the first keyword and the second keyword, outputting the table, as a filtered table among filtered tables, and, based on the table not containing the first keyword and the second keyword, not outputting the table, and

for the condition being OR, based on the table containing the first keyword or the second keyword, outputting the table, as a filtered table among the filtered tables;

outputting the filtered tables that include the at least one from among the first keyword and the second keyword based on the condition, among the plurality of tables;

executing a query on the filtered tables, the query including one or more keys composed based on the cell level information associated with the filtered tables;

identifying one or more cells among the plurality of cells based on the one or more keys; and

providing target content that corresponds to the one or more cells, wherein the target content is a response to the query.

2. The computer-implemented method of claim 1 , further comprising:

obtaining a set of conditions, the set of conditions comprising AND and OR, prior to the filtering; and

obtaining the condition for the filtering from the set of conditions.

3. The computer-implemented method of claim 1 , wherein the filtering further comprises:

initializing a vector comprising an array having a number of elements equal to a number of the plurality of keywords;

based on the table containing the first keyword, generating a value of 1 as a first element of the array;

based on the table containing the second keyword, generating a value of 1 as a second element of the array;

based on the condition being AND, outputting the table if the first element and the second element have the value of 1; and

based on the condition being OR, outputting the table if at least one from among the first element and the second element has the value of 1.

4. The computer-implemented method of claim 1 , wherein the executing the query further comprises:

obtaining the cell level information associated with each of the filtered tables, the cell level information comprising, for each of the filtered tables, coordinates of cells and table architecture; and

deriving grammar and rules specific to each of the filtered tables, using the coordinates of cells and the table architecture.

5. The computer-implemented method of claim 4 , wherein the executing the query further comprises creating the query based on the grammar and rules specific to each of the filtered tables.

6. The computer-implemented method of claim 5 , wherein:

at least one of the filtered tables is a nested table, and

the query comprises at least one composite key that expresses parent-child relationship of a header and at least one sub-header present in the nested table.

7. A computer system comprising:

one or more data processors; and

one or more non-transitory computer-readable storage media storing instructions that, when executed by the one or more data processors, cause the one or more data processors to perform a method including:

extracting, by using a table extraction model from an input document image, a plurality of tables as a plurality of objects, the plurality of tables including a plurality of cells, respectively;

receiving, as an output the table extraction model, the plurality of tables in an extensible Markup Language (XML) format and outputting cell level information of the plurality of cells;

filtering the plurality of tables based on a plurality of keywords and a condition comprising one from among AND and OR, to reduce a number of the plurality of tables, the filtering further including:

for each of the plurality of keywords, traversing each of the plurality of tables along a row direction and a column direction using one or more sub-keywords, each of the one or more sub-keywords being composed of one or more words of the keyword,

based on the traversing:

for each of the plurality of tables, determining whether a table comprises a first keyword from the plurality of keywords and determining whether the table comprises a second keyword from the plurality of keywords,

for the condition being AND, based on the table containing the first keyword and the second keyword, outputting the table, as a filtered table among filtered tables, and, based on the table not containing the first keyword and the second keyword, not outputting the table, and

for the condition being OR, based on the table containing the first keyword or the second keyword, outputting the table, as a filtered table among the filtered tables;

outputting the filtered tables that include the at least one from among the first keyword and the second keyword based on the condition, among the plurality of tables;

executing a query on the filtered tables, the query including one or more keys composed based on the cell level information associated with the filtered tables;

identifying one or more cells among the plurality of cells based on the one or more keys; and

providing target content that corresponds to the one or more cells, wherein the target content is a response to the query.

8. The computer system of claim 7 , wherein the method further includes:

obtaining a set of conditions, the set of conditions comprising AND and OR, prior to the filtering; and

obtaining the condition for the filtering from the set of conditions.

9. The computer system of claim 8 , wherein the filtering further includes:

initializing a vector comprising an array having a number of elements equal to a number of the plurality of keywords;

based on the table containing the first keyword, generating a value of 1 as a first element of the array;

based on the table containing the second keyword, generating a value of 1 as a second element of the array;

based on the condition being AND, outputting the table if the first element and the second element have the value of 1; and

based on the condition being OR, outputting the table if at least one from among the first element and the second element has the value of 1.

10. The computer system of claim 7 , wherein the executing the query further includes:

obtaining the cell level information associated with each of the filtered tables, the cell level information comprising, for each of the filtered tables, coordinates of cells and table architecture; and

deriving grammar and rules specific to each of the filtered tables, using the coordinates of cells and the table architecture.

11. The computer system of claim 10 , wherein the executing the query further includes creating the query based on the grammar and rules specific to each of the filtered tables.

12. The computer system of claim 11 , wherein:

at least one of the filtered tables is a nested table, and

the query comprises at least one composite key that expresses parent-child relationship of a header and at least one sub-header present in the nested table.

13. A computer-program product tangibly embodied in one or more non-transitory computer-readable storage media including instructions configured to cause one or more data processors to perform a method including:

extracting, by using a table extraction model from an input document image, a plurality of tables as a plurality of objects, the plurality of tables including a plurality of cells, respectively;

receiving, as an output the table extraction model, the plurality of tables in an extensible Markup Language (XML) format and outputting cell level information of the plurality of cells;

filtering the plurality of tables based on a plurality of keywords and a condition comprising one from among AND and OR, to reduce a number of the plurality of tables, the filtering further including:

for each of the plurality of keywords, traversing each of the plurality of tables along a row direction and a column direction using one or more sub-keywords, each of the one or more sub-keywords being composed of one or more words of the keyword,

based on the traversing:

for each of the plurality of tables, determining whether a table comprises a first keyword from the plurality of keywords and determining whether the table comprises a second keyword from the plurality of keywords,

for the condition being AND, based on the table containing the first keyword and the second keyword, outputting the table, as a filtered table among filtered tables, and, based on the table not containing the first keyword and the second keyword, not outputting the table, and

for the condition being OR, based on the table containing the first keyword or the second keyword, outputting the table, as a filtered table among the filtered tables;

outputting the filtered tables that include the at least one from among the first keyword and the second keyword based on the condition, among the plurality of tables;

executing a query on the filtered tables, the query including one or more keys composed based on the cell level information associated with the filtered tables;

identifying one or more cells among the plurality of cells based on the one or more keys; and

providing target content that corresponds to the one or more cells, wherein the target content is a response to the query.

14. The computer-program product of claim 13 , wherein the method further includes:

obtaining a set of conditions, the set of conditions comprising AND and OR, prior to the filtering; and

obtaining the condition for the filtering from the set of conditions.

15. The computer-program product of claim 13 , wherein the filtering further includes:

initializing a vector comprising an array having a number of elements equal to a number of the plurality of keywords;

based on the table containing the first keyword, generating a value of 1 as a first element of the array;

based on the table containing the second keyword, generating a value of 1 as a second element of the array;

based on the condition being AND, outputting the table if the first element and the second element have the value of 1; and

based on the condition being OR, outputting the table if at least one from among the first element and the second element has the value of 1.

16. The computer-program product of claim 13 , wherein the executing the query further includes:

obtaining the cell level information associated with each of the filtered tables, the cell level information comprising, for each of the filtered tables, coordinates of cells and table architecture; and

deriving grammar and rules specific to each of the filtered tables, using the coordinates of cells and the table architecture.

17. The computer-program product of claim 16 , wherein:

the executing the query further includes creating the query based on the grammar and rules specific to each of the filtered tables,

at least one of the filtered tables is a nested table, and

the query comprises at least one composite key that expresses parent-child relationship of a header and at least one sub-header present in the nested table.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2023
From: AGRAHARI, SHIVAM; SINGARAJU, DAKSHAYANI; JAIN, VERESH
To: ORACLE FINANCIAL SERVICES SOFTWARE LIMITED
Reel/Frame 065323/0398 →
Continuity (2)
Continuation 18174076 · Feb 24, 2023
Related Publication 20240290123A1 · Aug 29, 2024
References Cited (43)
US 4787035A · Bourne · 1988 [cited by applicant]
US 5995918A · Kendall et al. · 1999 [cited by applicant]
US 8443278B2 · Mansfield et al. · 2013 [cited by applicant]
US 8812537B1 · Kapoor · 2014 [cited by applicant]
US 8996568B2 · Swaminathan · 2015 [cited by examiner]
US 9940365B2 · Zoryn et al. · 2018 [cited by applicant]
US 10706218B2 · Milward et al. · 2020 [cited by applicant]
US 10769503B1 · Buhler et al. · 2020 [cited by applicant]
US 11010543B1 · Wesanekar et al. · 2021 [cited by applicant]
US 11194797B2 · Canim et al. · 2021 [cited by applicant]
US 11380116B2 · Zhong et al. · 2022 [cited by applicant]
US 11443416B2 · Liao et al. · 2022 [cited by applicant]
US 20010047372A1 · Gorelik et al. · 2001 [cited by applicant]
US 20040268235A1 · Wason · 2004 [cited by applicant]
US 20100299367A1 · Chakrabarti · 2010 [cited by examiner]
US 20110164813A1 · Enomoto · 2011 [cited by applicant]
US 20150134633A1 · Colgrove · 2015 [cited by examiner]
US 20160012052A1 · Zoryn et al. · 2016 [cited by applicant]
US 20170220858A1 · Stitz et al. · 2017 [cited by applicant]
US 20170329749A1 · Milward et al. · 2017 [cited by applicant]
US 20210117668A1 · Zhong et al. · 2021 [cited by applicant]
US 20210201013A1 · Makhija et al. · 2021 [cited by applicant]
US 20210390294A1 · Huang et al. · 2021 [cited by applicant]
US 20220044011A1 · Shanmugasundaram et al. · 2022 [cited by applicant]
US 20220067014A1 · Yu et al. · 2022 [cited by applicant]
US 20220284215A1 · Rastogi et al. · 2022 [cited by applicant]
US 20220318240A1 · Wei · 2022 [cited by applicant]
US 20220319217A1 · Paliwal et al. · 2022 [cited by applicant]
US 20230095673A1 · Dharmasiri et al. · 2023 [cited by applicant]
US 20230153641A1 · Manda et al. · 2023 [cited by applicant]
“Document Layout Analysis”, Available at https://learn.microsoft.com/en-us/azure/applied-ai-services/form-recognizer/concept-layout?view=form-recog-3.0.0, Oct. 27, 2022, pp. 1-11. [cited by applicant]
“How can I Extract Table Rows from a Document?”, Available online at https://support.docparser.com/article/1293-how-can-i-extract-table-rows-from-a-document, Oct. 11, 2021, 3 pages. [cited by applicant]
“Powerful Data Capture And Workflow Automation Features From Docparser”, Available online at https://docparser.com/features/, Accessed from Internet on Dec. 19, 2022, pp. 1-2. [cited by applicant]
“Your Zero-Coding Solution for Blazing Fast Data Processing”, Available online at https://docparser.com/, Accessed from Internet on Dec. 19, 2022, 2 pages. [cited by applicant]
Gilani et al., “Table Detection Using Deep Learning”, International Conference on Document Analysis and Recognition (ICDAR), Nov. 9-15, 2017, 6 pages. [cited by applicant]
Perez-Arriaga et al., “TAO: System for Table Detection and Extraction from PDF Documents”, Proceedings of the Twenty-Ninth International Florida Artificial Intelligence Research Society Conference, Mar. 30, 2016, pp. 59… [cited by applicant]
Prasad et al., “CascadeTabNet an Approach for End to End Table Detection and Structure Recognition from Image-based Documents”, Available online at https://arxiv.org/ftp/arxiv/papers/2004/2004.12629.pdf, Jun. 2020, 10 p… [cited by applicant]
Qasim et al., “Rethinking Table Recognition Using Graph Neural Networks”, Available Online at: https://arxiv.org/pdf/1905.13391.pdf, Jul. 3, 2019, 6 pages. [cited by applicant]
Sun et al., “Enhanced Table Extraction from Documents with form Recognizer”, Available online at https://techcommunity.microsoft.com/t5/ai-cognitive-services-blog/enhanced-table-extraction-from-documents-with-form-recog… [cited by applicant]
Wei et al., “Table Extraction for Answer Retrieval”, Information Retrieval Journal, vol. 9, No. 5, Nov. 2006, pp. 1-26. [cited by applicant]
“Excel 2016—Filtering Data”, GCFGlobal , Available Online at:https://web.archive.org/web/20220120111922/https://edu.gcfglobal.org/en/excel2016/filtering-data/1/, Jan. 20, 2022, 15 pages. [cited by applicant]
Mair et al., “Using Computer Vision to Identify and Extract Tables from Various File Formats”, GenInvo, Available Online at:https://www.lexjansen.com/phuse-us/2021/ml/PRE_ML06.pdf, Aug. 18, 2021, 22 pages. [cited by applicant]
ACAMPORA , “How to Add a Search Box to a Slicer to Quicklyfilter Pivot Tables and Charts + Video”, Excel Campus, Available Online at:https://www.excelcampus.com/pivot-tables/add-search-box-to-slicer/, Feb. 16, 2016, 17 … [cited by applicant]