Data identification and extraction from unstructured documents
Aspects of the method, apparatus, non-transitory computer readable medium, and system include obtaining a document and an information element. The aspects further include identifying, from the document, an anchor element that has an anchor type and a relationship type, wherein the anchor type describes a structure of a set of anchor elements, and the relationship type describes a relationship between the anchor element and the information element. The aspects further include extracting information corresponding to the information element based on the anchor element, the anchor type, and the relationship type, and displaying the extracted information to a user.
1 . A method comprising:
obtaining a set of documents including a first document and a second document having a different format from the first document, wherein an information element is located in each document of the set of documents;
determining that a first anchor element for the information element is present in the first document and that a second anchor element different from the first anchor element is present in the second document, wherein the first anchor element has an anchor type and a relationship type, wherein the anchor type indicates whether the first anchor element can be used to locate the information element in the first document, wherein the relationship type describes a relationship between the first anchor element and the information element and is selected from a predetermined set of relationship types including a position type and a structural hierarchy type, wherein the position type indicates a distance or a direction between the first anchor element and the information element, and wherein the structural hierarchy type indicates a location of the information element within a document object model;
extracting information corresponding to the information element from the first document and the second document, wherein a first part of the information is extracted from the first document based on the first anchor element, the anchor type, and the relationship type, and wherein a second part of the information is extracted from the second document based on the second anchor element; and
displaying the extracted information to a user.
2 . The method of claim 1 , further comprising:
identifying a plurality of anchor candidates for locating the information element, wherein the first anchor element is selected from the plurality of anchor candidates.
3 . The method of claim 2 , further comprising:
displaying the plurality of anchor candidates to the user; and
receiving an anchor selection input from the user, wherein the first anchor element is selected based on the anchor selection input.
4 . The method of claim 1 , wherein:
the anchor type comprises a single anchor type, a multiple anchor type, a hierarchical anchor type, a self-anchor type, or a combination anchor type.
5 . The method of claim 1 , wherein:
the predetermined set of relationship types further comprises a style type or a semantic similarity type.
6 . The method of claim 1 , further comprising:
locating the information element in the first document based on the first anchor element, the anchor type, and the relationship type, wherein the information is extracted from the information element.
7 . The method of claim 6 , further comprising:
determining that the relationship type comprises the position type; and
identifying a position relationship between the information element and the first anchor element, wherein the information element is located based on the position relationship.
8 . The method of claim 6 , further comprising:
determining that the relationship type comprises the structural hierarchy type; and
identifying a structural relationship between the information element and the first anchor element, wherein the information element is located based on the structural relationship.
9 . The method of claim 6 , further comprising:
determining that the relationship type comprises a style type; and
identifying a style relationship between the information element and the first anchor element, wherein the information element is located based on the style relationship.
10 . The method of claim 6 , further comprising:
determining that the relationship type comprises a semantic type; and
identifying a semantic relationship between the information element and the first anchor element, wherein the information element is located based on the semantic relationship.
11 . The method of claim 1 , further comprising:
determining that the anchor type comprises a multiple anchor type; and
identifying an additional anchor element, wherein a set of anchor elements includes the first anchor element and the additional anchor element, and the information is extracted based on the first anchor element and the additional anchor element.
12 . The method of claim 1 , further comprising:
determining that the anchor type comprises a hierarchical anchor type; and
identifying an additional anchor element having a hierarchical relationship to the first anchor element, wherein a set of anchor elements includes the first anchor element and the additional anchor element, and the information is extracted based on the first anchor element and the additional anchor element.
13 . The method of claim 1 , further comprising:
providing the first document and the information element to a machine learning model, wherein the first anchor element is identified based on an output of the machine learning model.
14 . A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
identifying a plurality of documents including a first document and a second document having a different format from the first document, wherein an information element is located in each document of the plurality of documents;
generating a set of anchor elements associated with the information element using a machine learning model trained to determine that a first anchor element for the information element is present in the first document and that a second anchor element different from the first anchor element is present in the second document, wherein the first anchor element has an anchor type and a relationship type, wherein the anchor type indicates whether the first anchor element can be used to locate the information element in the first document, wherein the relationship type describes a relationship between the first anchor element and the information element and is selected from a predetermined set of relationship types including a position type and a structural hierarchy type, wherein the position type indicates a distance or a direction between the first anchor element and the information element, and wherein the structural hierarchy type indicates a location of the information element within a document object model;
extracting information corresponding to the information element from the first document and the second document, wherein a first part of the information is extracted from the first document based on the first anchor element, the anchor type, and the relationship type, and wherein a second part of the information is extracted from the second document based on the second anchor element; and
generating data from the plurality of documents based on the set of anchor elements, wherein the data includes attributes corresponding to the information element.
15 . The non-transitory computer-readable medium of claim 14 , wherein the executable instructions further cause the processing device to perform operations comprising:
clustering the plurality of documents to obtain a document cluster, wherein the data is generated based on the document cluster.
16 . The non-transitory computer-readable medium of claim 14 , wherein:
the plurality of documents includes unstructured documents.
17 . The non-transitory computer-readable medium of claim 14 , wherein the executable instructions further cause the processing device to perform operations comprising:
receiving a query from a user;
analyzing the query using a natural language processor; and
generating a response to the query based on the data.
18 . A system, comprising:
a memory component; and
one or more processing devices coupled to the memory component, the one or more processing devices configured to perform operations comprising:
obtaining a set of documents including a first document and a second document having a different format from the first document, wherein an information element is located in each document of the set of documents;
determining, using an anchor component, that a first anchor element for the information element is present in the first document and that a second anchor element different from the first anchor element is present in the second document, wherein the first anchor element has an anchor type and a relationship type, wherein the anchor type indicates whether the first anchor element can be used to locate the information element in the first document, wherein the relationship type describes a relationship between the first anchor element and the information element and is selected from a predetermined set of relationship types including a position type and a structural hierarchy type, wherein the position type indicates a distance or a direction between the first anchor element and the information element, and wherein the structural hierarchy type indicates a location of the information element within a document object model;
extracting, using an extraction component, information corresponding to the information element from the first document and the second document, wherein a first part of the information is extracted from the first document based on the first anchor element, the anchor type, and the relationship type, and wherein a second part of the information is extracted from the second document based on the second anchor element; and
displaying, via a user interface, the extracted information to a user.
19 . The system of claim 18 , further comprising:
a natural language model configured to process queries related to the first document.
20 . The system of claim 18 , wherein:
the anchor component comprises a machine learning model trained to identify anchor elements from an unstructured document.