System and method for automated document generation and search
A semantic document generation and search system is described. The semantic document extraction system generates a knowledge graph representing a collection of documents, each document being represented as a sub-graph of the knowledge graph being linked to each other by common terms of a plurality of document terms. The system extracts a first filter criterion based on the plurality of terms of the sub-graphs representing the collection of documents, receives a first search value for the first filter criterion, and identifies a subset of sub-graphs, of the knowledge graph, that include a term corresponding to the first filter criterion and having a term value corresponding to the first search value. The system prunes the knowledge graph to include only the identified subset of sub-graphs, and extracts and outputs a subset of the collection of documents corresponding to the subset of sub-graphs included in the pruned knowledge graph.
1. A document extraction system comprising:
at least one memory configured to a store a program; and
at least one processor communicatively connected to the at least one memory and configured to execute the stored program to:
for each document of a collection of documents stored in a document database:
determine a type of the document, the type of the document defining a plurality of rules that govern terms of the document;
acquire a plurality of terms from a plurality of sources including at least a first set of terms of the document from a first source and a second set of terms of the document from a second source, wherein the first source is different from the second source;
convert the first set of terms into a first plurality of triples;
convert the second set of terms into a second plurality of triples;
generate a knowledge graph for the document, each knowledge graph having a plurality of nodes corresponding to the plurality of terms of the document, by applying the plurality of rules that govern the plurality of terms to the first plurality of triples and the second plurality of triples, wherein the first set of terms is determined to be compatible with the second set of terms in a case where the plurality of rules that govern the terms of the document is satisfied; and
update the generated knowledge graph for the document, in a case where at least one or more first terms of the first set of terms is non-compatible with one or more second terms of the second set of terms, to reconcile the non-compatible terms of the document until all terms of the document are compatible with each other;
generate a collection of knowledge graphs representing the collection of documents by associating the knowledge graph for each document of the collection of documents with other knowledge graphs for other documents of the collection of documents by nodes representing common terms of each plurality of terms of each document;
extract a first filter criterion based on a combined plurality of terms associated with the plurality of nodes of the collection of knowledge graphs representing the collection of documents;
receive a first search value for the first filter criterion;
identify a first subset of knowledge graphs, of the collection of knowledge graphs, that include a node representing a term corresponding to the first filter criterion and having a term value corresponding to the first search value;
prune the collection of knowledge graphs to include only the identified first subset of knowledge graphs; and
extract and output a subset of the collection of documents corresponding to the identified first subset of knowledge graphs included in the pruned collection of knowledge graphs.
2. The system according to claim 1 , wherein the at least one processor is further configured to execute the stored program to extract the first filter criterion by:
identifying a plurality of values associated with each term of the plurality of terms of the collection of the knowledge graphs; and
extracting the plurality of values for at least one of the plurality of terms as the first filter criterion.
3. The system according to claim 1 , wherein the at least one processor is further configured to execute the stored program to:
extract a second filter criterion based on the plurality of terms of the collection of the knowledge graphs representing the collection of documents;
receive a second search value for the second filter criterion;
identify a second subset of knowledge graphs, of the pruned collection of knowledge graphs, that include a second node representing a second term corresponding to the second filter criterion and having a second term value corresponding to the second search value;
further prune the collection of knowledge graphs to include only the identified second subset of knowledge graphs; and
extract and output a second subset of the collection of documents corresponding to the second subset of knowledge graphs included in the further pruned collection of knowledge graphs.
4. The system according to claim 1 , wherein the first filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.
5. The system according to claim 4 , wherein the second filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.
6. A method of extracting a document, the method executable by a programmed processor, the method comprising:
for each document of a collection of documents stored in a document database:
determining a type of the document, the type of the document defining a plurality of rules that govern terms of the document;
acquiring a plurality of terms from a plurality of sources including at least a first set of terms of the document from a first source and a second set of terms of the document from a second source, wherein the first source is different from the second source;
converting the first set of terms into a first plurality of triples;
converting the second set of terms into a second plurality of triples;
generating a knowledge graph for the document, each knowledge graph having a plurality of nodes corresponding to the plurality of terms of the document, by applying the plurality of rules that govern the plurality of terms to the first plurality of triples and the second plurality of triples, wherein the first set of terms is determined to be compatible with the second set of terms in a case where the plurality of rules that govern the terms of the document is satisfied; and
updating the generated knowledge graph for the document, in a case where at least one or more first terms of the first set of terms is non-compatible with one or more second terms of the second set of terms, to reconcile the non-compatible terms of the document until all terms of the document are compatible with each other;
generating a collection of knowledge graphs representing the collection of documents by associating the knowledge graph for each document of the collection of documents with other knowledge graphs for other documents of the collection of documents by nodes representing common terms of each plurality of terms of each document;
extracting a first filter criterion based on a combined plurality of terms associated with the plurality of nodes of the collection of knowledge graphs representing the collection of documents;
receiving a first search value for the first filter criterion;
identifying a first subset of knowledge graphs, of the collection of knowledge graphs, that include a node representing a term corresponding to the first filter criterion and having a term value corresponding to the first search value;
pruning the collection of knowledge graphs to include only the identified first subset of knowledge graphs; and
extracting and outputting a subset of the collection of documents corresponding to the identified first subset of knowledge graphs included in the pruned collection of knowledge graphs.
7. The method according to claim 6 , further comprising:
identifying a plurality of values associated with each term of the plurality of terms of the collection of the knowledge graphs; and
extracting the plurality of values for at least one of the plurality of terms as the first filter criterion.
8. The method according to claim 6 , further comprising:
extracting a second filter criterion based on the plurality of terms of the collection of the knowledge graphs representing the collection of documents;
receiving a second search value for the second filter criterion;
identifying a second subset of knowledge graphs, of the pruned collection of knowledge graphs, that include a second node representing a second term corresponding to the second filter criterion and having a second term value corresponding to the second search value;
further pruning the collection of knowledge graphs to include only the identified second subset of knowledge graphs; and
extracting and outputting a second subset of the collection of documents corresponding to the second subset of knowledge graphs included in the further pruned collection of knowledge graphs.
9. The method according to claim 6 , wherein the first filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.
10. The method according to claim 6 , wherein the second filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.
11. A non-transitory computer readable storage medium configured to store a program that causes a programmed processor to execute a method for extracting a document, the method comprising:
for each document of a collection of documents stored in a document database:
determining a type of the document, the type of the document defining a plurality of rules that govern terms of the document;
acquiring a plurality of terms from a plurality of sources including at least a first set of terms of the document from a first source and a second set of terms of the document from a second source, wherein the first source is different from the second source;
converting the first set of terms into a first plurality of triples;
converting the second set of terms into a second plurality of triples;
generating a knowledge graph for the document, each knowledge graph having a plurality of nodes corresponding to the plurality of terms of the document, by applying the plurality of rules that govern the plurality of terms to the first plurality of triples and the second plurality of triples, wherein the first set of terms is determined to be compatible with the second set of terms in a case where the plurality of rules that govern the terms of the document is satisfied; and
updating the generated knowledge graph for the document, in a case where at least one or more first terms of the first set of terms is non-compatible with one or more second terms of the second set of terms, to reconcile the non-compatible terms of the document until all terms of the document are compatible with each other;
generating a collection of knowledge graphs representing the collection of documents by associating the knowledge graph for each document of the collection of documents with other knowledge graphs for other documents of the collection of documents by nodes representing common terms of each plurality of terms of each document;
extracting a first filter criterion based on a combined plurality of terms associated with the plurality of nodes of the collection of knowledge graphs representing the collection of documents;
receiving a first search value for the first filter criterion;
identifying a first subset of knowledge graphs, of the collection of knowledge graphs, that include a node representing a term corresponding to the first filter criterion and having a term value corresponding to the first search value;
pruning the collection of knowledge graphs to include only the identified first subset of knowledge graphs; and
extracting and outputting a subset of the collection of documents corresponding to the identified first subset of knowledge graphs included in the pruned collection of knowledge graphs.
12. The storage medium according to claim 11 , wherein the method further comprises:
identifying a plurality of values associated with each term of the plurality of terms of the collection of the knowledge graphs; and
extracting the plurality of values for at least one of the plurality of terms as the first filter criterion.
13. The storage medium according to claim 11 , wherein the method further comprises:
extracting a second filter criterion based on the plurality of terms of the collection of the knowledge graphs representing the collection of documents;
receiving a second search value for the second filter criterion;
identifying a second subset of knowledge graphs, of the pruned collection of knowledge graphs, that include a second node representing a second term corresponding to the second filter criterion and having a second term value corresponding to the second search value;
further pruning the collection of knowledge graphs to include only the identified second subset of knowledge graphs; and
extracting and outputting a second subset of the collection of documents corresponding to the second subset of knowledge graphs included in the further pruned collection of knowledge graphs.
14. The storage medium according to claim 11 , wherein the first filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.
15. The storage medium according to claim 11 , wherein the second filter criterion includes at least one of contract people, contract meta data, contract type, and contract term.