Knowledge graph management based on multi-source data
Techniques for constructing and otherwise managing knowledge graphs in information processing system environments are disclosed. For example, a method comprises the following steps. The method collects data from a plurality of data sources. The method extracts structured data and unstructured data from the collected data, wherein unstructured data is extracted using an unsupervised machine learning process. The method forms a plurality of sub-graph structures comprising a sub-graph structure for each of the data sources based on at least a portion of the extracted structured data and unstructured data. The method combines the plurality of sub-graph structures to form a combined graph structure representing the collected data from the plurality of data sources. The resulting combined graph structure is a comprehensive knowledge graph.
1. An apparatus comprising:
at least one processing platform comprising at least one processor coupled to at least one memory, the at least one processing platform, when executing program code, is configured to:
collect data from a plurality of data sources;
extract structured data and unstructured data from the collected data using an unsupervised machine learning model, wherein the extracting comprises:
selecting and processing a portion of the unstructured data using one or more unstructured data filters based on one or more natural language processing metrics;
selecting a portion of the structured data using one or more structured data filters; and
applying the unsupervised machine learning model to the selected and processed portion of the unstructured data to determine one or more topics of the unstructured data;
form a plurality of sub-graph structures comprising a sub-graph structure for each of the data sources based on at least the selected portion of the structured data and the selected and processed portion of the unstructured data;
compute, for a given node of a given sub-graph structure, a title embedding based on a title portion of the selected portion of structured data and a topic embedding based on the one or more topics of the unstructured data and one or more other portions of the selected portion of the structured data;
combine the plurality of sub-graph structures to form a combined graph structure representing the collected data from the plurality of data sources, wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises:
using a graph and report generator to measure a distance between two nodes and adding an edge between the two nodes when the distance is at or below a given distance threshold value; and
apply the combined graph structure to one or more user tasks.
2. The apparatus of claim 1 , wherein extracting structured data from the collected data further comprises identifying one or more titles of content in the collected data for the plurality of data sources.
3. The apparatus of claim 1 , wherein the extracting further comprises saving the selected portion of structured data and the determined one or more topics to a graphical database to form the plurality of sub-graph structures.
4. The apparatus of claim 1 , wherein the unsupervised machine learning model comprises a Latent Dirichlet Allocation model.
5. The apparatus of claim 1 , wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises labelling the added edge based on the sources of the nodes.
6. The apparatus of claim 1 , wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises concatenating the title embedding and the topic embedding to obtain a final node embedding for the given node.
7. The apparatus of claim 6 , wherein when a given one of the determined one or more topics is extracted via a topic model, the topic embedding is derived from a weighted version of a topic word embedding.
8. The apparatus of claim 7 , wherein when the given one of the determined one or more topics is extracted directly from the collected data, the topic embedding is derived from an averaged version of the topic word embedding.
9. A method comprising:
collecting data from a plurality of data sources;
extracting structured data and unstructured data from the collected data using an unsupervised machine learning model, wherein the extracting comprises:
selecting and processing a portion of the unstructured data using one or more unstructured data filters based on one or more natural language processing metrics;
selecting a portion of the structured data using one or more structured data filters; and
applying the unsupervised machine learning model to the selected and processed portion of the unstructured data to determine one or more topics of the unstructured data;
forming a plurality of sub-graph structures comprising a sub-graph structure for each of the data sources based on at least the selected portion of the structured data and the selected and processed portion of the unstructured data;
computing, for a given node of a given sub-graph structure, a title embedding based on a title portion of the selected portion of structured data and a topic embedding based on the one or more topics of the unstructured data and one or more other portions of the selected portion of the structured data;
combining the plurality of sub-graph structures to form a combined graph structure representing the collected data from the plurality of data sources, wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises:
using a graph and report generator to measure a distance between two nodes and adding an edge between the two nodes when the distance is at or below a given distance threshold value; and
applying the combined graph structure to one or more user tasks;
wherein the method is executed by at least one processing platform comprising at least one processor coupled to at least one memory configured to execute program code.
10. The method of claim 9 , wherein the extracting further comprises saving the selected portion of structured data and the determined one or more topics to a graphical database to form the plurality of sub-graph structures.
11. The method of claim 9 , wherein the unsupervised machine learning model comprises a Latent Dirichlet Allocation model.
12. The method of claim 9 , wherein extracting structured data from the collected data further comprises identifying one or more titles of content in the collected data for the plurality of data sources.
13. The method of claim 9 , wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises concatenating the title embedding and the topic embedding to obtain a final node embedding for the given node.
14. The method of claim 13 , wherein when a given one of the determined one or more topics is extracted via a topic model, the topic embedding is derived from a weighted version of a topic word embedding, and further wherein when the given one of the determined one or more topics is extracted directly from the collected data, the topic embedding is derived from an averaged version of the topic word embedding.
15. The method of claim 13 , wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises:
labelling the added edge based on the sources of the nodes.
16. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing platform causes the at least one processing platform to:
collect data from a plurality of data sources;
extract structured data and unstructured data from the collected data using an unsupervised machine learning model, wherein the extracting comprises:
selecting and processing a portion of the unstructured data using one or more unstructured data filters based on one or more natural language processing metrics;
selecting a portion of the structured data using one or more structured data filters; and
applying the unsupervised machine learning model to the selected and processed portion of the unstructured data to determine one or more topics of the unstructured data;
form a plurality of sub-graph structures comprising a sub-graph structure for each of the data sources based on at least the selected portion of the structured data and the selected and processed portion of the unstructured data;
compute, for a given node of a given sub-graph structure, a title embedding based on a title portion of the selected portion of structured data and a topic embedding based on the one or more topics of the unstructured data and one or more other portions of the selected portion of the structured data;
combine the plurality of sub-graph structures to form a combined graph structure representing the collected data from the plurality of data sources, wherein combining the plurality of sub-graph structures to form a combined graph structure further comprises:
using a graph and report generator to measure a distance between two nodes and adding an edge between the two nodes when the distance is at or below a given distance threshold value; and
apply the combined graph structure to one or more user tasks.
17. The computer program product of claim 16 , wherein the combined graph structure representing the collected data from the plurality of data sources is a knowledge graph.
18. The computer program product of claim 16 , wherein extracting structured data from the collected data further comprises identifying one or more titles of content in the collected data for the plurality of data sources.
19. The computer program product of claim 16 , wherein the extracting further comprises saving the selected portion of structured data and the determined one or more topics to a graphical database to form the plurality of sub-graph structures.
20. The computer program product of claim 16 , wherein the combining the plurality of sub-graph structures to form a combined graph structure further comprises labelling the added edge based on the sources of the nodes.