Multi-tenant knowledge graph databases with dynamic specification and enforcement of ontological data models
A system and method for cybersecurity analysis utilizing high-performance, scalable, multi-tenant, dynamically specifiable, knowledge graph information storage and utilization. The system uses an in-memory associative array for high-performance graph storage and access, with a non-volatile distributed database for scalable backup storage, a scalable, distributed graph service for graph creation, an indexing search engine to increase searching performance, and a graph crawler for graph traversal. One or more of these components may be in the form of a cloud-based service, and in some embodiments the cloud-based services may be containerized to allow for multi-tenant co-existence with no possibility of data leakage or cross-over. The system uses a cyber-physical graph to represent an enterprise's cyber-physical system and can provide graph analysis, graph security, and graph fusion related tasks to identify potential cybersecurity threats.
1. A system for cybersecurity analysis utilizing high-performance, dynamically-specifiable, knowledge graph-based information storage and utilization, comprising:
a computing device comprising a processor and a memory;
an in-memory associative array comprising key-value pairs stored in the memory of the computing device and configured to represent a knowledge graph;
an integration platform comprising a first plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the first plurality of programming instructions, when operating on the processor, cause the computing device to:
compile a received schema into data types, each data type comprising an entity definition, one or more link definitions, and one or more attribute definitions;
receive one or more data clumps, each data clump comprising entity, link, and attribute data;
for each received data clump, determine whether the respective data clump complies with the schema;
send each compliant data clump to the knowledge graph engine for assembly into a new knowledge graph that is fully compliant with the schema; and
the knowledge graph engine comprising a second plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the second plurality of programming instructions, when operating on the processor, cause the computing device to:
instantiate the knowledge graph in the in-memory associative array;
receive data clumps from the integration platform wherein the data clumps are known to be fully compliant with the schema; and
populate the knowledge graph in the in-memory associative array with the entity, link, and attribute data in each fully-compliant data clump to generate a new fully-compliant knowledge graph.
2. The system of claim 1 , further comprising a graph access control module comprising a third plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the third plurality of programming instructions, when operating on the processor, cause the computing device to assign access rights to a user, the access rights allowing the user to interact with a at least one node, and its associated edges, of the knowledge graph.
3. The system of claim 1 , wherein the integration platform is further configured to:
retrieve one or more known schemas from a database;
apply a known schema to the one or more data clumps;
identify any errors in the application of the known schema to the data clumps and compute an error rate based on any identified errors;
wherein if the error rate is below a predetermined threshold value the known schema is added to a list; and
display the list to a user, wherein the user can optionally select a known schema from a plurality of known schemas on the list.
4. The system of claim 1 , further comprising a graph manager comprising a fourth plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the fourth plurality of programming instructions, when operating on the processor, cause the computing device to:
receive a network event, the network event related to a cybersecurity event;
apply a known schema to create a constrained knowledge graph;
analyze the constrained knowledge graph to identify a cybersecurity threat;
generate one or more subgraphs from the constrained knowledge graph based on the identified cybersecurity threat, wherein each subgraph maps the identified cybersecurity threat to a threat scenario;
perform graph fusion on the subgraphs to form a fused knowledge graph;
store the fused knowledge graph in a database; and
present the fused knowledge graph to a user for graph analysis.
5. The system of claim 1 , wherein the knowledge graph is a cyber-physical graph representing an enterprise's cyber-physical system.
6. A system for cybersecurity analysis utilizing high-performance, scalable, multi-tenant, dynamically specifiable, knowledge graph based information storage and utilization, comprising:
a computing device comprising a processor and a memory;
a distributed in-memory associative array comprising key-value pairs stored in the memory of a plurality of networked computing devices and configured to represent a knowledge graph;
an integration platform comprising a first plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the first plurality of programming instructions, when operating on the processor, cause the computing device to:
compile a received schema into data types, each data type comprising an entity definition, one or more link definitions, and one or more attribute definitions;
send the compiled schema to a cloud-based knowledge graph service, wherein the cloud-based knowledge graph service instantiates a knowledge graph;
retrieve the knowledge graph from the knowledge graph service and implement the knowledge graph in the distributed in-memory associative array;
receive one or more data clumps, each data clump comprising entity, link, and attribute data;
for each received data clump, determine whether each data clump complies with the schema;
send each compliant data clump to the cloud-based knowledge graph service engine for assembly into a new knowledge graph that is created to be fully compliant with the schema, wherein the cloud-based knowledge graph service:
receives data clumps from the integration platform wherein the data clumps are known to be fully compliant with the schema; and
populates the knowledge graph in the distributed in-memory associative array with the entity, link, and attribute data from each fully-compliant data clump to generate a new fully-compliant knowledge graph; and
retrieve the populated knowledge graph from the knowledge graph service and update the distributed in-memory associative array with the populated knowledge graph.
7. The system of claim 6 , wherein the integration platform uses a cloud-based graph access control module which is configured to assign access rights to a user, the access rights allowing the user to interact with a at least one node, and its associated edges, of the knowledge graph.
8. The system of claim 6 , wherein the integration platform is further configured to:
retrieve one or more known schemas from a database;
apply a known schema to the one or more data clumps;
identify any errors in the application of the known schema to the data clumps and compute an error rate based on any identified errors;
wherein if the error rate is below a predetermined threshold value the known schema is added to a list; and
display the list to a user, wherein the user can optionally select a known schema from a plurality of known schemas on the list.
9. The system of claim 6 , wherein the integration platform uses a cloud-based graph manager which is configured to:
receive a network event, the network event related to a cybersecurity event;
apply a known schema to create a constrained knowledge graph;
analyze the constrained knowledge graph to identify a cybersecurity threat;
generate one or more subgraphs from the constrained knowledge graph based on the identified cybersecurity threat, wherein each subgraph maps the identified cybersecurity threat to a threat scenario;
perform graph fusion on the subgraphs to form a fused knowledge graph;
store the fused knowledge graph in a database; and
present the fused knowledge graph to a user for graph analysis.
10. The system of claim 6 , wherein the knowledge graph is a cyber-physical graph representing an enterprise's cyber-physical system.
11. The system of claim 6 , wherein the integration platform, the distributed in-memory associative array, and the cloud-based knowledge graph service are instantiated as services in a containerized service management application.
12. The system of claim 11 , wherein separate copies of the integration platform, the distributed in-memory associative array, and the cloud-based knowledge graph service are instantiated for each client as services in a containerized service management application.
13. A method for system for cybersecurity analysis utilizing high-performance, scalable, multi-tenant, dynamically specifiable, knowledge graph-based information storage and utilization, comprising the steps of:
instantiating a distributed in-memory associative array comprising key-value pairs stored in the memory of a plurality of networked computing devices and configured to represent a knowledge graph;
compiling a received schema into data types, each data type comprising an entity definition, one or more link definitions, and one or more attribute definitions;
sending the compiled schema to a cloud-based knowledge graph service, wherein the cloud-based knowledge graph service
instantiates a knowledge graph;
retrieving the knowledge graph from the knowledge graph service and implementing the knowledge graph in the distributed in-memory associative array;
receiving one or more data clumps, each data clump comprising entity, link, and attribute data;
determining whether each data clump complies with the schema;
enforcing compliance of each data clump by accepting compliant data clumps and rejecting non-compliant data clumps;
sending each compliant data clump to the cloud-based knowledge graph service, wherein the cloud-based knowledge graph service
populates the knowledge graph in the distributed in-memory associative array with the entity, link, and attribute data in each data clump to generate a new fully-compliant knowledge graph; and
retrieving the populated knowledge graph from the knowledge graph service and updating the distributed in-memory associative array with the populated knowledge graph.
14. The method of claim 13 , wherein a cloud-based graph access control service is used and is configured to assign access rights to a user, the access rights allowing the user to interact with a at least one node, and its associated edges, of the knowledge graph.
15. The method of claim 13 , further comprising the steps of:
retrieving one or more known schemas from a database;
applying a known schema to the one or more data clumps;
identifying any errors in the application of the known schema to the data clumps and compute an error rate based on any identified errors;
wherein if the error rate is below a predetermined threshold value the known schema is added to a list; and
displaying the list to a user, wherein the user can optionally select a known schema from a plurality of known schemas on the list.
16. The method of claim 13 , wherein a cloud-based graph manager service is used and is configured to:
receive a network event, the network event related to a cybersecurity event;
apply a known schema to create a constrained knowledge graph;
analyze the constrained knowledge graph to identify a cybersecurity threat;
generate one or more subgraphs from the constrained knowledge graph based on the identified cybersecurity threat, wherein each subgraph maps the identified cybersecurity threat to a threat scenario;
perform graph fusion on the subgraphs to form a fused knowledge graph;
store the fused knowledge graph in a database; and
present the fused knowledge graph to a user for graph analysis.
17. The method of claim 13 , wherein the knowledge graph is a cyber-physical graph representing an enterprise's cyber-physical system.
18. The method of claim 13 , wherein the distributed in-memory associative array and the cloud-based knowledge graph service are instantiated as services in a containerized service management application.
19. The method of claim 18 , wherein separate copies of the distributed in-memory associative array, and the cloud-based knowledge graph service are instantiated for each client as services in a containerized service management application.