HYBRID ARCHITECTURE FOR PROCESSING GRAPH-BASED QUERIES
The disclosed embodiments provide a system for processing data. During operation, the system launches a set of child processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. When a query of the graph database is received, the system transmits the query to one or more of the child processes. Next, the system receives a result of the query from the one or more child processes. The system then provides the result in a response to the query.
1 . A method, comprising:
launching, by a management process executing on a computer system, a set of child processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates;
when a query of the graph database is received, processing the query at the management process by:
transmitting the query to one or more of the child processes;
receiving a result of the query from the one or more child processes; and
providing the result in a response to the query.
2 . The method of claim 1 , further comprising:
monitoring an execution condition associated with a child process in the set of child processes; and
managing, by the management process, execution of the child process based on the monitored execution condition.
3 . The method of claim 2 , wherein:
the execution condition comprises an execution error in the child process, and
managing execution of the child process comprises:
restarting the child process when the execution error is recoverable; and
terminating the management process and the set of child processes when the execution error is unrecoverable.
4 . The method of claim 2 , wherein:
the execution condition comprises a resource usage of the child process, and
managing execution of the child process comprises terminating the child process when the resource usage exceeds a resource usage limit for the child process.
5 . The method of claim 4 , wherein the resource usage is associated with at least one of:
a processor;
a memory;
a disk input/output (I/O); and
a processing time.
6 . The method of claim 1 , wherein receiving the result of the query from the one or more child processes comprises:
receiving, from the one or more child processes, an indication that the result is not available from the one or more child processes;
transmitting the query to one or more additional management processes executing on one or more additional computer systems;
receiving one or more subsets of the results from the one or more additional management processes;
providing the one or more subsets of the results to the one or more child processes for aggregating the one or more subsets into the result; and
receiving the aggregated result from the one or more child processes.
7 . The method of claim 1 , wherein transmitting the query to the one or more child processes comprises:
identifying a size of the query; and
transmitting the query to a child process with a resource usage limit that accommodates the size.
8 . The method of claim 1 , wherein the set of child processes comprise:
a write process that executes write requests to the graph database; and
one or more read processes that execute read requests to the graph database.
9 . The method of claim 1 , wherein:
the management process executes in a first runtime environment, and the child processes execute in a second runtime environment that is separate from the first runtime environment.
10 . The method of claim 9 , wherein the second runtime environment has a higher performance than the first runtime environment.
11 . The method of claim 9 , wherein the first runtime environment has a higher stability than the second runtime environment.
12 . An apparatus, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
launch a set of child processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates;
when a query of the graph database is received:
transmit the query to one or more of the child processes;
receive a result of the query from the one or more child processes; and
provide the result in a response to the query.
13 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
execute, by the one or more child processes, the query against the graph database.
14 . The apparatus of claim 12 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
monitor an execution condition associated with a child process in the set of child processes; and
manage execution of the child process based on the monitored execution condition.
15 . The apparatus of claim 14 , wherein:
the execution condition comprises an execution error in the child process, and
managing execution of the child process comprises:
restarting the child process when the execution error is recoverable; and
terminating the set of child processes when the execution error is unrecoverable
16 . The apparatus of claim 14 , wherein:
the execution condition comprises a resource usage of the child process, and
managing execution of the child process comprises terminating the child process when the resource usage exceeds a resource usage limit for the child process.
17 . The apparatus of claim 12 , wherein receiving the result of the query from the one or more child processes comprises:
receiving, from the one or more child processes, an indication that the result is not available from the one or more child processes;
transmitting the query to on one or more computer systems; and
receiving one or more subsets of the results from the one or more computer systems.
18 . The apparatus of claim 17 , wherein receiving the result of the query from the one or more child processes further comprises:
providing the one or more subsets of the results to the one or more child processes for aggregating the one or more subsets into the result; and
receiving the aggregated result from the one or more child processes.
19 . A system, comprising:
a management module comprising a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the system to:
launch a set of child processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates;
when a query of the graph database is received:
transmit the query to one or more of the child processes;
receive a result of the query from the one or more child processes; and
provide the result in a response to the query; and
a processing module comprising a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the system to execute, by the one or more child processes, the query against the graph database.
20 . The system of claim 19 , wherein the non-transitory computer-readable medium of the management module further comprises instructions that, when executed by the one or more processors, cause the system to:
monitor an execution condition associated with a child process in the set of child processes; and
manage execution of the child process based on the monitored execution condition.