Distributed storage system with pluggable query processing
View Patent ↗As part of a query-processing technique, in response to receiving queries for information stored in a distributed storage system and associated query-processing information identifying sets of predefined operations to perform on one or more databases in the distributed storage system, the distributed storage system accesses sets of predefined operations. Then, for the sets of predefined operations on the one or more databases, the distributed storage system uses one or more adaptors for multiple storage subsystems in the distributed storage system and one or more indexes corresponding to collections of information in the one or more databases. Moreover, the distributed storage system executes the sets of predefined operations on the multiple storage subsystems to obtain results for the queries. The sets of predefined operations may be defined by different users, so that the distributed storage system can be adapted to these users' needs.
1. A computer-system-implemented method for providing a result for a first query and a second query of one or more databases, the method comprising:
receiving, at a distributed storage system comprising the one or more databases:
the first query and first query-processing information identifying a first set of predefined operations; and
the second query and second query-processing information identifying a second set of predefined operations;
wherein the distributed storage system further comprises:
adaptors for multiple storage subsystems in the distributed storage system that store the one or more databases; and
multiple indexes, wherein each of the indexes corresponds to a different collection of information stored in the one or more databases;
using the computer system, performing the first set of predefined operations and the second set of predefined operations by:
using one or more of the adaptors and one or more of the indexes for the first set of predefined operations and the second set of predefined operations; and
executing the first set of predefined operations and the second set of predefined operations on the multiple storage subsystems in the distributed storage system; and
providing the result of the first set of predefined operations and the second set of predefined operations.
2. The method of claim 1 , wherein the one or more databases include different types of databases from different providers having different schemas.
3. The method of claim 2 , wherein performing the first set of predefined operations and the second set of predefined operations involves translating the first set of predefined operations and the second set of predefined operations to the different schemas associated with the different types of databases.
4. The method of claim 1 , wherein:
the multiple indexes include a first index coupled to a second index;
the first index is updated continuously as data in the one or more databases is changed; and
the second index is updated based on the first index after a time interval.
5. The method of claim 1 , wherein the first set of predefined operations and the second set of predefined operations provide user-defined functionality other than functionality supported by the structure of the one or more databases.
6. The method of claim 1 , wherein the adaptors, the multiple indexes, the first set of predefined operations and the second set of predefined operations provide dynamic configurability for the distributed storage system.
7. The method of claim 1 , the one or more databases store professional attributes of multiple individuals and their interconnections with one another as specified by a social graph.
8. An apparatus for providing a result for a first query and a second query of one or more databases, the apparatus comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
receive, at a distributed storage system comprising the one or more databases:
the first query and first query-processing information identifying a first set of predefined operations; and
the second query and second query-processing information identifying a second set of predefined operations;
wherein the distributed storage system further comprises:
adaptors for multiple storage subsystems in the distributed storage system that store the one or more databases; and
multiple indexes, wherein each of the indexes corresponds to a different collection of information stored in the one or more databases;
perform the first set of predefined operations and the second set of predefined operations by:
using one or more of the adaptors and one or more of the indexes for the first set of predefined operations and the second set of predefined operations; and
executing the first set of predefined operations and the second set of predefined operations on the multiple storage subsystems in the distributed storage system; and
provide the result of the first set of predefined operations and the second set of predefined operations.
9. The computer-program product of claim 8 , wherein the one or more databases include different types of databases from different providers having different schemas.
10. The computer-program product of claim 9 , wherein performing the first set of predefined operations and the second set of predefined operations involves translating the first set of predefined operations and the second set of predefined operations to the different schemas associated with the different types of databases.
11. The computer-program product of claim 8 , wherein:
the multiple indexes include a first index coupled to a second index;
the first index is updated continuously as data in the one or more databases is changed; and
the second index is updated based on the first index after a time interval.
12. The computer-program product of claim 8 , wherein the first set of predefined operations and the second set of predefined operations provide user-defined functionality other than functionality supported by the structure of the one or more databases.
13. The computer-program product of claim 8 , wherein the adaptors, the multiple indexes, the first set of predefined operations and the second set of predefined operations provide dynamic configurability for the distributed storage system.
14. The computer-program product of claim 8 , wherein the one or more databases store professional attributes of multiple individuals and their interconnections with one another as specified by a social graph.
15. A distributed data storage system that includes multiple storage nodes, wherein each storage node comprises:
one or more processors;
memory; and
a storage-engine program module stored in the memory and executed by the one or more processors to provide a result for a first query and a second query of one or more databases, the storage-engine program module including:
instructions for receiving, at the distributed storage system:
the first query and first query-processing information identifying a first set of predefined operations; and
the second query and second query-processing information identifying a second set of predefined operations;
wherein the distributed storage system comprises:
the one or more databases;
adaptors for multiple storage subsystems in the distributed storage system that store the one or more databases; and
multiple indexes, wherein each of the indexes corresponds to a different collection of information stored in the one or more databases;
instructions for performing the first set of predefined operations and the second set of predefined operations by:
using one or more of the adaptors and one or more of the indexes for the first set of predefined operations and the second set of predefined operations; and
executing the first set of predefined operations and the second set of predefined operations on the multiple storage subsystems in the distributed storage system; and
instructions for providing the result of the first set of predefined operations and the second set of predefined operations.
16. The computer system of claim 15 , wherein the one or more databases include different types of databases from different providers having different schemas.
17. The computer system of claim 16 , wherein performing the first set of predefined operations and the second set of predefined operations involves translating the first set of predefined operations and the second set of predefined operations to the different schemas associated with the different types of databases.
18. The computer system of claim 15 , wherein:
the multiple indexes include a first index coupled to a second index;
the first index is updated continuously as data in the one or more databases is changed; and
the second index is updated based on the first index after a time interval.
19. The computer system of claim 15 , wherein the first set of predefined operations and the second set of predefined operations provide user-defined functionality other than functionality supported by the structure of the one or more databases.
20. The computer system of claim 15 , wherein the one or more databases store professional attributes of multiple individuals and their interconnections with one another as specified by a social graph.