Distributed database driver middleware
JDBC-X, which is distributed database driver middleware, can extend the JDBC API to facilitate integrating an application with a distributed database. JDBC-X functions as an intermediary between the JDBC API and the JDBC drivers and provides a number of components that adapt JDBC SQL queries and corresponding result sets in a manner that is substantially transparent to the application. In this way, an application that was originally designed to access a relational database can be seamlessly converted to accessing a distributed database. Similarly, JDBC-X allows an application to employ a distributed database that is dynamically provisioned without needing to be aware of the current structure of the database.
1. A method, performed by distributed database driver middleware that operates between a call-level application programming interface (API) and one or more database drivers, for processing a structured query language (SQL) statement that an application submits via the call-level API, the method comprising:
receiving information that defines a structure of a table that is stored in a distributed database, the information defining that the table is shard across a plurality of database nodes of the distributed database;
receiving, from the application via the call-level API, a first SQL statement that identifies the table and an operation to be performed on the table;
accessing the information that defines the structure of the table to identify which database nodes of the distributed database store a shard of the table;
generating, from the first SQL statement and based on the identification of which database nodes of the distributed database store a shard of the table, multiple separate SQL statements that each includes routing information for routing the separate SQL statement to a particular database node on which a particular shard of the table is stored, wherein generating the multiple separate SQL statements from the first SQL statement comprises generating a separate SQL statement for each of the plurality of database nodes that stores a shard of the table;
for each of the multiple separate SQL statements, submitting the separate SQL statement to a database driver that corresponds to the particular database node to thereby cause the separate SQL statement to be directed to the particular shard of the table that is stored on the particular database node;
receiving a result set from each of the plurality of database nodes,
processing the plurality of result sets to generate a single result set; and
returning the single result set to the application via the call-level API.
2. The method of claim wherein each of the multiple separate SQL statements is a SELECT statement.
3. The method of claim 1 , wherein each of the multiple separate SQL statements is an UPDATE statement.
4. The method of claim wherein each of the multiple separate SQL statements is a DELETE statement.
5. The method of claim 1 , further comprising:
receiving, from the application via the call-level API, a second SQL statement that includes an INSERT statement;
employing the information to select a first database node of the plurality of database nodes;
generating, from the second SQL statement, a single SQL statement that includes routing information to the first database node of the plurality of database nodes.
6. The method of claim 5 , wherein the first database node is selected based on a load of the plurality of database nodes.
7. The method of claim 5 , wherein the first database node is selected based on a sequence value assigned to an entry to be inserted by the INSERT statement.
8. The method of claim 5 , wherein the first database node is selected randomly.
9. The method of claim 5 , wherein the first database node is selected based on a value of a field specified in the second SQL statement.
10. The method of claim 1 , wherein processing the plurality of result sets to generate the single result set comprises one or more of merging, ordering, or filtering entries in the plurality of result sets.
11. One or more non-transitory computer storage media storing computer executable instructions defining distributed database driver middleware that operates between a call-level application programming interface (API) and one or more database drivers, the distributed database middleware including:
a database metadata store configured to store information defining a structure of a table that is stored in a distributed database including defining which database nodes of a plurality of database nodes in the distributed database store a shard of the table;
a structure query language (SQL) provider configured to:
receive, from an application via the call-level API, a first SQL statement that identifies the table and an operation to be performed on the table;
access the database metadata store to identify which database nodes of the plurality of database nodes store a shard of the table;
generate, from the first SQL statement and based on the identification of which database nodes store a shard of the table, multiple separate SQL statements that each includes routing information for routing the separate SQL statement to a particular database node on which a particular shard of the table is stored, wherein generating the multiple separate SQL statements from the first SQL statement comprises generating a separate SQL statement for each of the plurality of database nodes that store a shard of the table when the first SQL statement comprises a SELECT, UPDATE, or DELETE; and
for each of the multiple separate SQL statements, submitting the separate SQL statement to a database driver that corresponds to the particular database node to thereby cause the separate SQL statement to be directed to the particular shard of the table that is stored on the particular database node; and
a ResultSet provider configured to:
receive a result set from each of the database nodes;
process the result sets to generate a single result set; and
return the single result set to the application via the call-level API.
12. The computer storage media of claim 11 , wherein the SQL provider is further configured to:
receive, from the application via the call-level API, a second SQL statement that includes an INSERT statement;
employ the information to select a first database node of the plurality of database nodes; and
generate, from the second SQL statement, a single SQL statement that includes routing information to the first database node.
13. The computer storage media of claim 12 , wherein the routing information is selected:
randomly;
based on a load of the database nodes;
based on a sequence number associated with the INSERT operation; or
based on a value of a field in the second SQL statement.
14. The computer storage media of claim 11 , wherein the distributed database driver middleware further includes:
a database monitor configured to detect changes to the structure of the distributed database and to maintain a connection pool bucket for each database node in the distributed database;
a pool monitor for monitoring the status of each connection pool bucket; and
a pool chooser for selecting a particular connection pool bucket to handle each of the multiple separate SQL statements.
15. A method, performed by distributed database driver middleware that operates between a call-level application programming interface (API) and one or more database drivers, for processing a structured query language (SQL) statement that an application submits via the call-level API, the method comprising:
receiving information that defines a structure of a table that is stored in a distributed database, the information defining that the table is shard across a plurality of database nodes of the distributed database;
receiving, from the application via the call-level API, a SELECT statement to be performed on the table;
accessing the information that defines the structure of the table to identify which database nodes of the distributed database store a shard of the table;
generating, from the SELECT statement and based on the identification of which database nodes of the distributed database store a shard of the table, a separate SELECT statement for each of the database nodes that stores a shard of the table, each separate SELECT statement including routing information for routing the separate SELECT statement to the corresponding database node;
submitting each of the separate SELECT statements to a database driver for the corresponding database node to thereby cause the separate SELECT statement to be directed to the shard of the table that is stored on the corresponding database node;
receiving a result set from each of the database nodes that stores a shard of the table;
merging the result sets to produce a single result set; and
returning the single result set to the application via the call-level API.
16. The method of claim 15 , further comprising:
receiving, from the application via the call-level API, an INSERT statement to be performed on the table;
employing the information to select a first database node of the plurality of database nodes;
generating, from the INSERT statement, a separate INSERT statement that includes routing information to the first database node; and
submitting the INSERT statement to a database driver corresponding to the first database node.