System and method for providing access to a sharded database using a cache and a shard technology
In accordance with an embodiment, the system enables access to a sharded database using a cache and a shard topology. A shard-aware client application connecting to a sharded database can use a connection pool (e.g., a Universal Connection Pool, UCP), to store or access connections to different shards or chunks of the sharded database within a shared pool. As new connections are created, a shard topology layer can be built at the database driver layer, which learns and caches shard key ranges to locations of shards. The shard topology layer enables subsequent connection requests from a client application to use a fast key path access to the appropriate shard or chunk.
1. A system for providing access to a sharded database, comprising:
a computer, including a processor;
an application server or database environment which provides access to a database having a plurality of shards by which data is stored and presented, and wherein the database is associated with
a database driver, and
a connection pool which creates and maintains a pool of connections for use with the database,
wherein the connection pool and database driver operate together to provide access by client applications to data stored at the database, using the connections;
wherein the database driver caches shard key ranges to locations of shards within the database, as a shard topology, for use in processing connection requests;
wherein the database driver is configured to enable a client application to access the database as part of a connection request, including determining a location of an appropriate shard within the database, using the shard topology, and providing access by the client application to the appropriate shard of the database; and
wherein the connection pool is configured to identify a connection by its shard key, and to allow re-use of the connection when a request for a same shard key is received from a client application.
2. The system of claim 1 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and re-use that connection.
3. The system of claim 1 , wherein the shard topology layer enables subsequent connection requests to bypass a shard director or listener component that operates to provide access by software client applications to database shards, and instead use a fast key path access to the appropriate shard or chunk.
4. The system of claim 1 , wherein the connection pool is configured to identify each of the connections by its shard key, and to determine and allow re-use of particular connections when requests for same shard keys are received from client applications.
5. The system of claim 1 , wherein a shard key or similar interface is provided for use by shard-aware client applications identify and build shard keys, and optionally a shard group, needed to fetch a connection to the sharded database.
6. The system of claim 1 , wherein when a shard-aware client application does not provide a shard key in association with the connection request, or if the connection pool or database driver does not have a mapping for a provided shard key, then the connection request is forwarded to a shard director or listener.
7. A method of providing access to a sharded database, comprising:
providing, by a computer, access to a database having a plurality of shards by which data is stored and presented, and wherein the database is associated with
a database driver, and
a connection pool which creates and maintains a pool of connections for use with the database,
wherein the connection pool and database driver operate together to provide access by client applications to data stored at the database, using the connections;
wherein the database driver caches shard key ranges to locations of shards within the database, as a shard topology, for use in processing connection requests;
wherein the database driver is configured to enable a client application to access the database as part of a connection request, including determining a location of an appropriate shard within the database, using the shard topology, and providing access by the client application to the appropriate shard of the database; and
wherein the connection pool is configured to identify a connection by its shard key, and to allow re-use of the connection when a request for a same shard key is received from a client application.
8. The method of claim 7 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and re-use that connection.
9. The method of claim 7 , wherein the shard topology layer enables subsequent connection requests to bypass a shard director or listener component that operates to provide access by software client applications to database shards, and instead use a fast key path access to the appropriate shard or chunk.
10. The method of claim 7 , wherein the connection pool is configured to identify each of the connections by its shard key, and to determine and allow re-use of particular connections when requests for same shard keys are received from client applications.
11. The method of claim 7 , wherein a shard key or similar interface is provided for use by shard-aware client applications identify and build shard keys, and optionally a shard group, needed to fetch a connection to the sharded database.
12. The method of claim 7 , wherein when a shard-aware client application does not provide a shard key in association with the connection request, or if the connection pool or database driver does not have a mapping for a provided shard key, then the connection request is forwarded to a shard director or listener.
13. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:
providing, by a computer, access to a database having a plurality of shards by which data is stored and presented, and wherein the database is associated with
a database driver, and
a connection pool which creates and maintains a pool of connections for use with the database,
wherein the connection pool and database driver operate together to provide access by client applications to data stored at the database, using the connections;
wherein the database driver caches shard key ranges to locations of shards within the database, as a shard topology, for use in processing connection requests;
wherein the database driver is configured to enable a client application to access the database as part of a connection request, including determining a location of an appropriate shard within the database, using the shard topology, and providing access by the client application to the appropriate shard of the database; and
wherein the connection pool is configured to identify a connection by its shard key, and to allow re-use of the connection when a request for a same shard key is received from a client application.
14. The non-transitory computer readable storage medium of claim 13 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and re-use that connection.
15. The non-transitory computer readable storage medium of claim 13 , wherein the shard topology layer enables subsequent connection requests to bypass a shard director or listener component that operates to provide access by software client applications to database shards, and instead use a fast key path access to the appropriate shard or chunk.
16. The non-transitory computer readable storage medium of claim 13 , wherein the connection pool is configured to identify each of the connections by its shard key, and to determine and allow re-use of particular connections when requests for same shard keys are received from client applications.
17. The non-transitory computer readable storage medium of claim 13 , wherein a shard key or similar interface is provided for use by shard-aware client applications identify and build shard keys, and optionally a shard group, needed to fetch a connection to the sharded database.
18. The non-transitory computer readable storage medium of claim 13 , wherein when a shard-aware client application does not provide a shard key in association with the connection request, or if the connection pool or database driver does not have a mapping for a provided shard key, then the connection request is forwarded to a shard director or listener.