System and method for providing access to a sharded database using a cache and a shard technology
In accordance with an embodiment, the system enables access to a sharded database using a cache and a shard topology. A shard-aware client application connecting to a sharded database can use a connection pool (e.g., a Universal Connection Pool, UCP), to store or access connections to different shards or chunks of the sharded database within a shared pool. As new connections are created, a shard topology layer can be built at the database driver layer, which learns and caches shard key ranges to locations of shards. The shard topology layer enables subsequent connection requests from a client application to use a fast key path access to the appropriate shard or chunk.
1. A system for providing access to a sharded database, comprising:
a computer, including a processor, an application server or database environment, and a connection pool, that provides access to a database having a plurality of shards by which data is stored and presented, including:
maintaining, at the connection pool, a pool of connections for use with a database driver in accessing the database in response to application requests;
receiving, from a client application, a shard key information during at least one of a checkout of a connection to the database, or at a later point in time; and
using the shard key information to identify a connection by its shard key and provide access by the client application to a corresponding shard of the database, for use by the client application.
2. The system of claim 1 , wherein the database driver and connection pool are configured to recognize shard keys specified by the client application, and enable the client application to connect to a specific shard or chunk associated with that client application.
3. The system of claim 1 , wherein given one or more shard keys in a connection request, the connection pool attempts to look up a matching chunk name for the keys; and
if a matching chunk name is not found, then the connection request is forwarded to a database listener for a new connection to be created.
4. The system of claim 3 , wherein a chunk name in the connection request is used to fetch a list of instances on which the chunk is present, using a connection selection algorithm, and a connection is chosen to one of these instances.
5. The system of claim 3 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and re-use that connection.
6. A method of providing access to a sharded database, comprising:
providing, by a computer, access to a database having a plurality of shards by which data is stored and presented;
maintaining, at the connection pool, a pool of connections for use with a database driver in accessing the database in response to application requests;
receiving, from a client application, a shard key information during at least one of a checkout of a connection to the database, or at a later point in time; and
using the shard key information to identify a connection by its shard key and provide access by the client application to a corresponding shard of the database, for use by the client application.
7. The method of claim 6 , wherein the database driver and connection pool are configured to recognize shard keys specified by the client application, and enable the client application to connect to a specific shard or chunk associated with that client application.
8. The method of claim 6 , wherein given one or more shard keys in a connection request, the connection pool attempts to look up a matching chunk name for the keys; and
if a matching chunk name is not found, then the connection request is forwarded to a database listener for a new connection to be created.
9. The method of claim 8 , wherein a chunk name in the connection request is used to fetch a list of instances on which the chunk is present, using a connection selection algorithm, and a connection is chosen to one of these instances.
10. The method of claim 8 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and re-use that connection.
11. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:
providing access to a database having a plurality of shards by which data is stored and presented;
maintaining, at the connection pool, a pool of connections for use with a database driver in accessing the database in response to application requests;
receiving, from a client application, a shard key information during at least one of a checkout of a connection to the database, or at a later point in time; and
using the shard key information to identify a connection by its shard key and provide access by the client application to a corresponding shard of the database, for use by the client application.
12. The non-transitory computer readable storage medium of claim 11 , wherein the database driver and connection pool are configured to recognize shard keys specified by the client application, and enable the client application to connect to a specific shard or chunk associated with that client application.
13. The non-transitory computer readable storage medium of claim 11 , wherein given one or more shard keys in a connection request, the connection pool attempts to look up a matching chunk name for the keys; and
if a matching chunk name is not found, then the connection request is forwarded to a database listener for a new connection to be created.
14. The non-transitory computer readable storage medium of claim 13 , wherein a chunk name in the connection request is used to fetch a list of instances on which the chunk is present, using a connection selection algorithm, and a connection is chosen to one of these instances.
15. The non-transitory computer readable storage medium of claim 13 , wherein, if there are no connections in the connection pool to a particular shard or chunk then an attempt is made to repurpose an existing available connection to another chunk and reuse that connection.