Elastic data partitioning of a database
A database entry may be stored in a container in a database table corresponding with a partition key. The partition key may be determined by applying one or more partition rules to one or more data values associated with the database entry. The database entry may be an instance of one of a plurality of data object definitions associated with database entries in the database. Each of the data object definitions may identify a respective one or more data fields included within an instance of the data object definition.
1. A method comprising:
receiving via a communication interface a request to retrieve data from a database table storing a plurality of database entries in one or more containers, the plurality of database entries including a respective one or more data values arranged in a respective one or more data fields in accordance with a respective data object definition, the database table being associated with a plurality of data object definitions including a first data object definition associated with a first partition rule and a second data object definition associated with a second partition rule different from the first partition rule, the request including a first database query identifying one or more characteristics of the data to be retrieved;
applying the first partition rule to the one or more characteristics to identify a subset of the one or more containers, the one or more characteristics identifying the first data object definition;
determining via a processor a second database query based on the first database query, the second database query being limited to the subset of the one or more containers; and
transmitting an instruction to execute the second database query to retrieve the data.
2. The method recited in claim 1 , wherein determining the second database query involves determining a respective computing cost estimate of each of a plurality of alternative database queries including the second database query.
3. The method recited in claim 1 , wherein the second database query is configured to conduct a full scan of the subset of the one or more containers.
4. The method recited in claim 1 , wherein the second database query is configured to employ an index that is local to a designated container of the subset of the one or more containers.
5. The method recited in claim 1 , wherein the first database query and the second database query are expressed in different query languages.
6. The method recited in claim 1 , the method further comprising:
querying a global index table based on the one or more characteristics, the global index table including a global index that spans the database table.
7. The method recited in claim 1 , wherein the first partition rule is formulated as a Boolean expression over one or more data fields.
8. The method recited in claim 7 , wherein the first data object definition identifies a respective data type for each of the data fields.
9. The method recited in claim 8 , wherein the first partition rule can be applied to the one or more data fields without accessing the first data object definition.
10. The method recited in claim 1 , wherein the first partition rule includes a partition key, and wherein the partition key corresponds to a designated container identifier associated with a designated container of the subset of the one or more containers.
11. The method recited in claim 10 , wherein applying a designated function to the partition key yields the designated container, the designated function selected from the group consisting of: a hash function and an identity function.
12. The method recited in claim 1 , wherein the database table resides within a multi-tenant database system, and wherein the plurality of database entries are associated with a plurality of tenant organizations, and wherein the first partition rule is specific to a designated tenant organization of the plurality of tenant organization.
13. A database system implemented using a server system, the database system comprising:
a communication interface configurable to a request to retrieve data from a database table storing a plurality of database entries in one or more containers, the plurality of database entries including a respective one or more data values arranged in a respective one or more data fields in accordance with a respective data object definition, the database table being associated with a plurality of data object definitions including a first data object definition associated with a first partition rule and a second data object definition associated with a second partition rule different from the first partition rule, the request including a first database query identifying one or more characteristics of the data to be retrieved; and
a processor configurable to:
apply the first partition rule to the one or more characteristics to identify a subset of the one or more containers, the one or more characteristics identifying the first data object definition,
determine a second database query based on the first database query, the second database query being limited to the subset of the one or more containers, and
transmit an instruction to execute the second database query to retrieve the data.
14. The database system recited in claim 13 , wherein determining the second database query involves determining a respective computing cost estimate of each of a plurality of alternative database queries including the second database query.
15. The database system recited in claim 13 , wherein the second database query is configured to conduct a full scan of the subset of the one or more containers.
16. The database system recited in claim 13 , wherein the second database query is configured to employ an index that is local to a designated container of the subset of the one or more containers.
17. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:
receiving via a communication interface a request to retrieve data from a database table storing a plurality of database entries in one or more containers, the plurality of database entries including a respective one or more data values arranged in a respective one or more data fields in accordance with a respective data object definition, the database table being associated with a plurality of data object definitions including a first data object definition associated with a first partition rule and a second data object definition associated with a second partition rule different from the first partition rule, the request including a first database query identifying one or more characteristics of the data to be retrieved;
applying the first partition rule to the one or more characteristics to identify a subset of the one or more containers, the one or more characteristics identifying the first data object definition;
determining via a processor a second database query based on the first database query, the second database query being limited to the subset of the one or more containers; and
transmitting an instruction to execute the second database query to retrieve the data.
18. The one or more non-transitory computer readable media recited in claim 17 , wherein determining the second database query involves determining a respective computing cost estimate of each of a plurality of alternative database queries including the second database query.
19. The one or more non-transitory computer readable media recited in claim 17 , wherein the second database query is configured to conduct a full scan of the subset of the one or more containers.
20. The one or more non-transitory computer readable media recited in claim 17 , wherein the second database query is configured to employ an index that is local to a designated container of the subset of the one or more containers.