Providing consistent tenant experiences for multi-tenant databases
The present invention extends to methods, systems, and computer program products for providing consistent tenant experiences at multi-tenant database. Embodiments of the invention include an import service that facilitates data imports without any distribution logic on a tenant. A tenant (caller) provides data in essentially any order. The import service understands the distribution of data across multiple databases and determines for any given piece of import data what database the import data is to be inserted into. Accordingly, the tenant (caller) is relieved from having to know how the data is distributed across a set (of potentially hundreds or thousands) of databases. The import service can optimize an import process to provide load balancing and fairness across a dynamic distributed database system. For example, the import service can insure that a large import by one tenant does not deprive the import other tenants of appropriate resources.
1. A distributed database system, the distributed database system comprising:
one or more processors;
system memory;
a plurality of nodes;
a plurality of databases distributed across the plurality of nodes;
a virtualization module, using the one or more processors, configured to:
initiate import of a one or more data portions into the distributed database system, the one or more data portions associated with a first tenant of the distributed database system, including for each of the one or more data portions:
schedule the data portion for insertion into an appropriate database for the first tenant through reference to a partition map for the distributed database system, the appropriate database for the first tenant included in the plurality of databases;
prior to completely importing the one or more data portions, suspend importing of the one or more data portions to free up resources for importing one or more other data portions to the distributed database system, the one or more other data portions associated with a second different tenant of the distributed database system;
subsequent to suspending import of the one or more data portions, initiate import of the one or more other data portions into the distributed database system, including for each of the one or more other data portions:
schedule the other data portion for insertion into an appropriate database for the second tenant through reference to the partition map, the appropriate database for the first tenant included in the plurality of databases.
2. The distributed database system of claim 1 , wherein a virtualization module, using the one or more processors, configured to schedule the data portion for insertion into an appropriate database for the first tenant comprises a virtualization module, using the one or more processors, configured to insert the data portion into a bucket for the appropriate database for the first tenant, the bucket configured to accumulate data for a specified data range, a data value in the data portion being within the data range.
3. The distributed database system of claim 2 , wherein a virtualization module, using the one or more processors, configured to schedule the data portion for insertion into an appropriate database for the first tenant comprises a virtualization module, using the one or more processors, configured to insert the bucket into a queue, the queue corresponding to a node where the appropriate database for the first tenant is located, the node from among the plurality of nodes.
4. The distributed database system of claim 2 , wherein a virtualization module, using the one or more processors, configured to schedule the other data portion for insertion into an appropriate database for the second tenant comprises a virtualization module, using the one or more processors, configured to insert the other data portion into another bucket for the appropriate database for the second tenant, the other bucket configured to accumulate data for a specified other data range, a data value in the other data portion being within the other data range.
5. The distributed database system of claim 4 , wherein a virtualization module, using the one or more processors, configured to schedule the other data portion for insertion into an appropriate database for the second tenant comprises a virtualization module, using the one or more processors, configured to insert the other bucket into another queue, the other queue corresponding to another node where the appropriate database for the second tenant is located, the other node from among the plurality of nodes.
6. The distributed database system of claim 1 , wherein a virtualization module, using the one or more processors, configured to schedule the data portion for insertion into an appropriate database for the first tenant comprises a virtualization module, using the one or more processors, configured to insert the data portion into a queue, the queue corresponding to a node where the appropriate database for the first tenant is located, the node from among the plurality of nodes.
7. The distributed database system of claim 6 , wherein a virtualization module, using the one or more processors, configured to schedule the other data portion for insertion into an appropriate database for the second tenant comprises a virtualization module, using the one or more processors, configured to insert the other data portion into another queue, the other queue corresponding to another node where the appropriate database for the second tenant is located, the other node from among the plurality of nodes.
8. The distributed database system of claim 7 , wherein the queue and the other queue are the same queue.
9. The distributed database system of claim 1 , further comprising the virtualization module, using the one or more processors, configured to resume importing the one or more data portions into the distributed database system subsequent to importing the one or more other data portions into the distributed database system.
10. The method of claim 1 , wherein suspending importing of the one or more data portions comprises suspending import of the one or more data portions in accordance with an import policy, the import policy defining resource usage thresholds for the first tenant and for the second tenant.
11. A method for use at a computer system, the computer system including a processor and system memory, the method for providing a consistent user experience when importing data into databases maintained by a distributed database system, the method comprising:
initiating import of a one or more data portions into the distributed database system, the one or more data portions associated with a first tenant of the distributed database system, including for each of the one or more data portions:
scheduling the data portion for insertion into an appropriate database for the first tenant through reference to a partition map for the distributed database system;
prior to completely importing the one or more data portions, suspending importing of the one or more data portions to free up resources for importing one or more other data portions to the distributed database system, the one or more other data portions associated with a second different tenant of the distributed database system;
subsequent to suspending importing the one or more data portions, initiating import of the one or more other data portions into the distributed database system, including for each of the one or more other data portions:
scheduling the other data portion for insertion into an appropriate database for the second tenant through reference to the partition map.
12. The method of claim 11 , wherein scheduling the data portion for insertion into an appropriate database for the first tenant comprises:
inserting the data portion into a bucket for the appropriate database for the first tenant, the bucket configured to accumulate data for a specified data range, a data value in the data portion being within the data range; and
inserting the bucket into a queue, the queue corresponding to a node of the distributed database system where the appropriate database for the first tenant is located.
13. The method of claim 12 , wherein scheduling the other data portion for insertion into an appropriate database for the second tenant comprises:
inserting the other data portion into another bucket for the appropriate database for the second tenant, the other bucket configured to accumulate data for a specified other data range, a data value in the other data portion being within the other data range; and
inserting the other bucket into another queue, the other queue corresponding to a node of the distributed database system where the appropriate database for the second tenant is located.
14. The method of claim 11 , wherein scheduling the data portion for insertion into an appropriate database for the first tenant comprises inserting the data portion into a queue, the queue corresponding to a node of the distributed database system where the appropriate database for the first tenant is located; and
wherein scheduling the other data portion for insertion into an appropriate database for the second tenant comprises inserting the other data portion into another queue, the other queue corresponding to a node of the distributed database system where the appropriate database for the second tenant is located.
15. The method of claim 14 , wherein the queue and the other queue are the same queue.
16. The method of claim 11 , further comprising resuming importing the one or more data portions into the distributed database system subsequent to importing the one or more other data portions into the distributed database system.
17. The method of claim 11 , wherein suspending importing of the one or more data portions comprises suspending import of the one or more data portions in accordance with an import policy, the import policy defining resource usage thresholds for the first tenant and for the second tenant.
18. A method for use at a computer system, the computer system including a processor and system memory, the method for providing a consistent user experience when importing data into databases maintained by a distributed database system, the method comprising:
initiating import of a one or more data portions into the distributed database system, the one or more data portions associated with a first tenant of the distributed database system, including for each of the one or more data portions:
scheduling the data portion for insertion into an appropriate database for the first tenant through reference to a partition map for the distributed database system;
prior to completely importing the one or more data portions, throttling import of the one or more data portions to free up resources for importing one or more other data portions to the distributed database system, the one or more other data portions associated with a second different tenant of the distributed database system;
subsequent to throttling import the one or more data portions, initiating import of the one or more other data portions into the distributed database system, including for each of the one or more other data portions:
scheduling the other data portion for insertion into an appropriate database for the second tenant through reference to the partition map.
19. The method of claim 18 , further comprising unthrottling import of the one or more data portions into the distributed database system subsequent to importing the one or more other data portions into the distributed database system.
20. The method of claim 18 , wherein throttling import of the one or more data portions comprises throttling import of the one or more portions of data in accordance with an import policy, the import policy defining resource usage thresholds for the first tenant and for the second tenant.