Adaptive resource allocation based upon observed historical usage
Allocation of resources across multiple consumers allows efficient utilization of shared resources. Observed usages of resources by consumers over time intervals are used to determine a total throughput of resources by the consumers. The total throughput of resources is used to determine allocation of resources for a subsequent time interval. The consumers are associated with priorities used to determine their allocations. Minimum and maximum resource guarantees may be associated with consumers. The resource allocation aims to allocate resources based on the priorities of the consumers while aiming to avoid starvation by any consumer. The resource allocation allows efficient usage of network resources in a database storage system storing multiple virtual databases.
1. A method for allocating a network resource used by a plurality of consumers based upon a total throughput of the network resource, wherein the total throughput is a measure of the aggregated network bandwidth available to the plurality of consumers using the network resource, the method comprising:
initially allocating the network resource for each consumer in the plurality of consumers using a low estimate of bandwidth based on a fraction of a stated link capacity as the total throughput of the network resource, wherein an allocation of the network resource for each consumer determines availability of network resource to the consumer for a subsequent time interval;
iterating, over one or more cycles:
determining a plurality of cumulative usage values of the network resource, wherein each cumulative usage value is associated with a time interval and is based on observed usages of the network resource by a corresponding consumer of the plurality of consumers over the time interval, wherein each consumer represents a process executing on a computer system and accessing a database storage system via the network resource;
determining a total observed throughput of the network resource based on an aggregate of the plurality of cumulative usage values of the network resource, wherein each of the plurality of cumulative usage values is weighted based upon a non-zero weight value determined as a decreasing function of the age of the time intervals relative to a current time associated with the cumulative usage values;
determining a total amount of the network resource to allocate to the plurality of consumers based upon the determined total observed throughput of the network resource increased by a predetermined factor;
responsive to determining that the determined total amount is above a predetermined threshold value, using a high estimate of bandwidth based on a fraction of the stated link capacity as the total amount; and
allocating the network resource for each consumer in the plurality of consumers, based on the determined total amount.
2. The method of claim 1 , wherein the weight values linearly decrease with the age of the time intervals associated with the cumulative usage values.
3. The method of claim 1 , wherein determining the total throughput value excludes cumulative usage values outside a predetermined time interval.
4. The method of claim 1 , wherein the weight values decrease with the age of the time intervals associated with the cumulative usage values based upon an exponential function.
5. A computer readable non-transitory storage medium, storing instructions for:
initially allocating the network resource for each consumer in the plurality of consumers using a low estimate of bandwidth based on a fraction of a stated link capacity as the total throughput of the network resource, wherein an allocation of the network resource for each consumer determines availability of network resource to the consumer for a subsequent time interval;
iterating, over one or more cycles:
determining a plurality of cumulative usage values of the network resource, wherein each cumulative usage value is associated with a time interval and is based on observed usages of the network resource by a corresponding consumer of the plurality of consumers over the time interval, wherein each consumer represents a process executing on a computer system and accessing a database storage system via the network resource;
determining a total observed throughput of the network resource based on an aggregate of the plurality of cumulative usage values of the network resource, wherein each of the plurality of cumulative usage values is weighted based upon a non-zero weight value determined as a decreasing function of the age of the time intervals relative to a current time associated with the cumulative usage values;
determining a total amount of the network resource to allocate to the plurality of consumers based upon the determined total observed throughput of the network resource increased by a predetermined factor;
responsive to determining that the determined total amount is above a predetermined threshold value, using a high estimate of bandwidth based on a fraction of the stated link capacity as the total amount; and
allocating the network resource for each consumer in the plurality of consumers based on the determined total amount.
6. The computer readable non-transitory storage medium of claim 5 , wherein the weight values linearly decrease with the age of the time interval associated with the cumulative usage values.
7. The computer readable non-transitory storage medium of claim 5 , wherein determining the total throughput value excludes cumulative usage values outside a predetermined time interval.
8. The computer readable non-transitory storage medium of claim 5 , wherein the weight values decrease with the age of the time intervals associated with the cumulative usage values based upon an exponential function.
9. A computer system, comprising:
a computer processor; and
non-transitory computer readable storage medium storing instructions for:
initially allocating the network resource for each consumer in the plurality of consumers using a low estimate of bandwidth based on a fraction of a stated link capacity as the total throughput of the network resource, wherein an allocation of the network resource for each consumer determines availability of network resource to the consumer for a subsequent time interval;
iterating, over one or more cycles:
determining a plurality of cumulative usage values of the network resource, wherein each cumulative usage value is associated with a time interval and is based on observed usages of the network resource by a corresponding consumer a plurality of consumers over the time interval, wherein each consumer represents a process executing on a computer system and accessing a database storage system via the network resource;
determining a total observed throughput of the network resource based on an aggregate of the plurality of cumulative usage values of the network resource, wherein each of the plurality of cumulative usage values is weighted based upon a non-zero weight value determined as a decreasing function of the age of the time intervals relative to a current time associated with the cumulative usage values;
determining a total amount of the network resource to allocate to the plurality of consumers based upon the determined observed throughput of the network resource increased by a predetermined factor;
responsive to determining that the determined total amount is above a predetermined threshold value, using a high estimate of bandwidth based on a fraction of the stated link capacity as the total amount; and
allocating the network resource for each consumer in the plurality of consumers based on the determined total amount.
10. The computer system of claim 9 , wherein the weight values linearly decrease with the age of the time interval associated with the cumulative usage values.
11. The computer system of claim 9 , wherein determining the total throughput value excludes cumulative usage values outside a predetermined time interval.
12. The computer system of claim 9 , wherein the weight values decrease with the age of the time intervals associated with the cumulative usage values based upon an exponential function.