Systems and methods for end-to-end quality of service control in distributed systems
A computer-implemented method for end-to-end quality of service control in distributed systems may include (1) identifying a plurality of computing systems, wherein each computing system (a) is coupled to a storage resource for the computing system, (b) hosts a plurality of applications that share the storage resource coupled to the computing system, (c) hosts a quality of service agent that limits throughput utilization of the storage resource by each of the plurality of applications, and (d) copies input/output data generated by the applications to a secondary computing system, (2) determining a throughput capacity of the secondary computing system, and (3) providing feedback to at least one quality of service agent hosted by at least one computing system to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for end-to-end quality of service control in distributed systems, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying a plurality of computing systems, wherein each computing system in the plurality of computing systems:
is coupled to a storage resource for the computing system, the storage resource having a limited throughput capacity;
hosts a plurality of applications that share the storage resource coupled to the computing system and that thereby consume the throughput capacity of the storage resource;
hosts a quality of service agent that limits throughput utilization of the storage resource by each of the plurality of applications to prevent overconsumption of the throughput capacity of the storage resource and to thereby maintain at least one specified throughput service level specified for at least one competing application within the plurality of applications that competes within the plurality of applications for throughput to the storage resource; and
copies input/output data generated by the plurality of applications to a secondary computing system, resulting in a separate copy of the input/output data being processed by the secondary computing system and thereby consuming a throughput capacity of the secondary computing system;
the method further comprising:
determining the throughput capacity of the secondary computing system, the throughput capacity of the secondary computing system being separate from the throughput capacity of the storage resource; and
providing feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system, and thereby reduce consumption of the throughput capacity of the storage resource, in response to determining the throughput capacity of the secondary computing system.
2. The computer-implemented method of claim 1 , wherein each computing system in the plurality of computing systems hosts a plurality of applications that share the storage resource coupled to the computing system by hosting a plurality of virtual machines coupled to a virtual storage appliance that provisions, for the plurality of virtual machines, virtual storage that relies on the storage resource.
3. The computer-implemented method of claim 1 , wherein each computing system in the plurality of computing systems copies the input/output data generated by the plurality of applications to the secondary computing system to offload at least one data protection operation from the computing system to the secondary computing system.
4. The computer-implemented method of claim 1 , wherein each computing system in the plurality of computing systems copies the input/output data generated by the plurality of applications to the secondary computing system by:
copying the input/output data to a writeback log;
episodically forwarding a queued portion of the writeback log to the secondary computing system.
5. The computer-implemented method of claim 1 , further comprising receiving, from each computing system within the plurality of computing systems to the secondary computing system, information indicating a demand for copying input/output data to the secondary computing system.
6. The computer-implemented method of claim 5 , further comprising determining that the throughput capacity of the secondary computing system is less than a collective demand for copying input/output data to the secondary computing system.
7. The computer-implemented method of claim 6 , wherein providing feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system is in response to determining that the throughput capacity of the secondary computing system is less than the collective demand.
8. The computer-implemented method of claim 6 , wherein the information indicating the demand projects an anticipated demand based on at least one observation of an application which generates the input/output data to be copied to the secondary computing system.
9. The computer-implemented method of claim 1 , further comprising receiving, from each computing system within the plurality of computing systems at the secondary computing system, information specifying a throughput service level for each application on the computing system.
10. The computer-implemented method of claim 9 , wherein providing feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system comprises providing, based on the information specifying the throughput service level for each application on the computing system, a maximum throughput allotment to the quality of service agent for the computing system that is sufficient to meet the throughput service level for each application on the computing system.
11. The computer-implemented method of claim 9 , wherein providing feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system comprises providing, based on the information specifying the throughput service level for each application on the computing system, a maximum throughput allotment to the quality of service agent for at least one selected application of the plurality of applications that is sufficient to meet the throughput service level for the selected application.
12. A system for end-to-end quality of service control in distributed systems, the system comprising:
an identification module, stored in memory, that identifies a plurality of computing systems, wherein each computing system in the plurality of computing systems:
is coupled to a storage resource for the computing system, the storage resource having a limited throughput capacity;
hosts a plurality of applications that share the storage resource coupled to the computing system and that thereby consume the throughput capacity of the storage resource;
hosts a quality of service agent that limits throughput utilization of the storage resource by each of the plurality of applications to prevent overconsumption of the throughput capacity of the storage resource and to thereby maintain at least one specified throughput service level specified for at least one competing application within the plurality of applications that competes within the plurality of applications for throughput to the storage resource; and
copies input/output data generated by the plurality of applications to a secondary computing system, resulting in a separate copy of the input/output data being processed by the secondary computing system and thereby consuming a throughput capacity of the secondary computing system;
a determination module, stored in memory, that determines the throughput capacity of the secondary computing system, the throughput capacity of the secondary computing system being separate from the throughput capacity of the storage resource;
a providing module, stored in memory, that provides feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system, and thereby reduce consumption of the throughput capacity of the storage resource, in response to determining the throughput capacity of the secondary computing system; and
at least one physical processor configured to execute the identification module, the determination module, and the providing module.
13. The system of claim 12 , wherein each computing system in the plurality of computing systems hosts a plurality of applications that share the storage resource coupled to the computing system by hosting a plurality of virtual machines coupled to a virtual storage appliance that provisions, for the plurality of virtual machines, virtual storage that relies on the storage resource.
14. The system of claim 12 , wherein each computing system in the plurality of computing systems copies the input/output data generated by the plurality of applications to the secondary computing system to offload at least one data protection operation from the computing system to the secondary computing system.
15. The system of claim 12 , wherein each computing system in the plurality of computing systems copies the input/output data generated by the plurality of applications to the secondary computing system by:
copying the input/output data to a writeback log;
episodically forwarding a queued portion of the writeback log to the secondary computing system.
16. The system of claim 12 , wherein the determination module further receives, from each computing system within the plurality of computing systems to the secondary computing system, information indicating a demand for copying input/output data to the secondary computing system.
17. The system of claim 16 , wherein the determination module further determines that the throughput capacity of the secondary computing system is less than a collective demand for copying input/output data to the secondary computing system.
18. The system of claim 17 , wherein the providing module provides feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system in response to determining that the throughput capacity of the secondary computing system is less than the collective demand.
19. The system of claim 17 , wherein the information indicating the demand projects an anticipated demand based on at least one observation of an application which generates the input/output data to be copied to the secondary computing system.
20. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify a plurality of computing systems, wherein each computing system in the plurality of computing systems:
is coupled to a storage resource for the computing system, the storage resource having a limited throughput capacity;
hosts a plurality of applications that share the storage resource coupled to the computing system and that thereby consume the throughput capacity of the storage resource;
hosts a quality of service agent that limits throughput utilization of the storage resource by each of the plurality of applications to prevent overconsumption of the throughput capacity of the storage resource and to thereby maintain at least one specified throughput service level specified for at least one competing application within the plurality of applications that competes within the plurality of applications for throughput to the storage resource; and
copies input/output data generated by the plurality of applications to a secondary computing system, resulting in a separate copy of the input/output data being processed by the secondary computing system and thereby consuming a throughput capacity of the secondary computing system;
determine the throughput capacity of the secondary computing system, the throughput capacity of the secondary computing system being separate from the throughput capacity of the storage resource; and
provide feedback to at least one quality of service agent hosted by at least one computing system within the plurality of computing systems to further limit throughput utilization of at least one of a plurality of applications hosted by the computing system, and thereby reduce consumption of the throughput capacity of the storage resource, in response to determining the throughput capacity of the secondary computing system.