Optimizing dynamical resource allocations in disaggregated data centers
Embodiments for optimizing dynamic resource allocations in a disaggregated computing environment. Data access patterns of data elements associated with a set of workloads are tracked across predetermined windows of time. The set of workloads is categorized into at least one of a plurality of classes, each class characterized by a method of data access of the data elements during the tracking of the data access patterns. Resource allocations are optimized in the disaggregated computing environment for a current iteration of the set of workloads based on the method of data access by allocating specific resources within the disaggregated data center to the current iteration of the set of workloads according to the data access patterns in real-time.
1. A method for optimizing dynamic resource allocations in a disaggregated computing environment, by a processor device, comprising:
tracking data access patterns of data elements across predetermined windows of time, the data elements used in execution of a set of workloads, wherein the tracked data access patterns are assembled into a heat map, the heat map including information on a workload data scope and a most recent access pattern including a most recent access location of the data elements;
categorizing the set of workloads into at least one of a plurality of classes, each class characterized by a method of data access of the data elements during the tracking of the data access patterns, wherein the data access patterns recorded in the heat map include an indication of a percentage and quantity of data requests of the data elements were fulfilled by each of a plurality of specific individual hardware resources, and wherein the heat map includes additional information of a cost per bit versus a locality of the data elements, as determined by the most recent access location, relative to respective others of the specific individual hardware resources that execute the set of workloads; and
optimizing resource allocations in the disaggregated computing environment for a current iteration of the set of workloads based on the information and the additional information in the heat map by allocating at least a portion of the specific individual hardware resources, each selected from respective pools of the hardware resources within the disaggregated computing environment, to the current iteration of the set of workloads according to the data access patterns in real-time; wherein the specific individual hardware resources each contribute only a portion of processing of the current iteration of the set of workloads notwithstanding each of the specific individual hardware resources are assigned to a single dynamically composed computing node executing the current iteration of the set of workloads irrespective of a physical location of the pools to which the specific individual hardware resources reside.
2. The method of claim 1 , wherein the optimizing further comprises optimizing a balance between an overall disaggregated framework utilization and a performance of each dynamically constructed hardware system within the disaggregated computing environment.
3. The method of claim 2 , further including categorizing a set of the dynamically constructed hardware systems into the at least one of the plurality of classes by allocating the specific individual hardware resources associated with respective ones of the classes to each respective dynamically constructed hardware system.
4. The method of claim 1 , wherein the plurality of classes include:
a cache-friendly class wherein the data elements exhibit data access patterns of having cache hits over a certain threshold;
a cache-needy class wherein the data elements exhibit data access patterns determined to become cache-friendly when additional cache is allocated;
a memory-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied with memory resources; and
a storage-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied using pre-fetched data in an appropriate storage medium.
5. The method of claim 1 , further including updating the heat map according to a sampling algorithm of a sampling of the tracked data access patterns, the sampling algorithm set as a parameter configurable by a system administrator.
6. The method of claim 5 , further including performing the optimization at predetermined intervals assigned by the system administrator or in response to detecting one of a set of special system triggers.
7. The method of claim 6 , wherein the set of special system triggers are selected from a group consisting of an initial allocation of a new dynamic hardware system, a hardware component failure, a system alert of performance degradation, and a Service Level Agreement (SLA) violation.
8. A system for optimizing dynamic resource allocations in a disaggregated computing environment, the system comprising:
a processor device executing instructions stored in a memory device, wherein the processor device:
tracks data access patterns of data elements across predetermined windows of time, the data elements used in execution of a set of workloads, wherein the tracked data access patterns are assembled into a heat map, the heat map including information on a workload data scope and a most recent access pattern including a most recent access location of the data elements;
categorizes the set of workloads into at least one of a plurality of classes, each class characterized by a method of data access of the data elements during the tracking of the data access patterns, wherein the data access patterns recorded in the heat map include an indication of a percentage and quantity of data requests of the data elements were fulfilled by each of a plurality of specific individual hardware resources, and wherein the heat map includes additional information of a cost per bit versus a locality of the data elements, as determined by the most recent access location, relative to respective others of the specific individual hardware resources that execute the set of workloads; and
optimizes resource allocations in the disaggregated computing environment for a current iteration of the set of workloads based on the information and the additional information in the heat map by allocating at least a portion of the specific individual hardware resources, each selected from respective pools of the hardware resources within the disaggregated computing environment, to the current iteration of the set of workloads according to the data access patterns in real-time; wherein the specific individual hardware resources each contribute only a portion of processing of the current iteration of the set of workloads notwithstanding each of the specific individual hardware resources are assigned to a single dynamically composed computing node executing the current iteration of the set of workloads irrespective of a physical location of the pools to which the specific individual hardware resources reside.
9. The system of claim 8 , wherein the optimizing further comprises optimizing a balance between an overall disaggregated framework utilization and a performance of each dynamically constructed hardware system within the disaggregated computing environment.
10. The system of claim 9 , wherein the processor device categorizes a set of the dynamically constructed hardware systems into the at least one of the plurality of classes by allocating the specific individual hardware resources associated with respective ones of the classes to each respective dynamically constructed hardware system.
11. The system of claim 8 , wherein the plurality of classes include:
a cache-friendly class wherein the data elements exhibit data access patterns of having cache hits over a certain threshold;
a cache-needy class wherein the data elements exhibit data access patterns determined to become cache-friendly when additional cache is allocated;
a memory-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied with memory resources; and
a storage-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied using pre-fetched data in an appropriate storage medium.
12. The system of claim 8 , wherein the processor device updates the heat map according to a sampling algorithm of a sampling of the tracked data access patterns, the sampling algorithm set as a parameter configurable by a system administrator.
13. The system of claim 12 , wherein the processor device performs the optimization at predetermined intervals assigned by the system administrator or in response to detecting one of a set of special system triggers.
14. The system of claim 13 , wherein the set of special system triggers are selected from a group consisting of an initial allocation of a new dynamic hardware system, a hardware component failure, a system alert of performance degradation, and a Service Level Agreement (SLA) violation.
15. A computer program product for optimizing dynamic resource allocations in a disaggregated computing environment, by a processor device, the computer program product embodied on a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion that tracks data access patterns of data elements across predetermined windows of time, the data elements used in execution of a set of workloads, wherein the tracked data access patterns are assembled into a heat map, the heat map including information on a workload data scope and a most recent access pattern including a most recent access location of the data elements;
an executable portion that categorizes the set of workloads into at least one of a plurality of classes, each class characterized by a method of data access of the data elements during the tracking of the data access patterns, wherein the data access patterns recorded in the heat map include an indication of a percentage and quantity of data requests of the data elements were fulfilled by each of a plurality of specific individual hardware resources, and wherein the heat map includes additional information of a cost per bit versus a locality of the data elements, as determined by the most recent access location, relative to respective others of the specific individual hardware resources that execute the set of workloads; and
an executable portion that optimizes resource allocations in the disaggregated computing environment for a current iteration of the set of workloads based on the information and the additional information in the heat map by allocating at least a portion of the specific individual hardware resources, each selected from respective pools of the hardware resources within the disaggregated computing environment, to the current iteration of the set of workloads according to the data access patterns in real-time; wherein the specific individual hardware resources each contribute only a portion of processing of the current iteration of the set of workloads notwithstanding each of the specific individual hardware resources are assigned to a single dynamically composed computing node executing the current iteration of the set of workloads irrespective of a physical location of the pools to which the specific individual hardware resources reside.
16. The computer program product of claim 15 , wherein the optimizing further comprises optimizing a balance between an overall disaggregated framework utilization and a performance of each dynamically constructed hardware system within the disaggregated computing environment.
17. The computer program product of claim 16 , further including an executable portion that categorizes a set of the dynamically constructed hardware systems into the at least one of the plurality of classes by allocating the specific individual hardware resources associated with respective ones of the classes to each respective dynamically constructed hardware system.
18. The computer program product of claim 15 , wherein the plurality of classes include:
a cache-friendly class wherein the data elements exhibit data access patterns of having cache hits over a certain threshold;
a cache-needy class wherein the data elements exhibit data access patterns determined to become cache-friendly when additional cache is allocated;
a memory-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied with memory resources; and
a storage-friendly class wherein the data elements exhibit data access patterns determined to be best satisfied using pre-fetched data in an appropriate storage medium.
19. The computer program product of claim 15 , further including an executable portion that updates the heat map according to a sampling algorithm of a sampling of the tracked data access patterns, the sampling algorithm set as a parameter configurable by a system administrator.
20. The computer program product of claim 19 , further including an executable portion that performs the optimization at predetermined intervals assigned by the system administrator or in response to detecting one of a set of special system triggers.
21. The computer program product of claim 20 , wherein the set of special system triggers are selected from a group consisting of an initial allocation of a new dynamic hardware system, a hardware component failure, a system alert of performance degradation, and a Service Level Agreement (SLA) violation.