Throttling processing threads
Embodiments of the present disclosure relate to throttling processing threads of a storage device. One or more input/output (I/O) workloads of a storage device can be monitored. One or more resources consumed by each thread of each storage device component to process each operation included in a workload can be analyzed. Based on the analysis, consumption of each resource consumed by each thread can be controlled.
1. An apparatus comprising a memory and at least one processor configured to:
monitor one or more input/output (I/O) workloads of one or more storage devices;
analyze one or more resources consumed by one or more threads of each storage device component to process one or more operations included in each workload;
control consumption of the one or more resources by each thread based on the analysis, wherein controlling resource consumption includes limiting one or more out-of-context threads from accessing the one or more resources.
2. The apparatus of claim 1 further configured to:
analyze the one or more I/O workloads for each storage device to identify workload patterns of each storage device; and
predict I/O workload characteristics corresponding to one or more anticipated workloads for each storage device based on the identified patterns.
3. The apparatus of claim 2 further configured to store the analysis each thread's resource consumption in global memory.
4. The apparatus of claim 3 further configured to:
analyze I/O workload processing of each storage device component;
predict I/O workload characteristics corresponding to one or more anticipated workloads for each storage device component; and
store the analysis and the predictions in the global memory.
5. The apparatus of claim 4 further configured to:
analyze the one or more resources consumed each storage device component thread based on the analyzed storage device component workload processing;
predict resource consumption for each component's threads based on the analyzed resource consumption of each thread; and
store the analyzed resource consumption of each thread and the predicted resource consumption for each component's threads in the global memory.
6. The apparatus of claim 5 further configured to generate one or more resource consumption policies for each component's thread based on each thread's predicted resource consumption.
7. The apparatus of claim 6 further configured to generate the one or more resource consumption policies based further on one or more of each analyzed workload, predicted workload, analyzed resource consumptions, predicted resource consumptions for each component of a subject storage device.
8. The apparatus of claim 6 further configured to:
retrieve information including each analyzed workload, predicted workload, analyzed resource consumptions, and predicted resource consumptions for each component of each storage device of one or more storage device clusters in a storage area network (SAN); and
generate the one or more resource consumption policies based further on the retrieved information.
9. The apparatus of claim 8 further configured to generate the one or more resource consumption policies using one or more machine learning (ML) engines, the one or more ML engines including on-policy reinforcement learning.
10. The apparatus of claim 9 further configured to:
generate one or more random resource consumption policies;
execute the one or more random resource consumption policies; and
optimize one or more current resource consumption policies based on analyzed results of the executed resource consumption policies using the one or more ML engines.
11. A method comprising:
monitoring one or more input/output (I/O) workloads of one or more storage devices;
analyzing one or more resources consumed by one or more threads of each storage device component to process one or more operations included in each workload; and
controlling consumption of the one or more resources by each thread based on the analysis, wherein controlling resource consumption includes limiting one or more out-of-context threads from accessing the one or more resources.
12. The method of claim 11 further comprising:
analyzing the one or more I/O workloads for each storage device to identify workload patterns of each storage device; and
predicting I/O workload characteristics corresponding to one or more anticipated workloads for each storage device based on the identified patterns.
13. The method of claim 12 further comprising storing the analysis of each thread's resource consumption in global memory.
14. The method of claim 13 further comprising:
analyzing I/O workload processing of each storage device component;
predicting I/O workload characteristics corresponding to one or more anticipated workloads for each storage device component; and
storing the analysis and the predictions in the global memory.
15. The method of claim 14 further comprising:
analyzing the one or more resources consumed each storage device component thread based on the analyzed storage device component workload processing;
predicting resource consumption for each component's threads based on the analyzed resource consumption of each thread; and
storing the analyzed resource consumption of each thread and the predicted resource consumption for each component's threads in the global memory.
16. The method of claim 15 further comprising generating one or more resource consumption policies for each component's thread based on each thread's predicted resource consumption.
17. The method of claim 16 further comprising generating the one or more resource consumption policies based further on one or more of each analyzed workload, predicted workload, analyzed resource consumptions, predicted resource consumptions for each component of a subject storage device.
18. The method of claim 16 further comprising:
retrieving information including each analyzed workload, predicted workload, analyzed resource consumptions, and predicted resource consumptions for each component of each storage device of one or more storage device clusters in a storage area network (SAN); and
generating the one or more resource consumption policies based further on the retrieved information.
19. The method of claim 18 further comprising generating the one or more resource consumption policies using one or more machine learning (ML) engines, the one or more including on-policy reinforcement learning.
20. The method of claim 19 further comprising:
generating one or more random resource consumption policies;
executing the one or more random resource consumption policies; and
optimizing one or more current resource consumption policies based on analyzed results of the executed resource consumption policies using the one or more ML engines.