Scheduler for handling IO requests of different priorities
In some examples, a data management and storage (DMS) system comprises peer DMS nodes in a node cluster, a distributed data store comprising local and cloud storage, and an IO request scheduler comprising at least one processor configured to perform operations in a method of scheduling IO requests. Example operations comprise implementing a kernel scheduler to schedule a flow of IO requests in the DMS system, and providing an adjustment layer to adjust the kernel scheduler based on an IO request prioritization. A flow of IO requests is identified and some examples implement an IO request prioritization based on the adjustments made by the adjustment layer.
1. In a data management and storage (DMS) system accessing a cluster comprising peer DMS nodes and a distributed data store, a computer-implemented method of scheduling IO requests including:
scheduling a flow of IO requests in the DMS system;
adjusting the scheduled flow of IO requests in the DMS system based on an IO request prioritization, wherein adjusting the scheduled flow of IO requests comprises;
assigning a grace period for first priority IO requests, wherein a length of the grace period assigned to each of the first priority IO requests is dependent on a respective type of the IO request,
throttling second priority IO requests during the grace period for the first priority IO requests, and
blocking third priority IO requests during the grace period for the first priority IO requests; and
implementing the adjusted flow of IO requests in the DMS system.
2. The method of claim 1 , wherein implementing the adjusted flow comprises providing or implementing the grace period for IO requests having a higher priority in the flow, based on a IO request priority threshold value or event.
3. The method of claim 1 , wherein implementing the adjusted flow comprises allowing a passage or processing of concurrent IO requests for a response, the concurrent IO requests being of a same or a different priority based on an IO request priority threshold value or event, the implementing of the IO request prioritization further comprising balancing of the concurrent IO requests.
4. The method of claim 3 , wherein implementing the adjusted flow further comprises enforcing a limit on an IO request concurrency, throughput, or both.
5. The method of claim 1 , wherein implementing the adjusted flow comprises enforcing a dynamic minimum throughput for the second priority IO requests or the third priority IO requests based on a IO request priority threshold value or event, such that the second priority IO requests or the third priority IO requests are not completely starved of a response thereto.
6. A data management and storage (DMS) system, comprising:
peer DMS nodes in a node cluster;
a distributed data store comprising local and cloud storage; and
an IO request scheduler comprising at least one processor configured to perform operations in a method of scheduling IO requests, the operations comprising, at least:
scheduling a flow of IO requests in the DMS system;
adjusting the scheduled flow of IO requests in the DMS system based on an IO request prioritization, wherein adjusting the scheduled flow of IO requests comprises:
assigning a grace period for priority IO requests, wherein a length of the grace period assigned to each of the first priority IO requests is dependent on a respective type of the IO request,
throttling second priority IO requests during the grace period for the first priority IO requests, and
blocking third priority IO requests durine the grace eriod for the first priority IO requests; and
implementing the adjusted flow of IO requests in the DMS system.
7. The DMS system of claim 6 , wherein implementing the adjusted flow comprises providing or implementing the grace period for IO requests having a higher priority in the flow, based on a IO request priority threshold value or event.
8. The DMS system of claim 6 , wherein implementing the adjusted flow comprises allowing a passage or processing of concurrent IO requests for a response, the concurrent IO requests being of a same or a different priority based on an IO request priority threshold value or event, the implementing of the IO request prioritization further comprising balancing of the concurrent IO requests.
9. The DMS system of claim 8 , wherein implementing the adjusted flow further comprises enforcing a limit on an IO request concurrency, throughput, or both.
10. The DMS system of claim 6 , wherein implementing the adjusted flow comprises enforcing a dynamic minimum throughput for the second priority IO requests or the third priority IO requests based on a IO request priority threshold value or event, such that the second priority IO requests or the third priority IO requests are not completely starved of a response thereto.
11. A non-transitory machine-readable medium comprising instructions which, when read by a machine of a data management and storage (DMS) system comprising peer DMS nodes in a node cluster and a distributed data store comprising local and cloud storage DMS system, cause the machine to implement operations in a method of scheduling IO requests, the operations comprising, at least:
scheduling a flow of IO requests in the DMS system;
adjusting the scheduled flow of IO requests in the DMS system based on an IO request prioritization, wherein adjusting the scheduled flow of IO requests comprises:
assigning a grace period for first priority IO requests, wherein a length of the grace period assigned to each of the first priority IO requests is dependent on a respective type of the IO request,
throttling second priority IO requests are during the grace period for the first priority IO requests, and
blocking third priority IO requests during the grace period for the first priority IO requests; and
implementing the adjusted flow of IO requests in the DMS system.
12. The medium of claim 11 , wherein implementing the adjusted flow comprises providing or implementing the grace period for IO requests having a higher priority in the flow, based on a IO request priority threshold value or event.
13. The medium of claim 11 , wherein implementing the adjusted flow comprises allowing a passage or processing of concurrent IO requests for a response, the concurrent IO requests being of a same or a different priority based on an IO request priority threshold value or event, the implementing of the IO request prioritization further comprising balancing of the concurrent IO requests.
14. The method of claim 3 , wherein implementing the adjusted flow further comprises enforcing a limit on an IO request concurrency, throughput, or both.
15. The medium of claim 11 , wherein implementing the adjusted flow comprises enforcing a dynamic minimum throughput for the second priority IO requests or the third priority IO requests based on a IO request priority threshold value or event, such that the second priority IO requests or the third priority IO requests are not completely starved of a response thereto.
16. The method of claim 1 , wherein the length of the grace period assigned to a IO request having type that is read is first duration and the length of the grace period assigned to a IO request having a type that is write is a second duration, wherein the first duration is shorter than the second duration.