Network filesystem asynchronous I/O scheduling
View Patent ↗Resource acquisition requests for a filesystem are executed under user configurable metering. Initially, a system administrator sets a ratio of N:M for executing N read requests for M write requests. As resource acquisition requests are received by a filesystem server, the resource acquisition requests are sorted into queues, e.g., where read and write requests have at least one queue for each type, plus a separate queue for metadata requests as they are executed ahead of any waiting read or write request. The filesystem server controls execution of the filesystem resource acquisition requests to maintain the ratio set by the system administrator.
1. A method for processing access requests, the method comprising:
configuring metering of access requests by setting a quota and an importance rating for each queue of a plurality of queues at a file system server based on user input, wherein the quota corresponds to a ratio of read requests versus write requests stored in the plurality of queues;
sorting received access requests into the plurality of queues based on respective types of access requests, wherein each of the plurality of queues is associated with a type of access request, the one or more access requests received at the file system server from one or more compute nodes; and
identifying which of the received access requests from the one or more compute nodes to process by:
scanning the plurality of queues in an order based on the importance rating of each queue of the plurality of queues, and
selecting an access request to process in a first queue of the plurality of queues by identifying that the first queue is associated with a first quota has not been exhausted when the number of read requests versus write requests stored in the plurality of queues is in accordance with the ratio, wherein the first quota corresponds to a number of access requests to be processed from the first queue while at least one other queue of the plurality of queues has a second quota that has not been exhausted and contains at least one access request that has not been processed.
2. The method of claim 1 , wherein an access request that is latency sensitive is sorted into a queue with a high importance rating.
3. The method of claim 1 , wherein types of access requests include read requests, write requests, and metadata requests.
4. The method of claim 3 , wherein at least one queue of the plurality of queues is associated with metadata requests.
5. The method of claim 4 , wherein the quota for the at least one queue associated with metadata requests is set to infinity such that a queued metadata request is processed before other queued access requests.
6. A non-transitory computer readable storage medium having embodied thereon a program executable by a processor to perform a method for processing access requests, the method comprising:
configuring metering of access requests by setting a quota and an importance rating for each queue of a plurality of queues at a file system server based on user input, wherein the quota corresponds to a ratio of read requests versus write requests stored in the plurality of queues;
sorting received access requests into the plurality of queues based on respective types of access requests, wherein each of the plurality of queues is associated with a type of access request, the one or more access requests received at the file system server from one or more compute nodes; and
identifying which of the received access requests from the one or more compute nodes to process by:
scanning the plurality of queues in an order based on the importance rating of each queue of the plurality of queues, and
selecting an access request to process in a first queue of the plurality of queues by identifying that the first queue is associated with a first quota has not been exhausted when the number of read requests versus write requests stored in the plurality of queues is in accordance with the ratio, wherein the first quota corresponds to a number of access requests to be processed from the first queue while at least one other queue of the plurality of queues has a second quota that has not been exhausted and contains at least one access request that has not been processed.
7. The non-transitory computer readable storage medium of claim 6 , wherein an access request that is latency sensitive is sorted into a queue with a high importance rating.
8. The non-transitory computer readable storage medium of claim 6 , wherein types of access requests include read requests, write requests, and metadata requests.
9. The non-transitory computer readable storage medium of claim 8 , wherein at least one queue of the plurality of queues is associated with metadata requests.
10. The non-transitory computer readable storage medium of claim 9 , wherein the quota for the at least one queue associated with metadata requests is set to infinity such that a queued metadata request is processed before other queued access requests.
11. A system for processing access requests, the system comprising:
a plurality of compute nodes; and
a file system server, wherein the file system server:
configures metering of access requests by setting a quota and an importance rating for each queue of a plurality of queues at a file system server based on one or more user settings, the quota corresponding to a ratio of read requests versus write requests stored in the plurality of queues;
sorts received access requests into the plurality of queues based on respective types of access requests, wherein each of the plurality of queues is associated with a type of access request, the one or more access requests received at the file system server over the network interface from the one or more compute nodes; and
identifies which of the received access requests from the one or more compute nodes to process by:
scanning the plurality of queues in an order based on the importance rating of each queue of the plurality of queues, and
selecting an access request to process in a first queue of the plurality of queues by identifying that the first queue is associated with a first quota that has not been exhausted, the first quota corresponding to a number of access requests to be processed from the first queue while at least one other queue of the plurality of queues has a second quota that has not been exhausted and contains at least one access request that has not been processed.
12. The system of claim 11 , wherein an access request that is latency sensitive is sorted into a queue with a high importance rating.
13. The system of claim 11 , wherein types of access requests include read requests, write requests, and metadata requests.
14. The system of claim 13 , wherein at least one queue of the plurality of queues is associated with metadata requests.
15. The system of claim 14 , wherein the quota for the at least one queue associated with metadata requests is set to infinity such that a queued metadata request is processed before other queued access requests.