Management of file system requests in a distributed storage system
A first computing device is part of a distributed electronic storage system (DESS) that also comprises one or more second computing devices. The first computing device comprises client process circuitry and DESS interface circuitry. The DESS interface circuitry is operable to: receive, from client process circuitry of the first computing device, a first client file system request that requires accessing a storage resource on one or more of the second computing devices; determine resources required for servicing of the first client file system request; generate a plurality of DESS file system requests for the first file system request; and transmit the plurality of DESS file system requests onto the one or more network links. How many such DESS file system requests are generated is determined based on the resources required for servicing the first client file system request.
1. A first computing device of a distributed electronic storage system (DESS), the first computing device comprising:
a receive buffer operable to receive and store a plurality of file system requests;
a transmit buffer operable to store and transmit a plurality of DESS requests to a second computing device of the DESS; and
a DESS processor operable to:
receive a first file system request followed by a second file system request;
determine a DESS usage metric, wherein the DESS usage metric comprises an average DESS request size over a determined period of time;
determine whether generating a single DESS request corresponding to the first file system request and a single DESS request corresponding to the second file system request will cause the DESS usage metric to exceed a threshold based on the respective sizes of the first and second file system requests;
generate a single DESS request corresponding to the first file system request and a single DESS request corresponding to the second file system request in response to determining that doing so will not cause the DESS usage metric to exceed the threshold;
generate a plurality of DESS requests corresponding to the first file system request and a single DESS request corresponding to the second file system request in response to determining that doing so will cause the DESS usage metric to exceed the threshold.
2. The DESS of claim 1 , wherein:
the plurality of file system requests are for access to a storage resource on the second computing device, and
the plurality of DESS requests are transmitted to the second computing device via a network link.
3. The DESS of claim 1 , wherein:
a state of the DESS is predicted according to resources required for servicing the first file system request and characteristics of the second file system request, and
the quantity of the plurality of DESS requests corresponding to the first file system request is determined according to the predicted state of the DESS.
4. The DESS of claim 3 , wherein the resources required for servicing the first file system request are determined according to an amount of information to be read during servicing of the first file system request.
5. The DESS of claim 3 , wherein the resources required for servicing the first file system request are determined according to an amount of information to be written during servicing of the first file system request.
6. The DESS of claim 1 , wherein the DESS processor is operable to:
determine a level of congestion of the DESS; and
determine the quantity of the plurality of DESS requests corresponding to the first file system request according to the level of congestion of the DESS.
7. The DESS of claim 6 , wherein the determination of the level of congestion of the DESS comprises a determination of a load on one or more resources of the DESS.
8. The DESS of claim 7 , wherein the one or more resources comprises one or more of processor resources, memory resources, storage resources, and networking resources.
9. The DESS of claim 1 , wherein the quantity of the plurality of DESS requests corresponding to the first file system request is determined according to whether a file system request, in the plurality of file system requests, is a data request or metadata request.
10. One or more non-transitory machine-readable storage medium having code stored thereon, that, when executed by a first computing device of a distributed electronic storage system (DESS), configures the first computing device of the DESS to:
receive a first file system request followed by a second file system request;
determine a DESS usage metric, wherein the DESS usage metric comprises an average DESS request size over a determined period of time;
determine whether generating a single DESS request corresponding to the first file system request and a single DESS request corresponding to the second file system request will cause the DESS usage metric to exceed a threshold based on the respective sizes of the first and second file system requests;
generate a single DESS request corresponding to the first file system request and a single DESS request corresponding to the second file system request in response to determining that doing so will not cause the DESS usage metric to exceed the threshold;
generate a plurality of DESS requests corresponding to the first file system request and a single DESS request corresponding to the second file system request in response to determining that doing so will cause the DESS usage metric to exceed the threshold;
wherein a transmit buffer of the first computing device is operable to store and transmit a plurality of DESS requests to a second computing device of the DESS; and
wherein a receive buffer of the first computing device is operable to receive and store a plurality of file system requests.
11. The one or more non-transitory machine-readable medium of claim 10 , wherein:
the plurality of file system requests are for access to a storage resource on the second computing device, and
the plurality of DESS requests are transmitted to the second computing device via a network link.
12. The one or more non-transitory machine-readable medium of claim 10 , wherein:
a state of the DESS is predicted according to resources required for servicing the first file system request and characteristics of the second file system request, and
the quantity of the plurality of DESS requests corresponding to the first file system request is determined according to the predicted state of the DESS.
13. The one or more non-transitory machine-readable medium of claim 12 , wherein the resources required for servicing the first file system request are determined according to an amount of information to be read during servicing of the first file system request.
14. The one or more non-transitory machine-readable medium of claim 12 , wherein the resources required for servicing the first file system request are determined according to an amount of information to be written during servicing of the first file system request.
15. The one or more non-transitory machine-readable medium of claim 10 , the quantity of the plurality of DESS requests corresponding to the first file system request is determined according to a level of congestion of the DESS.
16. The one or more non-transitory machine-readable medium of claim 15 , wherein the determination of the level of congestion of the DESS comprises a determination of a load on one or more resources of the DESS.
17. The one or more non-transitory machine-readable medium of claim 16 , wherein the one or more resources comprises one or more of processor resources, memory resources, storage resources, and networking resources.
18. The one or more non-transitory machine-readable medium of claim 10 , wherein the quantity of the plurality of DESS requests corresponding to the first file system request is determined according to whether a file system request, in the plurality of file system requests, is a data request or metadata request.