Sharing node storage resources with the entire cluster
Embodiments of the present disclosure relate to systems and methods for enabling a compute node to utilize all the resources of the cluster it is part of when accessing block storage resources. A deployment manager may receive configuration information for each of a set of compute nodes in a cluster, wherein the configuration information of each compute node in the cluster indicates a usage by the compute node of each of a set of block storage volumes (also referred to herein as storage volumes). The deployment manager determines that a first compute node cannot utilize a full amount of the storage volume bandwidth provided by one or more storage volumes assigned to it. A first storage volume of the one or more storage volumes may be reassigned to a second compute node in the cluster and deploys the cluster with the first storage volume mounted on the second compute node.
1. A method comprising:
receiving configuration information for each of a set of compute nodes in a cluster, wherein the configuration information of each compute node in the cluster indicates a usage by the compute node of each of a set of block storage volumes;
determining, based on the configuration information, that a first compute node cannot utilize a full amount of storage volume bandwidth provided by one or more block storage volumes assigned to it based on a storage subscription plan;
determining, by a processing device, a first block storage volume of the one or more block storage volumes to be reassigned to a second compute node in the cluster; and
deploying the cluster with the first block storage volume mounted on the second compute node in the cluster.
2. The method of claim 1 , further comprising:
receiving at the first compute node, a request directed to the first block storage volume;
redirecting the request to the second compute node; and
redirecting a response to the request to the first compute node.
3. The method of claim 2 , wherein the first compute node includes a driver to redirect the request to the second compute node based on a mapping of each of the one or more block storage volumes to a compute node of the cluster.
4. The method of claim 1 , further comprising:
receiving information indicating an available bandwidth of each of the set of compute nodes in the cluster; and
receiving user input regarding requests serviced by each of the one or more block storage volumes.
5. The method of claim 4 , wherein the first block storage volume is determined based on the user input regarding requests serviced by each of the one or more block storage volumes.
6. The method of claim 4 , further comprising:
determining the second compute node based on the available bandwidth of each of the set of compute nodes in the cluster.
7. The method of claim 4 , wherein the user input regarding requests serviced by each of the one or more block storage volumes comprises:
a priority of requests serviced by each of the one or more block storage volumes; and
a turn-around time of requests serviced by each of the one or more block storage volumes.
8. A system comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to:
receive configuration information for each of a set of compute nodes in a cluster, wherein the configuration information of each compute node in the cluster indicates a usage by the compute node of each of a set of block storage volumes;
determine, based on the configuration information, that a first compute node cannot utilize a full amount of storage volume bandwidth provided by one or more block storage volumes assigned to it based on a storage subscription plan;
determine a first block storage volume of the one or more block storage volumes to be reassigned to a second compute node in the cluster; and
deploy the cluster with the first block storage volume mounted on the second compute node in the cluster.
9. The system of claim 8 , wherein the processing device is further to:
receive at the first compute node, a request directed to the first block storage volume;
redirect, by the first compute node, the request to the second compute node; and
redirect, by the second compute node, a response to the request to the first compute node.
10. The system of claim 9 , wherein the first compute node includes a driver to redirect the request to the second compute node based on a mapping of each of the one or more block storage volumes to a compute node of the cluster.
11. The system of claim 8 , wherein the processing device is further to:
receive information indicating an available bandwidth of each of the set of compute nodes in the cluster; and
receive user input regarding requests serviced by each of the one or more block storage volumes.
12. The system of claim 11 , wherein the processing device determines the first block storage volume based on the user input regarding requests serviced by each of the one or more block storage volumes.
13. The system of claim 11 , wherein the processing device is further to:
determine the second compute node based on the available bandwidth of each of the set of compute nodes in the cluster.
14. The system of claim 11 , wherein the user input regarding requests serviced by each of the one or more block storage volumes comprises:
a priority of requests serviced by each of the one or more block storage volumes; and
a turn-around time of requests serviced by each of the one or more block storage volumes.
15. A non-transitory computer-readable medium, having instructions stored thereon which when executed by a processing device, cause the processing device to:
receive configuration information for each of a set of compute nodes in a cluster, wherein the configuration information of each compute node in the cluster indicates a usage by the compute node of each of a set of block storage volumes;
determine, based on the configuration information, that a first compute node cannot utilize a full amount of storage volume bandwidth provided by one or more block storage volumes assigned to it based on a storage subscription plan;
determine, by the processing device, a first block storage volume of the one or more block storage volumes to be reassigned to a second compute node in the cluster; and
deploy the cluster with the first block storage volume mounted on the second compute node in the cluster.
16. The non-transitory computer-readable medium of claim 15 , wherein the processing device is further to:
receive at the first compute node, a request directed to the first block storage volume;
redirect, by the first compute node, the request to the second compute node; and
redirect, by the second compute node, a response to the request to the first compute node.
17. The non-transitory computer-readable medium of claim 15 , wherein the processing device is further to:
receive information indicating an available bandwidth of each of the set of compute nodes in the cluster; and
receive user input regarding requests serviced by each of the one or more block storage volumes.
18. The non-transitory computer-readable medium of claim 17 , wherein the processing device determines the first block storage volume based on the user input regarding requests serviced by each of the one or more block storage volumes.
19. The non-transitory computer-readable medium of claim 17 , wherein the processing device is further to:
determine the second compute node based on the available bandwidth of each of the set of compute nodes in the cluster.
20. The non-transitory computer-readable medium of claim 17 , wherein the user input regarding requests serviced by each of the one or more block storage volumes comprises:
a priority of requests serviced by each of the one or more block storage volumes; and
a turn-around time of requests serviced by each of the one or more block storage volumes.