Granted Patent B2

US 11,704,160 · App. 17/145,646 · Granted Jul 18, 2023

Inventors: Yosef Shatsky (Karnei Shomron, IL); Doron Tal (Haifa, IL)

Assignee: EMC IP Holding Company LLC

G06F9/505G06F9/5077G06F9/5083G06F11/3051G06F2209/503

Abstract

Claims (98)

1. A method comprising:

obtaining measurement information comprising an indication of an amount of utilization of a hardware resource of a first server node by a plurality of processing groups assigned to the first server node, each processing group comprising an independent set of data and metadata of a log-structured array storage system;

determining that the amount of utilization of the hardware resource of the first server node is above a threshold amount of utilization based at least in part on the measurement information;

selecting a given processing group of the plurality of processing groups for redistribution based at least in part on the determination that the amount of utilization of the hardware resource of the first server node is above the threshold amount of utilization and on an amount of utilization of the hardware resource by the given processing group;

determining that a second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group; and

redistributing the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group;

wherein the obtaining measurement information comprises measuring resource utilization (i) on a per-group level for each of the plurality of processing groups to generate sets of per-group resource utilization information for respective ones of the processing groups and (ii) on a per-node level for each of the plurality of server nodes to generate sets of per-node resource utilization information for respective ones of the server nodes;

wherein the determining relating to the first server node, the selecting the given processing group, the determining relating to the second server node, and the redistributing the given processing group are performed based at least in part on the set of per-group resource utilization information and the set of per-node resource utilization information; and

wherein the method is performed by a storage control system of the log-structured array storage system, the storage control system comprising at least one processor coupled to memory.

2. The method of claim 1 wherein selecting a given processing group for redistribution comprises:

determining that the amount of utilization of the hardware resource by the given processing group is greater than an amount of utilization of the hardware resource by a second processing group of the plurality of processing groups; and

selecting the given processing group for redistribution based at least in part on the determination the amount of utilization of the hardware resource by the given processing group is greater than the amount of utilization of the hardware resource by the second processing group of the plurality of processing groups.

3. The method of claim 1 wherein the hardware resource comprises at least one of a central processing unit of the first server node, a memory of the first server node and network bandwidth of the first server node.

4. The method of claim 1 wherein:

the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node, the plurality of hardware resources further comprising a second hardware resource of the first server node;

the method further comprises:

determining an amount of utilization of the second hardware resource by the given processing group; and

determining that the second server node comprises enough available capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group; and

redistributing the given processing group to the second server node comprises redistributing the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity to service the determined amount of utilization of the second hardware resource by the given processing group.

5. The method of claim 4 the method further comprising:

determining that an amount of utilization of the second hardware resource on the second server node is above a second threshold amount;

selecting a third processing group of a second plurality of processing groups that are assigned to the second server node for redistribution based at least in part on a determination that the amount of utilization of the second hardware resource by the third processing group is above the second threshold amount;

determining that the first server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the third processing group; and

redistributing the third processing group to the first server node based at least in part on the determination that the first server node comprises enough capacity to service the determined amount of utilization of the second hardware resource by the third processing group.

6. The method of claim 5 wherein determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group comprises determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group based at least in part on the selection of the third processing group for redistribution.

7. The method of claim 1 :

wherein the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node;

wherein the method further comprises:

generating a first vector for the given processing group, the first vector comprising an amount of utilization of each hardware resource of the plurality of hardware resources by the given processing group;

generating a second vector for the second server node, the second vector comprising a total amount of utilization of each hardware resource of the plurality of hardware resources by the second server node;

computing a summation vector based at least in part on the first vector and the second vector; and

determining that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources below a threshold amount for that hardware resource for the second server node; and

wherein determining that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group comprises determining that the second server node comprises enough available capacity of the plurality of hardware resources based at least in part on the determination that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources that is below the threshold amount for each hardware resource for the second server node.

8. An apparatus comprising:

a storage control system of a log-structured array storage system, the storage control system comprising at least one processor coupled to memory, the at least one processor being configured:

to obtain measurement information comprising an indication of an amount of utilization of a hardware resource of a first server node by a plurality of processing groups assigned to the first server node, each processing group comprising an independent set of data and metadata of the log-structured array storage system;

to determine that the amount of utilization of the hardware resource of the first server node is above a threshold amount of utilization based at least in part on the measurement information;

to select a given processing group of the plurality of processing groups for redistribution based at least in part on the determination that the amount of utilization of the hardware resource of the first server node is above the threshold amount of utilization and on an amount of utilization of the hardware resource by the given processing group;

to determine that a second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group; and

to redistribute the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group;

wherein the obtaining measurement information comprises measuring resource utilization (i) on a per-group level for each of the plurality of processing groups to generate sets of per-group resource utilization information for respective ones of the processing groups and (ii) on a per-node level for each of the plurality of server nodes to generate sets of per-node resource utilization information for respective ones of the server nodes; and

wherein the determining relating to the first server node, the selecting the given processing group, the determining relating to the second server node, and the redistributing the given processing group are performed based at least in part on the set of per-group resource utilization information and the set of per-node resource utilization information.

9. The apparatus of claim 8 wherein selecting a given processing group for redistribution comprises:

determining that the amount of utilization of the hardware resource by the given processing group is greater than an amount of utilization of the hardware resource by a second processing group of the plurality of processing groups; and

selecting the given processing group for redistribution based at least in part on the determination the amount of utilization of the hardware resource by the given processing group is greater than the amount of utilization of the hardware resource by the second processing group of the plurality of processing groups.

10. The apparatus of claim 8 wherein the hardware resource comprises at least one of a central processing unit of the first server node, a memory of the first server node and network bandwidth of the first server node.

11. The apparatus of claim 8 :

wherein the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node, the plurality of hardware resources further comprising a second hardware resource of the first server node;

wherein the at least one processor is further configured:

to determine an amount of utilization of the second hardware resource by the given processing group; and

to determine that the second server node comprises enough available capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group; and

wherein redistributing the given processing group to the second server node comprises redistributing the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity to service the determined amount of utilization of the second hardware resource by the given processing group.

12. The apparatus of claim 11 wherein the at least one processor is further configured:

to determine that an amount of utilization of the second hardware resource on the second server node is above a second threshold amount;

to select a third processing group of a second plurality of processing groups that are assigned to the second server node for redistribution based at least in part on a determination that the amount of utilization of the second hardware resource by the third processing group is above the second threshold amount;

to determine that the first server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the third processing group; and

to redistribute the third processing group to the first server node based at least in part on the determination that the first server node comprises enough capacity to service the determined amount of utilization of the second hardware resource by the third processing group.

13. The apparatus of claim 12 wherein determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group comprises determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group based at least in part on the selection of the third processing group for redistribution.

14. The apparatus of claim 8 :

wherein the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node;

wherein the at least one processor is further configured:

to generate a first vector for the given processing group, the first vector comprising an amount of utilization of each hardware resource of the plurality of hardware resources by the given processing group;

to generate a second vector for the second server node, the second vector comprising a total amount of utilization of each hardware resource of the plurality of hardware resources by the second server node;

to compute a summation vector based at least in part on the first vector and the second vector; and

to determine that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources below a threshold amount for that hardware resource for the second server node; and

wherein determining that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group comprises determining that the second server node comprises enough available capacity of the plurality of hardware resources based at least in part on the determination that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources that is below the threshold amount for each hardware resource for the second server node.

15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, the program code when executed by a storage control system of a log-structured array storage system, the storage control system comprising at least one processor coupled to memory, causes the at least one processor:

to obtain measurement information comprising an indication of an amount of utilization of a hardware resource of a first server node by a plurality of processing groups assigned to the first server node, each processing group comprising an independent set of data and metadata of the log-structured array storage system;

to determine that the amount of utilization of the hardware resource of the first server node is above a threshold amount of utilization based at least in part on the measurement information;

to select a given processing group of the plurality of processing groups for redistribution based at least in part on the determination that the amount of utilization of the hardware resource of the first server node is above the threshold amount of utilization and on an amount of utilization of the hardware resource by the given processing group;

to determine that a second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group; and

to redistribute the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group;

wherein the obtaining measurement information comprises measuring resource utilization (i) on a per-group level for each of the plurality of processing groups to generate sets of per-group resource utilization information for respective ones of the processing groups and (ii) on a per-node level for each of the plurality of server nodes to generate sets of per-node resource utilization information for respective ones of the server nodes; and

wherein the determining relating to the first server node, the selecting the given processing group, the determining relating to the second server node, and the redistributing the given processing group are performed based at least in part on the set of per-group resource utilization information and the set of per-node resource utilization information.

16. The computer program product of claim 15 wherein selecting a given processing group for redistribution comprises:

determining that the amount of utilization of the hardware resource by the given processing group is greater than an amount of utilization of the hardware resource by a second processing group of the plurality of processing groups; and

selecting the given processing group for redistribution based at least in part on the determination the amount of utilization of the hardware resource by the given processing group is greater than the amount of utilization of the hardware resource by the second processing group of the plurality of processing groups.

17. The computer program product of claim 15 wherein the hardware resource comprises at least one of a central processing unit of the first server node, a memory of the first server node and network bandwidth of the first server node.

18. The computer program product of claim 15 :

wherein the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node, the plurality of hardware resources further comprising a second hardware resource of the first server node;

wherein the program code further causes the at least one processor:

to determine an amount of utilization of the second hardware resource by the given processing group; and

to determine that the second server node comprises enough available capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group; and

wherein redistributing the given processing group to the second server node comprises redistributing the given processing group to the second server node based at least in part on the determination that the second server node comprises enough available capacity to service the determined amount of utilization of the second hardware resource by the given processing group.

19. The computer program product of claim 18 wherein the program code further causes the at least one processor:

to determine that an amount of utilization of the second hardware resource on the second server node is above a second threshold amount;

to select a third processing group of a second plurality of processing groups that are assigned to the second server node for redistribution based at least in part on a determination that the amount of utilization of the second hardware resource by the third processing group is above the second threshold amount;

to determine that the first server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the third processing group; and

to redistribute the third processing group to the first server node based at least in part on the determination that the first server node comprises enough capacity to service the determined amount of utilization of the second hardware resource by the third processing group;

wherein determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group comprises determining that the second server node comprises enough capacity of the second hardware resource to service the determined amount of utilization of the second hardware resource by the given processing group based at least in part on the selection of the third processing group for redistribution.

20. The computer program product of claim 15 :

wherein the hardware resource comprises a first hardware resource of a plurality of hardware resources of the first server node;

wherein the program code further causes the at least one processor:

to generate a first vector for the given processing group, the first vector comprising an amount of utilization of each hardware resource of the plurality of hardware resources by the given processing group;

to generate a second vector for the second server node, the second vector comprising a total amount of utilization of each hardware resource of the plurality of hardware resources by the second server node;

to compute a summation vector based at least in part on the first vector and the second vector; and

to determine that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources below a threshold amount for that hardware resource for the second server node; and

wherein determining that the second server node comprises enough available capacity of the hardware resource to service the determined amount of utilization of the hardware resource by the given processing group comprises determining that the second server node comprises enough available capacity of the plurality of hardware resources based at least in part on the determination that the summation vector comprises a total amount of utilization of each hardware resource of the plurality of hardware resources that is below the threshold amount for each hardware resource for the second server node.

Assignments (9)