Smart load balancing replication when adding or removing storage disks in a distributed storage system

1. A method, in a data processing system, for balancing workload to one or more storage disks in a plurality of storage disks during redistribution or replication associated with adding or removing a storage disk to the plurality of storage disks, the method comprising:

utilizing collected information for the plurality of storage disks, analyzing historical information in the collected information to identify I/O operation patterns on a per storage disk level;

determining an average amount of I/O operations that occur within each storage disk in the plurality of storage disks for a given time period; and

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

generating a disk on/off-boarding plan that identifies a subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period using the average amount of I/O operations that historically occur within the storage disk that is impacted during the given time period; and

executing the subset of I/O operations in the given time period.

2. The method of claim 1 , further comprising:

executing external I/O operations from one or more applications alongside the set of I/O operations during the given time period.

3. The method of claim 1 , wherein the set of I/O operations comprises level 1 object operations (Level1IO) that are urgent to be moved or replicated and level 2 object operations (Level2IO) that are lower priority for replication or migration.

4. The method of claim 3 , wherein the level 1 object operations (Level1IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level1IO Ti =max(Weightlevel1*MaxBandwidth Ti , MaxBandwidth Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel1 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , and DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted.

5. The method of claim 3 , wherein the level 2 object operations (Level2IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level2IO Ti =max(Weightlevel2*MaxBandwidth Ti , MaxBandwidth Ti −Level1IO Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel2 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted, and Level1IO Ti are the level 1 operations that are to be performed in the given time period.

6. The method of claim 1 , further comprising:

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

responsive to a failure to complete all of the set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period, generating a revised disk on/off-boarding plan that identifies another subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in a next time period using the average amount of I/O operations that occur historically within the storage disk that is impacted during the next time period; and

executing the other subset of I/O operations in the given time period.

7. The method of claim 1 , wherein the collected information comprises one or more of whether an I/O operation is from an external user of an internal component, whether the I/O operation is a read I/O operation or a write I/O operation; a time that the I/O operation was initialized; a length of time required to execute the I/O operation; or a type of storage disk upon which the I/O operation is being performed on a per storage disk basis for the plurality of storage disks.

8. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

utilizing collected information for a plurality of storage disks, analyze historical information in the collected information to identify I/O operation patterns on a per storage disk level;

determine an average amount of I/O operations that occur within each storage disk in the plurality of storage disks for a given time period; and

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

generate a disk on/off-boarding plan that identifies a subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period using the average amount of I/O operations that historically occur within the storage disk that is impacted during the given time period; and

execute the subset of I/O operations in the given time period.

9. The computer program product of claim 8 , wherein the computer readable program further causes the computing device to:

execute external I/O operations from one or more applications alongside the set of operations during the given time period.

10. The computer program product of claim 8 , wherein the set of I/O operations comprises level 1 object operations (Level1IO) that are urgent to be moved or replicated and level 2 object operations (Level2IO) that are lower priority for replication or migration.

11. The computer program product of claim 10 , wherein the level 1 object operations (Level1IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level1IO Ti =max(Weightlevel1*MaxBandwidth Ti , MaxBandwidth Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel1 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , and DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted.

12. The computer program product of claim 10 , wherein the level 2 object operations (Level2IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level2IO Ti =max(Weightlevel2*MaxBandwidth Ti , MaxBandwidth Ti −Level1IO Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel2 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted, and Level1IO Ti are the level 1 operations that are to be performed in the given time period.

13. The computer program product of claim 8 , wherein the computer readable program further causes the computing device to:

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

responsive to a failure to complete all of the set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period, generate a revised disk on/off-boarding plan that identifies another subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in a next time period using the average amount of I/O operations that occur historically within the storage disk that is impacted during the next time period; and

execute the other subset of I/O operations in the given time period.

14. The computer program product of claim 8 , wherein the collected information comprises one or more of whether an I/O operation is from an external user of an internal component, whether the I/O operation is a read I/O operation or a write I/O operation; a time that the I/O operation was initialized; a length of time required to execute the I/O operation; or a type of storage disk upon which the I/O operation is being performed on a per storage disk basis for the plurality of storage disks.

15. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:

utilizing collected information for a plurality of storage disks, analyze historical information in the collected information to identify I/O operation patterns on a per storage disk level;

determine an average amount of I/O operations that occur within each storage disk in the plurality of storage disks for a given time period; and

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

generate a disk on/off-boarding plan that identifies a subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period using the average amount of I/O operations that historically occur within the storage disk that is impacted during the given time period; and

execute the subset of I/O operations in the given time period.

16. The apparatus of claim 15 , wherein the instructions further cause the processor to:

execute external I/O operations from one or more applications alongside the set of I/O operations during the given time period.

17. The apparatus of claim 15 , wherein the set of I/O operations comprises level 1 object operations (Level1IO) that are urgent to be moved or replicated and level 2 object operations (Level2IO) that are lower priority for replication or migration.

18. The apparatus of claim 17 , wherein the level 1 object operations (Level1IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level1IO Ti =max(Weightlevel1*MaxBandwidth Ti , MaxBandwidth Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel1 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , and DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted.

19. The apparatus of claim 17 , wherein the level 2 object operations (Level2IO) that are part of the set of I/O operations for the given time period T i are identified using:

Level2IO Ti =max(Weightlevel2*MaxBandwidth Ti , MaxBandwidth Ti −Level1IO Ti −DeltaIO Ti-1 −AvgIO Ti )

where Weightlevel2 is a configurable number, AvgIO Ti is the historical average I/O operations performed on the storage disk that is impacted during the given time period T i , MaxBandwidth Ti is the maximum workload bandwidth of the storage disk that is impacted during the time period T i , DeltaIO Ti-1 are the delta external I/O operations that did not finish in a previous time period on the storage disk that is impacted, and Level1IO Ti are the level 1 operations that are to be performed in the given time period.

20. The apparatus of claim 15 , wherein the instructions further cause the processor to:

for each storage disk that is impacted based on the adding or removing of the storage disk to the plurality of storage disks:

responsive to a failure to complete all of the set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in the given time period, generate a revised disk on/off-boarding plan that identifies another subset of I/O operations from a set of I/O operations associated with the adding or removing of the storage disk to the plurality of storage disks to execute in a next time period using the average amount of I/O operations that occur historically within the storage disk that is impacted during the next time period; and

execute the other subset of I/O operations in the given time period.