Dynamic and optimized management of grid system resources

1. A computer implemented method, comprising

sensing, by a processor of a grid server in a plurality of grid servers, an ingestion rate of data received by the grid server for a backup, the ingestion rate corresponding to a speed of transferring of the received data into a memory of the grid server, the data is received from at least one data management application communicatively coupled to the plurality of grid servers;

monitoring, by the processor of the grid server, a deduplication and replication rate of the received data, the ingestion rate is being dependent on the deduplication and replication rate,

the deduplication including deduplicating the received data by the grid server based on a determination that at least one zone stamp of a zone within the received data is similar to at least another zone stamp of another zone within the received data, and

the replication including replicating the deduplicated received data to another server in the plurality of servers, the another server being communicatively coupled to the grid server using a communications network; and

adjusting, by the processor of the grid server, the deduplication and replication rate based on the sensed ingestion rate, wherein the adjusting includes:

monitoring at least one of the following: a backup process, deduplication and replication processes, a restore process, a copy-to-tape process, a cross-replication process, a purge process, a rebalance process as performed by the grid server; and

performing, based on the monitoring, sampling of at least one of the following: the ingestion rate, and the deduplication and replication rate;

performing ingestion, deduplication and replication of data in parallel; and

minimizing, based on the sampling and parallel performance of ingestion, deduplication and replication of data, a recovery point objective time associated with the received data, the recovery point objective time representing a difference between times when the data is received into the memory of the grid server and when the data is received into a memory of another grid server.

2. The method according to claim 1 , wherein the adjusting further includes increasing the deduplication and replication rate when the ingestion rate is less than or equal to a first predetermined threshold.

3. The method according to claim 2 , wherein the adjusting further includes unchanging the deduplication and replication rate when the ingestion rate is greater than the first predetermined threshold and less than or equal to a second predetermined threshold.

4. The method according to claim 3 , wherein the adjusting further includes reducing the deduplication and replication rate when the ingestion rate is greater than the second predetermined threshold and less than or equal to a third predetermined threshold.

5. The method according to claim 4 , wherein the adjusting further include setting the deduplication and replication rate to a predetermined minimum deduplication and replication rate when the ingestion rate is greater than the third predetermined threshold.

6. The method according to claim 1 , further comprising

performing, based on the monitoring and the sampling, using the grid server, at least one of the following: increasing the deduplication and replication rate, decreasing the deduplication and replication rate, increasing the ingestion rate, decreasing the ingestion rate, increasing a protection rate corresponding a combination of the deduplication rate and the replication rate, decreasing the protection rate, and/or any combination thereof.

7. The method according to claim 1 , wherein the sampling is performed based on at least one of the following: periodically, automatically, and during a predetermined time interval.

8. The method according to claim 1 , wherein the memories of the grid server and another grid server include at least one of the following: a non-volatile storage medium and a volatile storage medium.

9. The method according to claim 1 , wherein the sampling is performed based on at least one of the following parameter: amount of data stored in a network socket receive buffer associated with the grid server, data packet round trip time associate with transmission of data between at least two grid servers, transmission control protocol advertised receive buffer size, bandwidth of a network including the grid server, a time to complete at least one disk read/write input/output request associated with the grid server, and/or any combination thereof.

10. The method according to claim 9 , further comprising

performing smoothing of the at least one parameter over a predetermined period of time;

generating, based on the smoothing, an ingestion score value;

comparing the generated ingestion score value with a threshold ingestion score value; and

performing, based on the comparing, the adjusting of the deduplication and replication rate.

11. The method according to claim 1 , wherein the received data is stored temporarily in a temporary storage location of the grid server.

12. The method according to claim 11 , further comprising transferring the received data from the temporary storage location of the grid server for deduplication and replication by the grid server.

13. The method according to claim 11 , further comprising performing, based on the received data temporarily stored in the temporary storage location of the grid server, at an increased speed at least one of the following: generating at least one backup tape, generating at least one virtual machine image, performing an instant recovery of the received data from the temporary storage location, and restoring at least one file.

14. A system comprising:

at least one programmable processor; and

a machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

sensing, by a processor of a grid server in a plurality of grid servers, an ingestion rate of data received by the grid server for a backup, the ingestion rate corresponding to a speed of transferring of the received data into a memory of the grid server, the data is received from at least one data management application communicatively coupled to the plurality of grid servers;

monitoring, by the processor of the grid server, a deduplication and replication rate of the received data, the ingestion rate is being dependent on the deduplication and replication rate,

the deduplication including deduplicating the received data by the grid server based on a determination that at least one zone stamp of a zone within the received data is similar to at least another zone stamp of another zone within the received data, and

the replication including replicating the deduplicated received data to another server in the plurality of servers, the another server being communicatively coupled to the grid server using a communications network; and

adjusting, by the processor of the grid server, the deduplication and replication rate based on the sensed ingestion rate, wherein the adjusting includes:

monitoring at least one of the following: a backup process, deduplication and replication processes, a restore process, a copy-to-tape process, a cross-replication process, a purge process, a rebalance process as performed by the grid server; and

performing, based on the monitoring, sampling of at least one of the following: the ingestion rate, and the deduplication and replication rate;

performing ingestion, deduplication and replication of data in parallel; and

minimizing, based on the sampling and parallel performance of ingestion, deduplication and replication of data, a recovery point objective time associated with the received data, the recovery point objective time representing a difference between times when the data is received into the memory of the grid server and when the data is received into a memory of another grid server.

15. The system according to claim 14 , wherein the adjusting further includes increasing the deduplication and replication rate when the ingestion rate is less than or equal to a first predetermined threshold.

16. The system according to claim 15 , wherein the adjusting further includes unchanging the deduplication and replication rate when the ingestion rate is greater than the first predetermined threshold and less than or equal to a second predetermined threshold.

17. The system according to claim 16 , wherein the adjusting further includes reducing the deduplication and replication rate when the ingestion rate is greater than the second predetermined threshold and less than or equal to a third predetermined threshold.

18. The system according to claim 17 , wherein the adjusting further include setting the deduplication and replication rate to a predetermined minimum deduplication and replication rate when the ingestion rate is greater than the third predetermined threshold.

19. The system according to claim 14 , wherein the operations further comprise

performing, based on the monitoring and the sampling, using the grid server, at least one of the following: increasing the deduplication and replication rate, decreasing the deduplication and replication rate, increasing the ingestion rate, decreasing the ingestion rate, increasing a protection rate corresponding a combination of the deduplication rate and the replication rate, decreasing the protection rate, and/or any combination thereof.

20. The system according to claim 14 , wherein the sampling is performed based on at least one of the following: periodically, automatically, and during a predetermined time interval.

21. The system according to claim 14 , wherein the memories of the grid server and another grid server include at least one of the following: a non-volatile storage medium and a volatile storage medium.

22. The system according to claim 14 , wherein the sampling is performed based on at least one of the following parameter: amount of data stored in a network socket receive buffer associated with the grid server, data packet round trip time associate with transmission of data between at least two grid servers, transmission control protocol advertised receive buffer size, bandwidth of a network including the grid server, a time to complete at least one disk read/write input/output request associated with the grid server, and/or any combination thereof.

23. The system according to claim 22 , wherein the operations further comprise

performing smoothing of the at least one parameter over a predetermined period of time;

generating, based on the smoothing, an ingestion score value;

comparing the generated ingestion score value with a threshold ingestion score value; and

performing, based on the comparing, the adjusting of the deduplication and replication rate.

24. The system according to claim 14 , wherein the received data is stored temporarily in a temporary storage location of the grid server.

25. The system according to claim 24 , wherein the operations further comprise transferring the received data from the temporary storage location of the grid server for deduplication and replication by the grid server.

26. The system according to claim 24 , wherein the operations further comprise performing, based on the received data temporarily stored in the temporary storage location of the grid server, at an increased speed at least one of the following: generating at least one backup tape, generating at least one virtual machine image, performing an instant recovery of the received data from the temporary storage location, and restoring at least one file.

27. A computer program product comprising a machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

sensing, by a processor of a grid server in a plurality of grid servers, an ingestion rate of data received by the grid server for a backup, the ingestion rate corresponding to a speed of transferring of the received data into a memory of the grid server, the data is received from at least one data management application communicatively coupled to the plurality of grid servers;

monitoring, by the processor of the grid server, a deduplication and replication rate of the received data, the ingestion rate is being dependent on the deduplication and replication rate,

the deduplication including deduplicating the received data by the grid server based on a determination that at least one zone stamp of a zone within the received data is similar to at least another zone stamp of another zone within the received data, and

the replication including replicating the deduplicated received data to another server in the plurality of servers, the another server being communicatively coupled to the grid server using a communications network; and

adjusting, by the processor of the grid server, the deduplication and replication rate based on the sensed ingestion rate, wherein the adjusting includes:

monitoring at least one of the following: a backup process, deduplication and replication processes, a restore process, a copy-to-tape process, a cross-replication process, a purge process, a rebalance process as performed by the grid server; and

performing, based on the monitoring, sampling of at least one of the following: the ingestion rate, and the deduplication and replication rate;

performing ingestion, deduplication and replication of data in parallel; and

minimizing, based on the sampling and parallel performance of ingestion, deduplication and replication of data, a recovery point objective time associated with the received data, the recovery point objective time representing a difference between times when the data is received into the memory of the grid server and when the data is received into a memory of another grid server.

28. The computer program product according to claim 27 , wherein the adjusting further includes increasing the deduplication and replication rate when the ingestion rate is less than or equal to a first predetermined threshold.

29. The computer program product according to claim 28 , wherein the adjusting further includes unchanging the deduplication and replication rate when the ingestion rate is greater than the first predetermined threshold and less than or equal to a second predetermined threshold.

30. The computer program product according to claim 29 , wherein the adjusting further includes reducing the deduplication and replication rate when the ingestion rate is greater than the second predetermined threshold and less than or equal to a third predetermined threshold.

31. The computer program product according to claim 30 , wherein the adjusting further include setting the deduplication and replication rate to a predetermined minimum deduplication and replication rate when the ingestion rate is greater than the third predetermined threshold.

32. The computer program product according to claim 27 , wherein the operations further comprise

performing, based on the monitoring and the sampling, using the grid server, at least one of the following: increasing the deduplication and replication rate, decreasing the deduplication and replication rate, increasing the ingestion rate, decreasing the ingestion rate, increasing a protection rate corresponding a combination of the deduplication rate and the replication rate, decreasing the protection rate, and/or any combination thereof.

33. The computer program product according to claim 27 , wherein the sampling is performed based on at least one of the following: periodically, automatically, and during a predetermined time interval.

34. The computer program product according to claim 27 , wherein the memories of the grid server and another grid server include at least one of the following: a non-volatile storage medium and a volatile storage medium.

35. The computer program product according to claim 27 , wherein the sampling is performed based on at least one of the following parameter: amount of data stored in a network socket receive buffer associated with the grid server, data packet round trip time associate with transmission of data between at least two grid servers, transmission control protocol advertised receive buffer size, bandwidth of a network including the grid server, a time to complete at least one disk read/write input/output request associated with the grid server, and/or any combination thereof.

36. The computer program product according to claim 35 , wherein the operations further comprise

performing smoothing of the at least one parameter over a predetermined period of time;

generating, based on the smoothing, an ingestion score value;

comparing the generated ingestion score value with a threshold ingestion score value; and

performing, based on the comparing, the adjusting of the deduplication and replication rate.

37. The computer program product according to claim 27 , wherein the received data is stored temporarily in a temporary storage location of the grid server.

38. The computer program product according to claim 37 wherein the operations further comprise transferring the received data from the temporary storage location of the grid server for deduplication and replication by the grid server.

39. The computer program product according to claim 37 , wherein the operations further comprise performing, based on the received data temporarily stored in the temporary storage location of the grid server, at an increased speed at least one of the following: generating at least one backup tape, generating at least one virtual machine image, performing an instant recovery of the received data from the temporary storage location, and restoring at least one file.