Limiting bandwidth usage on asynchronous replication sessions for a destination storage system

1. A method for implementing bandwidth control on asynchronous replication sessions for a destination storage system, the method comprising:

receiving a request to set a bandwidth limit on replication sessions for the destination storage system;

updating, by a system manager of the destination storage system, a remote system object with a value representing the bandwidth limit;

creating a list of routing modules having ownership of each of a plurality of links going to the destination storage system;

sending, by the system manager, a value representing a maximum bandwidth per link to each of a plurality of control modules of the storage system;

calculating, by each of the control modules, a proportionate share of the bandwidth limit based on a number of slices owned by each of the control modules, the proportionate share of the bandwidth limit calculated by dividing the maximum bandwidth limit per link by a number of the plurality of control modules, the proportionate share of the bandwidth limit indicated by a budget value in a budget byte window for each of the routing modules corresponding to a respective control module;

prior to initiating a scan of the slices by an initiator thread with respect to a selected routing module, identifying a corresponding budget byte window having a budget value equal to or exceeding a predetermined value, allocating the budget value for the budget byte window to the initiator thread, and reducing the budget value corresponding to the budget byte window by the predetermined value;

initiating, by the routing module, the initiator thread for slices corresponding to the identified budget byte window, the initiator thread scanning the slices;

sending, by the routing module, values representing a number of bytes transmitted and corresponding number of pages to the respective control module resulting from the scanning; and

updating, by the control module, the corresponding budget byte window based on the values sent by the routing module.

2. The method of claim 1 , further comprising:

suppressing scanning, by the initiator thread, of slices corresponding to the control module identified with a budget byte window having a budget value that is less than the predetermined value.

3. The method of claim 1 , wherein the request to set the bandwidth limit includes a maximum amount of bandwidth to be used for replication sessions occurring at the destination storage system and a unique identifier of the destination storage system.

4. The method of claim 1 , wherein each of the plurality of control modules manages 256 slices of storage devices in the storage system and the budget byte window is represented as a page size multiplied by the 256 slices.

5. The method of claim 4 , wherein the budget byte window reflects a maximum number of address/hash pairs for a control-to-routing task between a control module and a routing module.

6. The method of claim 1 , further comprising:

during the scanning of one of the slices, creating control-to-routing tasks for pages up to a specified size limit;

if the scanning is not completed after the specified size limit is reached, marking the one of the slices as incomplete, saving the one of the slices and a last-offset pair in a corresponding bitmap; and

if the scanning is completed but the specified size limit is not reached, updating a corresponding budget byte window for the routing module to reflect a number of pages used.

7. The method of claim 1 , further comprising:

upon completion of the scanning of each of the plurality of control modules, updating corresponding budget byte windows for corresponding routing modules, the updating comprising:

for each of the plurality of control modules:

determining a number of bytes currently sent during the replication session as a running sum of bytes transmitted from routing modules plus pages inflight to the routing modules multiplied by a running average of bytes per page from the routing modules; and

determining an effective number of bytes to send as a time since a last reset in milliseconds multiplied by the budget value minus the number of bytes currently sent;

wherein the updating the corresponding budget byte window comprises updating the budget byte window with the effective number of bytes to send;

wherein an updated budget byte window is used to determine whether sufficient budget exists to send a selected control-to-routing task to a respective routing module.

8. A system for implementing bandwidth control on asynchronous replication sessions for a destination storage system, comprising;

a memory comprising computer-executable instructions; and

a processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:

receiving a request to set a bandwidth limit on replication sessions for the destination storage system;

updating, by a system manager of the destination storage system, a remote system object with a value representing the bandwidth limit;

creating a list of routing modules having ownership of each of a plurality of links going to the destination storage system;

sending, by the system manager, a value representing a maximum bandwidth per link to each of a plurality of control modules of the storage system;

calculating, by each of the control modules, a proportionate share of the bandwidth limit based on a number of slices owned by each of the control modules, the proportionate share of the bandwidth limit calculated by dividing the maximum bandwidth limit per link by a number of the plurality of control modules, the proportionate share of the bandwidth limit indicated by a budget value in a budget byte window for each of the routing modules corresponding to a respective control module;

prior to initiating a scan of the slices by an initiator thread with respect to a selected routing module, identifying a corresponding budget byte window having a budget value equal to or exceeding a predetermined value, allocating the budget value for the budget byte window to the initiator thread, and reducing the budget value corresponding to the budget byte window by the predetermined value;

initiating, by the routing module, the initiator thread for slices corresponding to the identified budget byte window, the initiator thread scanning the slices;

sending, by the routing module, values representing a number of bytes transmitted and corresponding number of pages to the respective control module resulting from the scanning; and

updating, by the control module, the corresponding budget byte window based on the values sent by the routing module.

9. The system of claim 8 , wherein the operations further comprise:

suppressing scanning, by the initiator thread, of slices corresponding to the control module identified with a budget byte window having a budget value that is less than the predetermined value.

10. The system of claim 8 , wherein the request to set the bandwidth limit includes a maximum amount of bandwidth to be used for replication sessions occurring at the destination storage system and a unique identifier of the destination storage system.

11. The system of claim 8 , wherein each of the plurality of control modules manages 256 slices of storage devices in the storage system and the budget byte window is represented as a page size multiplied by the 256 slices.

12. The system of claim 11 , wherein the budget byte window reflects a maximum number of address/hash pairs for a control-to-routing task between a control module and a routing module.

13. The system of claim 8 , wherein the operations further comprise:

during the scanning of one of the slices, creating control-to-routing tasks for pages up to a specified size limit;

if the scanning is not completed after the specified size limit is reached, marking the one of the slices as incomplete, saving the one of the slices and a last-offset pair in a corresponding bitmap; and

if the scanning is completed but the specified size limit is not reached, updating a corresponding budget byte window for the routing module to reflect a number of pages used.

14. The system of claim 8 , wherein the operations further comprise:

upon completion of the scanning of each of the plurality of control modules, updating corresponding budget byte windows for corresponding routing modules, the updating comprising:

for each of the plurality of control modules:

determining a number of bytes currently sent during the replication session as a running sum of bytes transmitted from routing modules plus pages inflight to the routing modules multiplied by a running average of bytes per page from the routing modules; and

determining an effective number of bytes to send as a time since a last reset in milliseconds multiplied by the budget value minus the number of bytes currently sent;

wherein the updating the corresponding budget byte window comprises updating the budget byte window with the effective number of bytes to send;

wherein an updated budget byte window is used to determine whether sufficient budget exists to send a selected control-to-routing task to a respective routing module.

15. A computer program product for implementing bandwidth control on asynchronous replication sessions for a destination storage system, the computer program product embodied on a non-transitory computer readable storage medium, and the computer program product including instructions that, when executed by a computer causes the computer to perform operations, the operations comprising:

receiving a request to set a bandwidth limit on replication sessions for the destination storage system;

updating, by a system manager of the destination storage system, a remote system object with a value representing the bandwidth limit;

creating a list of routing modules having ownership of each of a plurality of links going to the destination storage system;

sending, by the system manager, a value representing a maximum bandwidth per link to each of a plurality of control modules of the storage system;

calculating, by each of the control modules, a proportionate share of the bandwidth limit based on a number of slices owned by each of the control modules, the proportionate share of the bandwidth limit calculated by dividing the maximum bandwidth limit per link by a number of the plurality of control modules, the proportionate share of the bandwidth limit indicated by a budget value in a budget byte window for each of the routing modules corresponding to a respective control module;

prior to initiating a scan of the slices by an initiator thread with respect to a selected routing module, identifying a corresponding budget byte window having a budget value equal to or exceeding a predetermined value, allocating the budget value for the budget byte window to the initiator thread, and reducing the budget value corresponding to the budget byte window by the predetermined value;

initiating, by the routing module, the initiator thread for slices corresponding to the identified budget byte window, the initiator thread scanning the slices;

sending, by the routing module, values representing a number of bytes transmitted and corresponding number of pages to the respective control module resulting from the scanning; and

updating, by the control module, the corresponding budget byte window based on the values sent by the routing module.

16. The computer program product of claim 15 , wherein the operations further comprise:

suppressing scanning, by the initiator thread, of slices corresponding to the control module identified with a budget byte window having a budget value that is less than the predetermined value.

17. The computer program product of claim 15 , wherein the request to set the bandwidth limit includes a maximum amount of bandwidth to be used for replication sessions occurring at the destination storage system and a unique identifier of the destination storage system.

18. The computer program product of claim 15 , wherein each of the plurality of control modules manages 256 slices of storage devices in the storage system and the budget byte window is represented as a page size multiplied by the 256 slices; and

wherein the budget byte window reflects a maximum number of address/hash pairs for a control-to-routing task between a control module and a routing module.

19. The computer program product of claim 15 , wherein the operations further include:

during the scanning of one of the slices, creating control-to-routing tasks for pages up to a specified size limit;

if the scanning is not completed after the specified size limit is reached, marking the one of the slices as incomplete, saving the one of the slices and a last-offset pair in a corresponding bitmap; and

if the scanning is completed but the specified size limit is not reached, updating a corresponding budget byte window for the routing module to reflect a number of pages used.

20. The computer program product of claim 15 , wherein the operations further comprise:

upon completion of the scanning of each of the plurality of control modules, updating corresponding budget byte windows for corresponding routing modules, the updating comprising:

for each of the plurality of control modules:

determining a number of bytes currently sent during the replication session as a running sum of bytes transmitted from routing modules plus pages inflight to the routing modules multiplied by a running average of bytes per page from the routing modules; and

determining an effective number of bytes to send as a time since a last reset in milliseconds multiplied by the budget value minus the number of bytes currently sent;

wherein the updating the corresponding budget byte window comprises updating the budget byte window with the effective number of bytes to send;

wherein an updated budget byte window is used to determine whether sufficient budget exists to send a selected control-to-routing task to a respective routing module.