Request manager framework
Techniques for managing sequencing requests for storage node operations based on types of operations being sequenced. The techniques manage sequencing requests for different types of operations, such as backup and recovery operations, replication operations, migration operations, and so on, in a manner that avoids overwhelming storage node capacity. The techniques include receiving a sequencing request for an operation of a specified operation type at a request manager of a storage node and determining whether a capacity of the storage node is available for running the operation by a capacity manager for the specified operation type, in which the capacity manager is provided by a storage client and implemented on the storage node separate from the request manager. In this way, throttling, limiting, and/or prioritization requirements of the operation of the specified operation type can be defined by the storage client in a manner agnostic to the request manager.
1. A method of managing sequencing requests for storage node operations, comprising:
receiving, at a storage node from a storage client, a first capacity manager plugin for a request manager framework implemented on the storage node, the first capacity manager plugin defining rules for managing a capacity of the storage node for a specified first operation type, the request manager framework including a request manager component;
receiving a first sequencing request for a first operation of the specified first operation type at the request manager component of the request manager framework; and
initiating a first transaction involving at least the request manager component and the first capacity manager plugin, the first transaction comprising:
determining, by the first capacity manager plugin, whether a capacity of the storage node is available for running the first operation of the specified first operation type, the request manager framework implementing the first capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified first operation type;
in response to the capacity of the storage node being determined available for running the first operation, granting, by the request manager component, the first sequencing request for the first operation of the specified first operation type; and
in response to the capacity of the storage node being determined not available for running the first operation, postponing granting, by the request manager component, the first sequencing request for the first operation of the specified first operation type.
2. The method of claim 1 further comprising:
receiving, at the storage node from the storage client, a first sequencing request persister plugin for the request manager framework, the first sequencing request persister plugin defining capabilities and/or priorities for managing persistence of sequencing requests for the specified first operation type, the request manager framework being further agnostic to the defined capabilities and/or priorities for managing the persistence of sequencing requests for the specified first operation type, the first transaction further involving the first sequencing request persister plugin, the first transaction further comprising:
in response to the capacity of the storage node being determined available for running the first operation, initiating, by the request manager component, writing a persistent record of the first sequencing request to a repository for the specified first operation type by the first sequencing request persister plugin.
3. The method of claim 1 further comprising:
receiving a second sequencing request for a second operation of the specified first operation type at the request manager component of the request manager framework.
4. The method of claim 3 further comprising: initiating, by the request manager component, writing a persistent record of the second sequencing request to a repository for the specified first operation type by the first sequencing request persister plugin.
5. The method of claim 4 further comprising:
initiating, by the request manager component, releasing the capacity of the storage node for running the first operation by the first capacity manager plugin for the specified first operation type by the first capacity manager plugin.
6. The method of claim 5 further comprising:
in response to the capacity of the storage node for running the first operation being released, initiating a second transaction involving at least the request manager component and the first capacity manager plugin, the second transaction comprising:
initiating, by the request manager component, obtaining the persistent record of the second sequencing request from the repository for the specified first operation type by the first sequencing request persister plugin.
7. The method of claim 6 wherein the second transaction further comprises:
determining, by the first capacity manager plugin, whether a capacity of the storage node is available for running the second operation of the specified first operation type.
8. The method of claim 7 wherein the second transaction further comprises:
in response to that the capacity of the storage node being determined available for running the second operation, granting, by the request manager component, the second sequencing request for the second operation of the specified first operation type.
9. The method of claim 8 wherein the second transaction further comprises:
in response to the capacity of the storage node is being determined not available for running the second operation, postponing granting, by the request manager component, the second sequencing request for the second operation of the specified first operation type.
10. The method of claim 1 further comprising:
receiving, at the storage node from the storage client, a second capacity manager plugin for the request manager framework, the second capacity manager plugin defining rules for managing a capacity of the storage node for a specified second operation type, the request manager component being further agnostic to the defined rules for manacling the capacity of the storage node for the specified second operation type; and
receiving a second sequencing request for a second operation of the specified second operation type at the request manager component of the request manager framework.
11. The method of claim 10 further comprising:
receiving, at the storage node from the storage client, a second sequencing request persister plugin for the request manager framework, the second sequencing request persister plugin defining capabilities and/or priorities for managing persistence of sequencing requests for the specified second operation type, the request manager framework being further agnostic to the defined capabilities and/or priorities for managing the persistence of sequencing requests for the specified second operation type, the first transaction further comprising:
initiating, by the request manager component, writing a first persistent record of the first sequencing request to a first repository for the specified first operation type by the first sequencing request persister plugin; and
initiating a second transaction involving at least the request manager component and the second sequencing request persister plugin, the second transaction comprising:
initiating, by the request manager component, writing a second persistent record of the second sequencing request to a second repository for the specified second operation type by the second sequencing request persister plugin.
12. The method of claim 10 further comprising:
initiating a second transaction involving at least the request manager component and the second capacity manager plugin, the second transaction comprising:
determining, by the second capacity manager plugin, whether capacity of the storage node is available for running the second operation of the specified second operation type, the request manager framework implementing the second capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified second operation type.
13. The method of claim 12 wherein the second transaction further comprises:
in response to the capacity of the storage node being determined available for running the second operation, granting, by the request manager component, the second sequencing request for the second operation of the specified second operation type.
14. The method of claim 13 wherein the second transaction further comprises:
in response to the capacity of the storage node is being determined not available for running the second operation, postponing granting, by the request manager component, the second sequencing request for the second operation of the specified second operation type.
15. A system for managing sequencing requests for storage node operations, comprising:
a memory; and
processing circuitry configured to run executable code out of the memory to:
receive, at a storage node, from a storage client, a first capacity manager plugin for a request manager framework implemented on the storage node, the first capacity manager plugin defining rules for managing a capacity of the storage node for a specified first operation type, the request manager framework including a request manager component;
receive a first sequencing request for a first operation of the specified first operation type at the request manager component of the request manager framework; and
initiate a first transaction involving at least the request manager component and the first capacity manager plugin, the first transaction comprising:
determine, by the first capacity manager plugin, whether a capacity of the storage node is available for running the first operation of the specified first operation type, the request manager framework implementing the first capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified first operation type;
in response to the capacity of the storage node being determined available for running the first operation, grant, by the request manager component, the first sequencing request for the first operation of the specified first operation type; and
in response to the capacity of the storage node being determined not available for running the first operation, postpone granting, by the request manager component, the first sequencing request for the first operation of the specified first operation type.
16. The system of claim 15 wherein the processing circuitry is further configured to run the executable code out of the memory to:
receive, at the storage node from the storage client, a second capacity manager plugin for the request manager framework, the second capacity manager plugin defining rules for managing a capacity of the storage node for a specified second operation type, the request manager component being further agnostic to the defined rules for managing the capacity of the storage node for the specified second operation type; and
receive a second sequencing request for a second operation of the specified second operation type at the request manager component of the request manager framework;
receive, at the storage node from the storage client, a second sequencing request persister plugin for the request manager framework, the second sequencing request persister plugin defining capabilities and/or priorities for managing persistence of sequencing requests for the specified second operation type, the request manager framework being further agnostic to the defined capabilities and/or priorities for managing the persistence of sequencing requests for the specified second operation type, the first transaction further comprising:
initiate, by the request manager component, writing a first persistent record of the first sequencing request to a first repository for the specified first operation type by the first sequencing request persister plugin; and
initiate a second transaction involving at least the request manager component and the second sequencing request persister plugin, the second transaction comprising:
initiate, by the request manager component, writing a second persistent record of the second sequencing request to a second repository for the specified second operation type by the second sequencing request persister plugin.
17. The system of claim 16 wherein the processing circuitry is further configured to run the executable code out of the memory to:
initiate a third transaction involving at least the request manager component and the second capacity manager plugin, the third transaction comprising:
determine, by the second capacity manager plugin, whether capacity of the storage node is available for running the second operation of the specified second operation type, the request manager framework implementing the second capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified second operation type.
18. A computer program product including a set of non-transitory, computer-readable media having instructions that, when executed by processing circuitry, cause the processing circuitry to perform a method comprising:
receiving, at a storage node from a storage client, a first capacity manager plugin for a request manager framework implemented on the storage node, the first capacity manager plugin defining rules for managing a capacity of the storage node for a specified first operation type, the request manager framework including a request manager component;
receiving a first sequencing request for a first operation of the specified first operation type at the request manager component of the request manager framework; and
initiating a first transaction involving at least the request manager component and the first capacity manager plugin, the first transaction comprising:
determining, by the first capacity manager plugin, whether a capacity of the storage node is available for running the first operation of the specified first operation type, the request manager framework implementing the first capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified first operation type;
in response to the capacity of the storage node being determined available for running the first operation, granting, by the request manager component, the first sequencing request for the first operation of the specified first operation type; and
in response to the capacity of the storage node being determined not available for running the first operation, postponing granting, by the request manager component, the first sequencing request for the first operation of the specified first operation type.
19. The computer program product of claim 18 wherein the method further comprises:
receiving, at the storage node from the storage client, a second capacity manager plugin for the request manager framework, the second capacity manager plugin defining rules for managing a capacity of the storage node for a specified second operation type, the request manager component being further agnostic to the defined rules for managing the capacity of the storage node for the specified second operation type;
receiving a second sequencing request for a second operation of the specified second operation type at the request manager component of the request manager framework;
receiving, at the storage node from the storage client, a second sequencing request persister plugin for the request manager framework, the second sequencing request persister plugin defining capabilities and/or priorities for managing persistence of sequencing requests for the specified second operation type, the request manager framework being further agnostic to the defined capabilities and/or priorities for managing the persistence of sequencing requests for the specified second operation type, the first transaction further comprising:
initiating, by the request manager component, writing a first persistent record of the first sequencing request to a first repository for the specified first operation type by the first sequencing request persister plugin; and
initiating a second transaction involving at least the request manager component and the second sequencing request persister plugin, the second transaction comprising:
initiating, by the request manager component, writing a second persistent record of the second sequencing request to a second repository for the specified second operation type by the second sequencing request persister plugin.
20. The computer program product of claim 19 wherein the method further comprises:
initiating a second transaction involving at least the request manager component and the second capacity manager plugin, the second transaction comprising:
determining, by the second capacity manager plugin, whether capacity of the storage node is available for running the second operation of the specified second operation type, the request manager framework implementing the second capacity manager plugin separate from the request manager component, which is agnostic to the defined rules for managing the capacity of the storage node for the specified second operation type.