Storage performance-based distribution of deduplicated data to nodes within a clustered storage environment
Described is a system that provides the ability to distribute backup data according to performance characteristics associated with nodes that form a clustered storage environment. For example, the clustered storage environment may organize nodes into tiers that have certain storage performance-related characteristics such as read/write latency, deduplication processing efficiency, bandwidth, etc. Accordingly, the system may associate the backup data with an identifier for an appropriate storage performance tier. The identifier may then be utilized by the clustered storage environment to redirect storage operations to a node within the appropriate storage performance tier. Thus, the system allows for the distribution of backup data to particular storage performance tiers based on particular storage or workload requirements.
1. A system comprising:
one or more processor; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
receive backup data to be stored within a clustered storage environment, the clustered storage environment comprising a set of nodes organized into a set of storage performance tiers;
assign the backup data to a first storage performance tier based on at least one of a predicted workload or a predicted storage efficiency associated with the backup data, the predicted workload or the predicted storage efficiency being determined based on attributes associated with the backup data, the attributes including at least one of type of data, access frequency, storage duration, or deduplication ratio associated with the backup data;
associate the backup data with an identifier for the first storage performance tier and provide the identifier to the clustered storage environment;
initiate the clustered storage environment to store the backup data, the clustered storage environment redirecting a write operation for the backup data to a first node within the first storage performance tier based on the identifier provided to the cluster storage environment;
redistribute a first backup file storing the backup data on the first node within the first storage performance tier to a second node within a second storage performance tier based at least one of type of data, access frequency, or storage duration of the first backup file; and
update a global namespace to associate the first backup file with the second storage performance tier without altering any existing global namespace entries for other backup files.
2. The system of claim 1 , wherein the set of storage performance tiers includes one or more of a low latency storage tier, a high deduplication storage tier, a low deduplication storage tier, and a long term retention storage tier.
3. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
initiate storing the identifier as part of an entry to the global namespace for the first backup file storing the backup data on the first node, the global namespace maintained by the clustered storage environment and storing entries for all backup files stored on the set of nodes.
4. The system of claim 1 , wherein the redistributing the first backup file includes determining the first backup file is associated with a high access frequency, and wherein the second storage performance tier is associated with a lower latency than the first storage performance tier.
5. The system of claim 3 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
determine a storage capacity of the first storage performance tier has reached a predetermined threshold;
provision an additional node to the first storage performance tier; and
update the global namespace to include an additional branch of a namespace tree for the additional node without altering any existing global namespace entries for other backup files.
6. The system of claim 1 , wherein providing the identifier to the clustered storage environment includes providing the identifier as an input to an operation for initiating the clustered storage environment to store the backup data.
7. The system of claim 1 , wherein providing the identifier to the clustered storage environment includes storing the identifier as an attribute of a first backup file storing the backup data on the first node within the first storage performance tier.
8. A method comprising:
receiving backup data to be stored within a clustered storage environment, the clustered storage environment comprising a set of nodes organized into a set of storage performance tiers;
assigning the backup data to a first storage performance tier based on at least one of a predicted workload or a predicted storage efficiency associated with the backup data, the predicted workload or the predicted storage efficiency being determined based on attributes associated with the backup data, the attributes including at least one of type of data, access frequency, storage duration, or deduplication ratio associated with the backup data;
associating the backup data with an identifier for the first storage performance tier and provide the identifier to the clustered storage environment;
initiating the clustered storage environment to store the backup data, the clustered storage environment redirecting a write operation for the backup data to a first node within the first storage performance tier based on the identifier provided to the cluster storage environment;
redistributing a first backup file storing the backup data on the first node within the first storage performance tier to a second node within a second storage performance tier based at least one of type of data, access frequency, or storage duration of the first backup file; and
updating a global namespace to associate the first backup file with the second storage performance tier without altering any existing global namespace entries for other backup files.
9. The method of claim 8 , wherein the set of storage performance tiers includes one or more of a low latency storage tier, a high deduplication storage tier, a low deduplication storage tier, and a long term retention storage tier.
10. The method of claim 8 , further comprising:
initiating storing the identifier as part of an entry to the global namespace for the first backup file storing the backup data on the first node, the global namespace maintained by the clustered storage environment and storing entries for all backup files stored on the set of nodes.
11. The method of claim 8 , further comprising:
determining a storage capacity of the first storage performance tier has reached a predetermined threshold;
provisioning an additional node to the first storage performance tier; and
updating the global namespace to include an additional branch of a namespace tree for the additional node without altering any existing global namespace entries for other backup files.
12. The method of claim 8 , wherein providing the identifier to the clustered storage environment includes providing the identifier as an input to an operation for initiating the clustered storage environment to store the backup data, the clustered storage environment storing the identifier as an attribute of a first backup file storing the backup data on the first node within the first storage performance tier.
13. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
receive backup data to be stored within a clustered storage environment, the clustered storage environment comprising a set of nodes organized into a set of storage performance tiers;
assign the backup data to a first storage performance tier based on at least one of a predicted workload or a predicted storage efficiency associated with the backup data, the predicted workload or the predicted storage efficiency being determined based on attributes associated with the backup data, the attributes including at least one of type of data, access frequency, storage duration, or deduplication ratio associated with the backup data;
associate the backup data with an identifier for the first storage performance tier and provide the identifier to the clustered storage environment;
initiate the clustered storage environment to store the backup data, the clustered storage environment redirecting a write operation for the backup data to a first node within the first storage performance tier based on the identifier provided to the cluster storage environment;
redistribute a first backup file storing the backup data on the first node within the first storage performance tier to a second node within a second storage performance tier based at least one of type of data, access frequency, or storage duration of the first backup file; and
update a global namespace to associate the first backup file with the second storage performance tier without altering any existing global namespace entries for other backup files.
14. The computer program product of claim 13 , wherein the set of storage performance tiers includes one or more of a low latency storage tier, a high deduplication storage tier, a low deduplication storage tier, and a long term retention storage tier.
15. The computer program product of claim 13 , wherein the program code includes further instructions to:
initiate storing the identifier as part of an entry to the global namespace for the first backup file storing the backup data on the first node, the global namespace maintained by the clustered storage environment and storing entries for all backup files stored on the set of nodes.
16. The computer program product of claim 15 , wherein the program code includes further instructions to:
determine a storage capacity of the first storage performance tier has reached a predetermined threshold;
provision an additional node to the first storage performance tier; and
update the global namespace to include an additional branch of a namespace tree for the additional node without altering any existing global namespace entries for other backup files.
17. The computer program product of claim 13 , wherein providing the identifier to the clustered storage environment includes providing the identifier as an input to an operation for initiating the clustered storage environment to store the backup data, the clustered storage environment storing the identifier as an attribute of a first backup file storing the backup data on the first node within the first storage performance tier.