Prioritized backup segmenting
A method, article of manufacture, and apparatus for backing up data. The method for backing up data includes backing up an object to a repository, segmenting the object into a plurality of sub-objects, and assigning a priority to a sub-object. Segmenting the object may include segmenting the object based on file access patterns. Assigning a priority may include assigning a high priority based on file access patterns, or based on policy, among others.
1. A method for storing data, comprising:
using a processor to back up a plurality of data files to a repository;
using the processor to segment the plurality of data files into a corresponding plurality of segments based at least in part on an access pattern associated with each of at least a subset of the plurality of segments, wherein at least a portion of one of the plurality of data files is segmented into one or more segments of a first size based at least in part on an access pattern associated with a first segment;
using the processor to assign a priority to each of at least a subset of the plurality of segments; and
using the processor to store the assigned priorities in a data structure, wherein a process to recover the segments from the repository recovers the segments based at least in part on the assigned priorities, and the assigned priorities being updated in real-time with respect to a recovery of the segments based at least in part on an input or output of an application during the recovery of the segments.
2. The method of claim 1 , wherein the first segment has a size corresponding to the first size.
3. The method of claim 1 , wherein the priority of each segment is assigned based at least in part on the access pattern associated with that segment.
4. The method of claim 1 , further comprising:
receiving, from the application, an indication of an event associated with an access of one of the plurality of segments during the process to recover the segments; and
in response to receiving the indication of the event associated with the access of the one of the plurality of segments associated with one of the plurality of data files, updating the priorities respectively assigned to two or more of the plurality of segments corresponding to the one of the plurality of data files.
5. The method of claim 4 , wherein the indication of the event associated with the access of the one of the plurality of segments is associated with or corresponds to a request to access the one of the plurality of segments.
6. The method of claim 1 , wherein segmenting the plurality of data files includes segmenting the object into equal sized segments.
7. The method of claim 1 , wherein assigning a priority includes assigning a high priority to at least one of the plurality of segments based on the access pattern.
8. The method of claim 1 , wherein priority is assigned further based at least in part on policy.
9. The method of claim 1 , wherein priority is assigned further based at least in part on metadata.
10. The method of claim 1 , wherein the plurality of data files comprises a version of one or more of the plurality of data files.
11. The method of claim 1 , further comprising pinning at least one of the plurality of segments, wherein pinning the segments includes restricting a pinned sub-object to Remote I/O during a recovery operation.
12. The method of claim 1 wherein the data structure is stored in the repository.
13. The method of claim 1 , wherein the data structure comprises at least a portion of a header for each of the plurality of segments.
14. The method of claim 1 , wherein a first portion of the one of the plurality of data files is segmented into one or more segments of the first size and a second portion of the one of the plurality of data files is segmented into one or more segments of a second size based at least in part on a different criteria.
15. The method of claim 1 , wherein one of the plurality of data files is segmented into the corresponding plurality of segments further based at least in part on one or more of the following: metadata associated with the object, policy, and user command.
16. The method of claim 1 , wherein the input or output of the application during the recovery of the segments corresponds to an input received from a user during the recovery.
17. A system for storing data, comprising:
a storage device;
at least one processor and a memory to store instructions, the instructions are executed by the at least one processor to perform:
back up a plurality of data files to a repository;
segment the plurality of data files into a corresponding plurality of segments based at least in part on an access pattern associated with each of at least a subset of the plurality of segments, wherein at least a portion of one of the plurality of data files is segmented into one or more segments of a first size based at least in part on an access pattern associated with a first segment;
assign a priority to each of at least a subset of the plurality of segments; and
store the assigned priorities in a data structure, wherein a process to recover the segments from the repository accesses the data structure recovers the segments based at least in part on the assigned priorities, and the assigned priorities being updated in real-time with respect to a recovery of the segments based at least in part on an input or output of an application during the recovery of the segments.
18. The system of claim 17 , wherein the first segment has a size corresponding to the first size.
19. The system of claim 17 , wherein the priority of each segment is assigned based at least in part on the access pattern associated with that segment.
20. The system of claim 17 , wherein the instructions are further executed by the at least one processor to perform:
receive, from the application, an indication of an event associated with an access of one of the plurality of segments during the process to recover the segments; and
in response to receiving the indication of the event associated with the access of the one of the plurality of segments associated with one of the plurality of data files, update the priorities respectively assigned to two or more of the plurality of segments corresponding to the one of the plurality of data files.
21. A computer program product for storing data, comprising a non-transitory computer usable medium having machine readable code embodied to perform:
backing up a plurality of data files to a repository;
segmenting the plurality of data files into a corresponding plurality of segments based at least in part on an access pattern associated with each of at least a subset of the plurality of segments, wherein at least a portion of one of the plurality of data files is segmented into one or more segments of a first size based at least in part on an access pattern associated with a first segment;
assigning a priority to each of at least a subset of the plurality of segments; and
storing store the assigned priorities in a data structure, wherein a process to recover the segments from the repository accesses the data structure recovers the segments based at least in part on the assigned priorities, and the assigned priorities being updated in real-time with respect to a recovery of the segments based at least in part on an input or output of an application during the recovery of the segments.