Transferring or migrating portions of data objects, such as block-level data migration or chunk-based data migration
A system and method for migrating data objects based on portions of the data objects is described. The system may transfer portions of files, folders, and other data objects from primary storage to secondary storage based on certain criteria, such as time-based criteria, age-based criteria, and so on. An increment may be one or more blocks of a data object, or one or more chunks of a data object, or other segments that combine to form or store a data object. For example, the system identifies one or more blocks of a data object that satisfy a certain criteria, and migrates the identified blocks. The system may determine that a certain number of blocks of a file have not been modified or called by a file system in a certain time period, and migrate these blocks to secondary storage.
1. At least one non-transitory, computer-readable storage medium comprising instructions, which when executed by at least one data processor, enable restoration from secondary storage of a portion of a file in a data storage system, comprising:
receiving, via a user interface associated with a file system, a request from a user to modify a portion of a file in the file system,
wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;
identifying one or more data blocks within the second storage device that contain data associated with the portion of the file requested to be modified;
retrieving from the second storage device the one or more data blocks without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;
presenting to the user data contained by the one or more data blocks retrieved from the second storage device, via the user interface associated with the file system;
upon receiving input from the user to modify the portion of the file, transferring data associated with the received input for storage by one of the first storage device and the second storage device; and
maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the received input to modify the portion of the file.
2. The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the data structure comprises one or more of: a bitmap, a table, and a log.
3. The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.
4. The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the identifying one or more data blocks identifies one or more chunks within the second storage device.
5. The at least one non-transitory, computer-readable storage medium of claim 1 , further comprising:
dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of: a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and
updating an index associated with the file to include information associating the identified one or more data blocks with the second storage device.
6. A method in a data storage system for restoring a portion of a file, the method comprising:
receiving, via a user interface associated with a file system that comprises a file, a request from a user to modify only a portion of the file in the file system,
wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;
determining one or more data blocks stored within the second storage device that contain data associated with the portion of the file requested to be modified;
retrieving from the second storage device the one or more data blocks that contain data associated with the portion of the file requested to be modified without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;
in response to the request, presenting to the user via the user interface associated with the file system one or more data blocks retrieved from the second storage device;
upon receiving input from the user, via the user interface, to modify the portion of the file, transferring data blocks that were changed by the received input for storage by one of the first storage device and the second storage device; and
maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the received input to modify the portion of the file.
7. The method of claim 6 , wherein the data structure at the intermediate component comprises a bitmap.
8. The method of claim 6 , wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.
9. The method of claim 6 , wherein the identifying one or more data blocks identifies one or more chunks within the second storage device.
10. The method of claim 6 , further comprising:
dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of: a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and
updating an index associated with the file to include information associating the identified one or more data blocks with the storage device.
11. A system for restoring a portion of a file, the system comprising:
at least one processor;
means, at a file system, for receiving a request from a user to modify only a portion of a file in the file system,
wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;
means for identifying one or more data blocks stored within the second storage device that contain data associated with the portion of the file requested to be modified;
means for retrieving from the second storage device the one or more data blocks without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;
means for presenting in a user interface the portion of the file requested to be modified, including the data blocks retrieved from the second storage device;
means for transferring data blocks that were changed by input received from the user to modify the portion of the file, for storage by one or more of the first storage device and the second storage device; and
means for maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the input to modify the portion of the file.
12. The system of claim 11 , wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.
13. The system of claim 11 , wherein the means for identifying one or more data blocks identifies one or more chunks within the second storage device.
14. The system of claim 11 , wherein the data structure is configured as a bitmap.
15. The system of claim 11 , further comprising:
means for dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of: a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and
means for updating an index associated with the file to include information associating the identified one or more data blocks with the second storage device.
16. The system of claim 11 , wherein the file is a slide presentation and wherein the data contained by the identified one or more data blocks includes data associated with a single slide of the slide presentation.
17. The system of claim 11 , wherein the file is a .pst file for an email mailbox associated with the user and wherein the identified one or more data blocks comprise one or more emails within the .pst file.