Method and device for migrating data
Embodiments of the present disclosure relate to a method and device for migrating data. The method comprises identifying cold data in a primary storage system. The method further comprises, in response to determining that the cold data is in a non-compression state, obtaining the cold data from the primary storage system via a first interface, the first interface being configured for a user to access the primary storage system. The method further comprises obtaining, in response to determining the cold data is in a compression state, the cold data in the compression state from the primary storage system via a second interface that is different from the first interface. The method further comprises migrating the obtained cold data from the primary storage system to a secondary storage system.
1. A method of migrating data, comprising:
identifying, via a primary server, cold data in a primary storage system;
in response to determining that the cold data is in a non-compressed state, obtaining, via a first interface, the cold data from a primary storage system, the first interface being configured for a user to access the primary storage system;
in response to determining that the cold data is in a compressed state, obtaining, via a second interface, the cold data in the compressed state from the primary storage system, the second interface being different from the first interface;
preprocessing, via a secondary server, uncompressed data from the primary storage system prior to migrating the uncompressed data to a secondary storage system, wherein preprocessing the uncompressed data includes, at least in part, caching the uncompressed data, deleting duplicated uncompressed data, compressing the uncompressed data, and encrypting the uncompressed data;
adding a flag to metadata associated with the obtained cold data in the compressed state;
processing, via the secondary server, the obtained cold data in the compressed state without decompressing the obtained cold data in the compressed state;
migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system;
dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state, wherein dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state occurs after the cold data in the primary storage system is identified and before deleting cold data from the primary storage system; and
recalling, in response to receiving a request from a user for access to a part of uncompressed data corresponding to compressed cold data, only a portion of the compressed data that corresponds to the part of uncompressed data requested, wherein the portion of the compressed data is associated with at least one of the plurality of logical segments from the secondary storage system.
2. The method according to claim 1 , further comprising:
in response to receiving a request for access to a part of the uncompressed data, determining a state of the uncompressed data; and
in response to determining that the uncompressed data is in a migrated state, determining, based on the request, at least one of the plurality of logical segments that corresponds to the part of the uncompressed data.
3. The method according to claim 1 , wherein the cold data is organized as a file, a data block, or a data object.
4. The method according to claim 1 , wherein identifying, via a primary server, cold data in a primary storage system is based upon, at least in part, a predefined policy.
5. The method according to claim 4 , wherein the predefined policy includes an identification mode.
6. The method according to claim 1 , further comprising:
updating, via the primary server, contents of metadata associated with the cold data in the primary storage system after migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system.
7. The method according to claim 1 , further comprising:
storing information in the primary server regarding a position of the migrated obtained cold data in the non-compressed state and the obtained cold data in the compressed state in the secondary storage system, wherein the information comprises at least a Uniform Resource Locator (URL) with a universally unique identifier (UUID).
8. The method according to claim 1 , wherein the mapping between indexes of a plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state is implemented in the form of a mapping relation table and one or more of a matrix and a vector.
9. The method according to claim 1 , wherein the secondary storage system comprises a lower access performance in comparison to the primary storage system.
10. The method according to claim 1 , wherein the secondary storage system further comprises a larger storage space in comparison to the primary storage system.
11. The method according to claim 1 , wherein the primary storage system is an enterprise-level storage system and the secondary storage system is a cloud storage system.
12. An electronic device, comprising:
at least one processor; and
at least one memory coupled to the at least one processor and storing instructions to be executed by the at least one processor, the instructions, when executed by the at least one processor, causing the device to perform acts including:
identifying, via a primary server, cold data in a primary storage system;
in response to determining that the cold data is in a non-compressed state, obtaining, via a first interface, the cold data from a primary storage system, the first interface being configured for a user to access the primary storage system;
in response to determining that the cold data is in a compressed state, obtaining, via a second interface, the cold data in the compressed state from the primary storage system, the second interface being different from the first interface;
preprocessing, via a secondary server, uncompressed data from the primary storage system prior to migrating the uncompressed data to a secondary storage system, wherein preprocessing the uncompressed data includes, at least in part, caching the uncompressed data, deleting duplicated uncompressed data, compressing the uncompressed data, and encrypting the uncompressed data;
adding a flag to metadata associated with the obtained cold data in the compressed state;
processing, via the secondary server, the obtained cold data in the compressed state without decompressing the obtained cold data in the compressed state;
migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system;
dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state, wherein dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state occurs after the cold data in the primary storage system is identified and before deleting cold data from the primary storage system; and
recalling, in response to receiving a request from a user for access to a part of uncompressed data corresponding to compressed cold data, only a portion of the compressed data that corresponds to the part of uncompressed data requested, wherein the portion of the compressed data is associated with at least one of the plurality of logical segments from the secondary storage system.
13. The device according to claim 12 , wherein the acts further include:
in response to receiving a request for access to a part of the uncompressed data, determining a state of the uncompressed data; and
in response to determining that the uncompressed data is in a migrated state, determining, based on the request, at least one of the plurality of logical segments that corresponds to the part of the uncompressed data.
14. The device according to claim 12 , wherein the cold data is organized as a file, a data block, or a data object.
15. A computer program product for migrating data, the computer program product comprising:
a non-transitory computer readable medium encoded with computer-executable program code for migrating data, the code configured to enable the execution of:
identifying, via a primary server, cold data in a primary storage system;
in response to determining that the cold data is in a non-compressed state, obtaining, via a first interface, the cold data from a primary storage system, the first interface being configured for a user to access the primary storage system;
in response to determining that the cold data is in a compressed state, obtaining, via a second interface, the cold data in the compressed state from the primary storage system, the second interface being different from the first interface;
preprocessing, via a secondary server, uncompressed data from the primary storage system prior to migrating the uncompressed data to a secondary storage system, wherein preprocessing the uncompressed data includes, at least in part, caching the uncompressed data, deleting duplicated uncompressed data, compressing the uncompressed data, and encrypting the uncompressed data;
adding a flag to metadata associated with the obtained cold data in the compressed state;
processing, via the secondary server, the obtained cold data in the compressed state without decompressing the obtained cold data in the compressed state;
migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system;
dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state, wherein dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state occurs after the cold data in the primary storage system is identified and before deleting cold data from the primary storage system; and
recalling, in response to receiving a request from a user for access to a part of uncompressed data corresponding to compressed cold data, only a portion of the compressed data that corresponds to the part of uncompressed data requested, wherein the portion of the compressed data is associated with at least one of the plurality of logical segments from the secondary storage system.
16. The computer program product according to claim 15 , wherein the code is further configured to enable the execution of:
in response to receiving a request for access to a part of the uncompressed data, determining a state of the uncompressed data; and
in response to determining that the uncompressed data is in a migrated state, determining, based on the request, at least one of the plurality of logical segments that corresponds to the part of the uncompressed data.
17. The computer program product according to claim 15 , wherein the cold data is organized as a file, a data block, or a data object.