Methods and apparatuses for data migration of a storage device
Embodiments of the present disclosure provide methods and apparatuses for data migration of storage devices including registering at least one executing unit for data migration, each of the at least one executing unit corresponding to description file; extracting and storing information contained in the description file corresponding to each of the at least one executing unit; receiving a data migration request from a user; in response to the data migration request from the user, selecting an executing unit for data migration of the user at least based on part of the stored information contained in the description file; and scheduling an instance of the selected executing unit to execute data migration of the user. The methods or apparatuses according to embodiments of the present disclosure can implement, in a uniform and scalable manner, data migration for various formats, various performance requirements, and application scenarios.
1. A method for data migration, comprising:
registering at least one executing unit for data migration, each of the at least one executing unit corresponding to a description file;
extracting and storing information contained in the description file corresponding to each of the at least one executing unit;
receiving a data migration request from a user, wherein the data migration request includes an indication of a source address and a target address for data migration, and information about a performance requirement for the data migration;
in response to the data migration request from the user, selecting an executing unit for data migration of the user at least based on part of the stored information contained in the description file;
scheduling an instance of the selected executing unit to perform the data migration of the user;
monitoring a performance of the instance of the selected executing unit, wherein monitoring the performance of the instance of the selected executing unit includes:
detecting at least one of a failure of the instance of the selected executing unit and a performance abnormality of the instance of the selected executing unit;
upon detecting the failure of the instance of the selected executing unit, at least one of re-starting the instance of the selected executing unit, and killing the instance of the selected executing unit and applying a new resource to replace the instance of the selected executing unit upon which the failure was detected; and
upon detecting a performance abnormality of the instance of the selected executing unit, at least one of launching one or more instances of the selected executing unit on an existing resource, and increasing an amount of resources being applied from a resource pool, the launching of the one or more instances of the selected executing unit and increasing of the amount of resources being respective to a degree of degradation associated with the performance abnormality;
transmitting a data migration response to the user; and
receiving, from the user, a query for at least one of a running state and a running log of the data migration.
2. The method according to claim 1 , wherein each of the at least one executing unit is encapsulated into a container.
3. The method according to claim 1 , wherein the description file indicates at least one of the following of a corresponding executing unit:
a type,
a capacity,
a configuration,
a supported data format,
a running requirement, and
a supportable performance.
4. The method according to claim 1 , wherein the data migration request further includes information about at least one of:
whether deleting duplications is required during the data migration,
whether to perform at least one of compression during the data migration and a supported compression format, and
whether to perform at least one of encryption during the data migration and a supported encryption manner.
5. The method according to claim 1 , wherein one of the at least one executing unit is further configured to execute at least one of: data extracting, data buffering, data checking, duplicate data deleting, data compressing, data encrypting, and data transmitting.
6. The method according to claim 1 , wherein selecting an executing unit for data migration of the user at least based on part of the stored information contained in the description file comprises:
analyzing the data migration request from the user; and
determining an executing unit matching the data migration request based on the analyzing and the part of the stored information contained in the description file.
7. The method according to claim 6 , wherein selecting an executing unit for data migration of the user at least based on part of the stored information contained in the description file further comprise:
determining, as a most matching executing unit, an instance of a running executing unit from executing units matching the data migration request.
8. The method according to claim 6 , wherein selecting an executing unit for data migration of the user at least based on part of the stored information contained in the description file further comprises:
evaluating resources required for running of the executing unit; and
determining an executing unit matching the data migration request further based on the evaluating.
9. The method according to claim 1 , further comprising at least one of:
displaying at least one of the running state and the running log of the data migration of the user, in response to the query from the user; and
receiving, from the user, data to be migrated.
10. The method according to claim 1 , further comprising:
updating at least one of a progress, a state, a log, a resource usage, and a performance of the scheduled instance of the selected executing unit.
11. The method according to claim 10 , further comprising:
monitoring at least one of a progress, a state, a log, and a resource usage of the scheduled instance of the selected executing unit; and
increasing, reducing, or changing the scheduled instance of the selected executing unit based on the monitoring.
12. A method for data migration, comprising:
transmitting a data migration request to a data migration device configured to perform data migration, wherein the data migration request includes an indication of a source address and a target address for the data migration, and at least one of the following information:
a performance requirement for the data migration,
whether deleting duplications is required during the data migration,
whether to perform at least one of compression during the data migration and a supported compression format, and
whether to perform at least one of encryption during the data migration and a supported encryption manner;
selecting an executing unit for the data migration from the data migration device at least based on part of the stored information contained in a description file;
scheduling an instance of the selected executing unit to perform the data migration from the data migration device;
monitoring a performance of the instance of the selected executing unit, wherein monitoring the performance of the instance of the selected executing unit includes:
detecting at least one of a failure of the instance of the selected executing unit and a performance abnormality of the instance of the selected executing unit;
upon detecting the failure of the instance of the selected executing unit, at least one of re-starting the instance of the selected executing unit, and killing the instance of the selected executing unit and applying a new resource to replace the instance of the selected executing unit upon which the failure was detected; and
upon detecting a performance abnormality of the instance of the selected executing unit, at least one of launching one or more instances of the selected executing unit on an existing resource, and increasing an amount of resources being applied from a resource pool, the launching of the one or more instances of the selected executing unit and increasing of the amount of resources being respective to a degree of degradation associated with the performance abnormality;
transmitting a data migration response to the user; and
receiving, from the user, a query for at least one of a running state and a running log of the data migration.
13. The method according to claim 12 , further comprising:
receiving, from the data migration device, at least one of a running state and a running log of the data migration.
14. An apparatus for data migration, comprising a registering unit, a storing unit, a first receiving unit, a second receiving unit, a scheduling unit, a monitoring unit, a first transmitting unit, and at least one executing unit, wherein:
each of the at least one executing unit corresponds to a description file in a predefined format, and each executing unit is configured to perform data migration when being scheduled by the scheduling unit;
the registering unit is configured to register the at least one executing unit;
the storing unit is configured to extract and store information contained in the description file corresponding to each of the at least one executing unit;
the first receiving unit is configured to receive a data migration request from a user, wherein the data migration request includes an indication of a source address and a target address for data migration, and information about a performance requirement for the data migration;
the scheduling unit is configured to, in response to the data migration request from the user, select an executing unit for data migration of the user at least based on part of the stored information contained in the description file, and schedule an instance of the selected executing unit to perform the data migration of the user;
the monitoring unit is configured to monitor a performance of the instance of the selected executing unit, detect at least one of a failure of the instance of the selected executing unit and a performance abnormality of the instance of the selected executing unit and, upon detecting the failure of the instance of the selected executing unit, at least one of re-start the instance of the selected executing unit, and kill the instance of the selected executing unit and apply a new resource to replace the instance of the selected executing unit upon which the failure was detected, and upon detecting a performance abnormality of the instance of the selected executing unit, at least one of launch one or more instances of the selected executing unit on an existing resource, and increase an amount of resources being applied from a resource pool, the launch of the one or more instances of the selected executing unit and increase of the amount of resources being respective to a degree of degradation associated with the performance abnormality;
the first transmitting unit is configured to transmit a data migration response to the user; and
the second receiving unit is configured to receive, from the user, a query for at least one of a running state and a running log of the data migration.
15. The apparatus according to claim 14 , wherein each of the at least one executing unit is encapsulated into a container.
16. The apparatus according to claim 14 , wherein the description file indicates at least one of the following of a corresponding executing unit:
a type,
a capacity,
a configuration,
a supported data format,
a running requirement, and
supportable performance.
17. The apparatus according to claim 14 , wherein the data migration request further comprises information about at least one of:
whether deleting duplications is required to during the data migration,
whether to perform compression during at least one of the data migration and a supported compression format, and
whether to perform at least one of encryption during the data migration and a supported encryption manner.
18. The apparatus according to claim 14 , wherein one of the at least one executing unit is further configured to execute at least one of: data extracting, data buffering, data checking, duplicate data deleting, data compressing, data encrypting, and data transmitting.
19. The apparatus according to claim 14 , wherein the scheduling unit is further configured to:
analyze a data migration request from the user,
determine an executing unit matching the data migration request based on the analyzing and the part of the stored information contained in the description file, and
schedule an instance of the determined executing unit to perform the data migration.