Seamless data migration across databases
Techniques are disclosed for enabling the migration of data with minimized impact on consumers of the data. A data migration agent updates pointers to active data locations and coordinates a migration from a first data resource to a second data resource so that seamless migration may be carried out. A data access layer of a distributed computing environment can take advantage of the pointers and metadata written by the data migration agent to serve requests to resources of the distributed computing environment so that read availability is available regardless of migration status.
1. A method of seamless data migration, comprising:
while a pointer of a data locator indicates a source partition as a location of a data, transferring, by a data migration server, a replica of a database partition from the source partition to a target partition while maintaining read and write properties of the source partition, wherein changes in the source partition during transfer are tracked as a source delta;
while the pointer of the data locator indicates the source partition as the location of the data, disabling, by a data migration server, write operations on the source partition after the transfer of the replica is complete;
while the pointer of the data locator indicates the source partition as the location of the data, applying, by a data migration server, the source delta to the target partition upon disabling the write operations on the source partition; and
updating, by a data migration server, the pointer of the data locator to indicate the target partition as the location of the data after applying the source delta to the target partition.
2. The method of claim 1 , further comprising, after applying the source delta to the target partition, writing to metadata of the source partition an indication that the source partition is inactive and writing to metadata of the target partition an indication that the target partition is active.
3. The method of claim 1 , further comprising modifying metadata of the source partition and metadata of the target partition according to a migration status.
4. The method of claim 1 , further comprising enabling write operations on the target partition after applying the source delta to the target partition.
5. A method of seamless data migration, comprising:
transferring, by a data migration server, a replica of a database partition from a source partition to a target partition while maintaining read and write properties of the source partition, wherein changes in the source partition during transfer are tracked as a source delta;
disabling, by a data migration server, write operations on the source partition after the transfer of the replica is complete;
applying, by a data migration server, the source delta to the target partition upon disabling the write operations on the source partition;
updating, by a data migration server, a pointer of a data locator to indicate the target partition after applying the source delta to the target partition; and
modifying, by a data migration server, metadata of the source partition and metadata of the target partition according to a migration status, wherein the migration status comprises a migration full-synchronization phase, a migration delta synchronization phase, and a switchover phase to a normal operation.
6. The method of claim 5 , wherein modifying the metadata of the source partition and the metadata of the target partition at the switchover phase to the normal operation, comprises writing to the metadata of the source partition an indication that the source partition is inactive and writing to the metadata of the target partition an indication that the target partition is active after updating the pointer of the data locator to indicate the target partition.
7. The method of claim 5 , further comprising enabling write operations on the target partition after applying the source delta to the target partition.
8. The method of claim 5 , further comprising:
adjusting a length of time for the migration full-synchronization phase according to a predefined minimum duration of time corresponding to an amount of time a data access layer caches metadata of the database to which the source partition belongs.
9. The method of claim 5 , further comprising, after applying the source delta to the target partition, writing to metadata of the source partition an indication that the source partition is inactive and writing to metadata of the target partition an indication that the target partition is active.
10. A method of seamless data migration, comprising:
transferring, by a data migration server, a replica of a database partition from a source partition to a target partition while maintaining read and write properties of the source partition, wherein changes in the source partition during transfer are tracked as a source delta;
disabling, by a data migration server, write operations on the source partition after the transfer of the replica is complete;
applying, by a data migration server, the source delta to the target partition upon disabling the write operations on the source partition;
updating, by a data migration server, a pointer of a data locator to indicate the target partition after applying the source delta to the target partition; and
adjusting, by a data migration server, a length of time for a migration full-synchronization phase according to a predefined minimum duration of time corresponding to an amount of time a data access layer caches metadata of the database to which the source partition belongs.
11. The method of claim 10 , further comprising, after applying the source delta to the target partition, writing to metadata of the source partition an indication that the source partition is inactive and writing to metadata of the target partition an indication that the target partition is active.
12. The method of claim 10 , further comprising modifying metadata of the source partition and metadata of the target partition according to a migration status.
13. The method of claim 10 , further comprising enabling write operations on the target partition after applying the source delta to the target partition.