System and methods for live data migration
Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.
1. A system for live data migration, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to perform:
receiving a first query requesting a modification of at least one field in the first table of a first database schema during a live migration of the first table to a second table of a second database schema, the first table storing live data;
modifying, in response to receiving the first query, at least one corresponding field of the second table based on the first query;
updating a mutation table based on the modification, wherein the mutation table tracks one or more modifications to the second table;
receiving a second query for accessing the at least one field in the first table during the live migration of the first table to the second table of the second database schema; and
determining, in response to receiving the second query and based on the mutation table, the corresponding field of the second table has been modified;
providing, based on the determination the corresponding field of the second table has been modified, data from the at least one corresponding field of the second table in response to the second query.
2. The system of claim 1 , wherein the instructions further cause the system to perform:
receiving a third query for accessing at least one other field of the first table during the live migration of the first table to the second table of the second database schema;
determining, in response to receiving the third query and based on the mutation table, at least one corresponding other field of the second table has not been modified; and
providing, based on the determination the corresponding field of the second table has not been modified, data from the at least one field of the first table in response to the third query.
3. The system of claim 1 , wherein the first table becomes immutable during the live migration.
4. The system of claim 1 , wherein modifying the first table includes at least one of modifying data, inserting data, or deleting data in the first table.
5. The system of claim 1 , wherein updating the mutation table further comprises:
determining a row in the second table to which the modification was performed; and
updating the mutation table to indicate that the row has been modified.
6. The system of claim 1 , wherein the mutation table indicates the requested modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.
7. The system of claim 1 , wherein the first query requests to delete at least one row in the first table, wherein the instructions further cause the system to perform:
receiving a third query for accessing the at least one row during the live migration of the first table to the second table of the second database schema;
determining, in response to receiving the second query and based on the mutation table, at least one corresponding row of the second table has been deleted;
determining that the row has been deleted from the second table; and
providing a null value in response to the third query.
8. A computer-implemented method, the method being implemented by a computing system including one or more processors and storage media storing machine-readable instructions, the method comprising:
receiving a first query requesting a modification of at least one field in the first table of a first database schema during a live migration of the first table to a second table of a second database schema, the first table storing live data;
modifying, in response to receiving the first query, at least one corresponding field of the second table based on the first query;
updating a mutation table based on the modification, wherein the mutation table tracks one or more modifications to the second table;
receiving a second query for accessing the at least one field in the first table during the live migration of the first table to the second table of the second database schema; and
determining, in response to receiving the second query and based on the mutation table, the corresponding field of the second table has been modified;
providing, based on the determination the corresponding field of the second table has been modified, data from the at least one corresponding field of the second table in response to the second query.
9. The computer-implemented method of claim 8 , wherein the instructions further cause the system to perform:
receiving a third query for accessing at least one other field of the first table during the live migration of the first table to the second table of the second database schema;
determining, in response to receiving the third query and based on the mutation table, at least one corresponding other field of the second table has not been modified; and
providing, based on the determination the corresponding field of the second table has not been modified, data from the at least one field of the first table in response to the third query.
10. The computer-implemented method of claim 8 , wherein the first table becomes immutable during the live migration.
11. The computer-implemented method of claim 8 , wherein modifying the first table includes at least one of modifying data, inserting data, or deleting data in the first table.
12. The computer-implemented method of claim 8 , wherein updating the mutation table further comprises:
determining a row in the second table to which the modification was performed; and
updating the mutation table to indicate that the row has been modified.
13. The computer-implemented method of claim 12 , wherein the mutation table indicates the requested modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.
14. The computer-implemented method of claim 8 , wherein the first query requests to delete at least one row in the first table, wherein the instructions further cause the system to perform:
receiving a third query for accessing the at least one row during the live migration of the first table to the second table of the second database schema;
determining, in response to receiving the second query and based on the mutation table, at least one corresponding row of the second table has been deleted;
determining that the row has been deleted from the second table; and
providing a null value in response to the third query.
15. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
receiving a first query requesting a modification of at least one field in the first table of a first database schema during a live migration of the first table to a second table of a second database schema, the first table storing live data;
modifying, in response to receiving the first query, at least one corresponding field of the second table based on the first query;
updating a mutation table based on the modification, wherein the mutation table tracks one or more modifications to the second table;
receiving a second query for accessing the at least one field in the first table during the live migration of the first table to the second table of the second database schema; and
determining, in response to receiving the second query and based on the mutation table, the corresponding field of the second table has been modified;
providing, based on the determination the corresponding field of the second table has been modified, data from the at least one corresponding field of the second table in response to the second query.
16. The non-transitory computer readable medium of claim 15 , wherein the instructions further cause the system to perform:
receiving a third query for accessing at least one other field of the first table during the live migration of the first table to the second table of the second database schema;
determining, in response to receiving the third query and based on the mutation table, at least one corresponding other field of the second table has not been modified; and
providing, based on the determination the corresponding field of the second table has not been modified, data from the at least one field of the first table in response to the third query.
17. The non-transitory computer readable medium of claim 15 , wherein the first table becomes immutable during the live migration.
18. The non-transitory computer readable medium of claim 15 , wherein modifying the first table includes at least one of modifying data, inserting data, or deleting data in the first table.
19. The non-transitory computer readable medium of claim 15 , wherein updating the mutation table
further comprises: determining a row in the second table to which the modification was performed; and
updating the mutation table to indicate that the row has been modified.
20. The non-transitory computer readable medium of claim 19 , wherein the mutation table indicates the requested modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.