Efficient live-migration of remotely accessed data
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for migrating data from a first storage system to a second storage system. That data is for a process (e.g., a virtual machine, application, or some other process) operating on a computer system that is separate from the first storage system and the second storage system. That data is stored according to data subsets that are each exclusive of each other. As each data set is migrated, access to the data set by the process is precluded, but access for the remaining data sets is not affected. Once the data migration is complete, access to the data set by the process is restored.
1. A method performed by a data processing system, the method comprising:
storing, in a first physical machine, a mapping for a data processing process that processes data stored in the first physical machine, the data stored according to a plurality of data subsets;
for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;
enabling read and write access to the data for the data processing process according to the logical addressing; and
in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets.
2. The method of claim 1 , wherein:
the determination that the first data subset stored on the first physical machine is to be migrated to the second physical machine is performed by the first physical machine; and
blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets comprises blocking, by the first physical machine, read and write access for the data processing process to the first data subset.
3. The method of claim 2 , further comprising:
receiving, from the first physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and
wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to the data processing process receiving the notification provided by the first physical machine.
4. The method of claim 3 , wherein the notification instructs the data processing process to not access the first data subset.
5. The method of claim 3 , further comprising:
receiving, from the second physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and
wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to receiving the notification by the data processing process.
6. The method of claim 1 , further comprising determining, by the data processing process, that the first data subset stored on the first physical machine is to be migrated to the second physical machine; and
migrating, from the first physical machine to the second physical machine, the first data subset to store the data subset on the second physical machine comprises:
instructing, by the data processing process, the first physical machine to send the first data subset to the data processing process; and
sending, by the data processing process, the first data subset received from the first physical machine to the second physical machine;
the updating the mapping is done by the data processing process.
7. The method of claim 1 , further comprising restoring read and write access to the first data subset for the data processing process after the migration of the first data subset from the first physical machine to the second physical machine.
8. The method of claim 1 , wherein the data subsets are data chunks logically mapped to a physical address range.
9. The method of claim 1 , wherein the data subsets are data blocks.
10. The method of claim 1 , further comprising, in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.
11. A system, comprising:
a data processing device; and
a non-transitory computer-readable memory in data communication with the data processing device and storing instructions executable by the data processing device and that upon such execution cause the data processing device to perform operations comprising:
storing, in a first physical machine, a mapping for a data processing process that processes data stored in the first physical machine, the data stored according to a plurality of data subsets;
for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;
enabling read and write access to the data for the data processing process according to the logical addressing; and
in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets.
12. The system of claim 11 , wherein:
the determination that the first data subset stored on the first physical machine is to be migrated to the second physical machine is performed by the first physical machine; and
blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets comprises blocking, by the first physical machine, read and write access for the data processing process to the first data subset.
13. The system of claim 11 , further comprising, in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.
14. The system of claim 13 , the operation further comprising:
receiving, from the first physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and
wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to the data processing process receiving the notification provided by the first physical machine.
15. The system of claim 14 , wherein the notification instructs the data processing process to not access the first data subset.
16. The system of claim 14 , the operations further comprising:
receiving, from the second physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and
wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to receiving the notification by the data processing process.
17. The system of claim 11 , the operations further comprising determining, by the data processing process, that the first data subset stored on the first physical machine is to be migrated to the second physical machine; and
migrating, from the first physical machine to the second physical machine, the first data subset to store the data subset on the second physical machine comprises:
instructing, by the data processing process, the first physical machine to send the first data subset to the data processing process; and
sending, by the data processing process, the first data subset received from the first physical machine to the second physical machine;
the updating the mapping is done by the data processing process.
18. The system of claim 11 , wherein the data subsets are data chunks logically mapped to a physical address range.
19. The system of claim 11 , wherein the data subsets are data blocks.
20. A non-transitory computer-readable memory in data communication with the data processing device and storing instructions executable by the data processing device and that upon such execution cause the data processing device to perform operations comprising:
storing, in a first physical machine, a mapping for a data processing process that processes data stored in the first physical machine, the data stored according to a plurality of data subsets;
for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;
enabling read and write access to the data for the data processing process according to the logical addressing;
in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets; and
in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.