Restoring virtual machine data to cloud using a virtual server cloud file system
Uploads of restored virtual machine (“VM”) data to cloud storage, e.g., VM restore-to-cloud operations, are performed without having to write whole restored virtual disk files to a proxy server before the virtual disk data begins uploading to cloud. Restored data blocks from a backup source are locally cached, staged for efficiency, and asynchronously uploaded to the cloud page-by-page without tapping mass storage resources on the proxy. Downloads of VM data from cloud storage, e.g., VM backup-from-cloud, are performed without having to download a virtual disk file in its entirety to the proxy server before the backup operation begins generating a backup copy. This speeds up “pulling” VM data from the cloud by pre-fetching and locally caching downloaded data blocks. The cached data blocks are processed for backup and stored page-by-page directly into a secondary copy of the cloud VM virtual-disk file without tapping mass storage resource at the proxy.
1. A system comprising:
a first computing device comprising one or more hardware processors and computer memory for executing computer programming instructions, and further comprising a data storage area that comprises a plurality of pages, wherein each page is sized to match a page-size configured in a cloud-based account;
wherein the first computing device is configured to restore a secondary copy of a second virtual-disk file to a first virtual-disk file that is configured in the cloud-based account, comprising:
execute a driver that presents a file system comprising the data storage area as a local mass storage device,
store first data blocks to a first page in the data storage area, wherein the first data blocks are restored to a primary data format from a backup format of the secondary copy, wherein the backup format is distinct from the primary data format,
wherein the first data blocks restored from the secondary copy fill the first page, based on the page-size,
wherein the second virtual-disk file is associated with a second virtual machine hosted by a second computing device that is distinct from the first computing device, and
wherein a size of the secondary copy restored in its entirety is larger than the data storage area on the first computing device,
upload the first data blocks in the first page from the data storage area to a page in the first virtual-disk file configured in the cloud-based account, and
further upload page-by-page data blocks from the data storage area, which are restored thereinto from the secondary copy, to the first virtual-disk file configured in the cloud-based account, without first restoring the secondary copy of the second virtual-disk file in its entirety to the first computing device, and
based on the first computing device determining that (i) no more of the secondary copy remains to be restored to the data storage area and (ii) a second page in the plurality of pages in the data storage area is not filled by data blocks restored thereinto from the secondary copy, upload the data blocks in the second page from the data storage area to the first virtual-disk file to finish restoring the secondary copy of the second virtual-disk file to the first virtual-disk file configured in the cloud-based account.
2. The system of claim 1 , wherein data in the first virtual-disk file, restored in its entirety to the cloud-based account, is available as primary data to a first virtual machine that executes in the cloud-based account.
3. The system of claim 1 , wherein the first computing device is further configured to: based on determining that (a) more of the secondary copy of the secondary copy remains to be restored, and (b) a least-recently used page among the plurality of pages in the data storage area is not filled by data blocks restored from the secondary copy, and (c) the data storage area is running below a threshold on storage space, upload the data blocks in the least-recently used page from the data storage area to the first virtual-disk file configured in the cloud-based account.
4. The system of claim 3 , wherein the first computing device is further configured to re-use the least-recently used page for storing other data blocks restored from the secondary copy.
5. The system of claim 1 , wherein the first computing device is further configured to: track, based on the page-size, whether each page in the plurality of pages in the data storage area is filled by data blocks restored from the secondary copy.
6. The system of claim 1 , wherein the first computing device is further configured to track a time of when a given page among the plurality of pages in the data storage area receives one or more data blocks restored from the secondary copy.
7. The system of claim 6 , wherein the first computing device is further configured to determine which page among the plurality of pages in the data storage area is least-recently used.
8. The system of claim 1 , wherein the system enables a first virtual machine in the cloud-based account to use data that is restored from the secondary copy of the second virtual-disk file and uploaded into the first virtual-disk file configured in the cloud-based account as primary data for the first virtual machine.
9. The system of claim 1 , wherein the first computing device is further configured to create a mount point on the first computing device for the data storage area presented by the driver as the local mass storage device, and
wherein the data storage area is defined as a destination for restoring the secondary copy of the second virtual-disk file to the cloud-based account.
10. The system of claim 1 , wherein the data storage area is configured as a cache on the first computing device.
11. A computer-implemented method performed by one or more hardware processors, the method comprising:
a first computing device comprising one or more hardware processors and computer memory for executing computer programming instructions, and further comprising a data storage area that comprises a plurality of pages, wherein each page is sized to match a page-size configured in a cloud-based account;
wherein the first computing device is configured to restore a secondary copy of a second virtual-disk file to a first virtual-disk file that is configured in the cloud-based account, comprising:
execute a driver that presents a file system comprising the data storage area as a local mass storage device,
store first data blocks to a first page in the data storage area, wherein the first data blocks are restored to a primary data format from a backup format of the secondary copy, wherein the backup format is distinct from the primary data format,
wherein the first data blocks restored from the secondary copy fill the first page, based on the page-size,
wherein the second virtual-disk file is associated with a second virtual machine hosted by a second computing device that is distinct from the first computing device, and
wherein a size of the secondary copy restored in its entirety is larger than the data storage area on the first computing device,
upload the first data blocks in the first page from the data storage area to a page in the first virtual-disk file configured in the cloud-based account, and
further upload page-by-page data blocks from the data storage area, which are restored thereinto from the secondary copy, to the first virtual-disk file configured in the cloud-based account, without first restoring the secondary copy of the second virtual-disk file in its entirety to the first computing device, and
based on the first computing device determining that (i) no more of the secondary copy remains to be restored to the data storage area and (ii) a second page in the plurality of pages in the data storage area is not filled by data blocks restored thereinto from the secondary copy, upload the data blocks in the second page from the data storage area to the first virtual-disk file to finish restoring the secondary copy of the second virtual-disk file to the first virtual-disk file configured in the cloud-based account.
12. The computer-implemented method of claim 11 , wherein data in the first virtual-disk file, restored in its entirety to the cloud-based account, is available as primary data to a first virtual machine that executes in the cloud-based account.
13. The computer-implemented method of claim 11 further comprising: based on determining that (a) more of the secondary copy of the secondary copy remains to be restored, and (b) a least-recently used page among the plurality of pages in the data storage area is not filled by data blocks restored from the secondary copy, and (c) the data storage area is running below a threshold on storage space, uploading the data blocks in the least-recently used page from the data storage area to the first virtual-disk file configured in the cloud-based account.
14. The computer-implemented method of claim 13 , further comprising: re-using the least-recently used page for storing other data blocks restored from the secondary copy.
15. The computer-implemented method of claim 11 , further comprising: tracking, based on the page-size, whether each page in the plurality of pages in the data storage area is filled by data blocks restored from the secondary copy.
16. The computer-implemented method of claim 11 , further comprising: tracking a time of when a given page among the plurality of pages in the data storage area receives one or more data blocks restored from the secondary copy.
17. The computer-implemented method of claim 16 , further comprising: determining which page among the plurality of pages in the data storage area is least-recently used.
18. The computer-implemented method of claim 11 , wherein a first virtual machine in the cloud-based account uses data that is restored from the secondary copy of the second virtual-disk file and uploaded into the first virtual-disk file configured in the cloud-based account as primary data for the first virtual machine.
19. The computer-implemented method of claim 11 , further comprising: creating a mount point on the first computing device for the data storage area presented by the driver as the local mass storage device, and
wherein the data storage area is defined as a destination for restoring the secondary copy of the second virtual-disk file to the cloud-based account.
20. The computer-implemented method of claim 11 , wherein the data storage area is configured as a cache on the first computing device.