Virtual server cloud file system for backing up cloud-based virtual machine data
Uploads of restored virtual machine (“VM”) data to cloud storage, e.g., VM restore-to-cloud operations, are performed without having to write whole restored virtual disk files to a proxy server before the virtual disk data begins uploading to cloud. Restored data blocks from a backup source are locally cached, staged for efficiency, and asynchronously uploaded to the cloud page-by-page without tapping mass storage resources on the proxy. Downloads of VM data from cloud storage, e.g., VM backup-from-cloud, are performed without having to download a virtual disk file in its entirety to the proxy server before the backup operation begins generating a backup copy. This speeds up “pulling” VM data from the cloud by pre-fetching and locally caching downloaded data blocks. The cached data blocks are processed for backup and stored page-by-page directly into a secondary copy of the cloud VM virtual-disk file without tapping mass storage resource at the proxy.
1. A system comprising:
a computing device comprising one or more hardware processors and computer memory, wherein the computing device is configured to:
execute a driver that presents a cache storage area on the computing device as a source of data to be backed up;
download, into a first page of the cache storage area of the computing device, first data blocks from a first page of a virtual-disk file associated with a virtual machine hosted by a cloud-based account, wherein the virtual-disk file is stored in its entirety in the cloud-based account,
wherein the cache storage area is configured with a plurality of pages, including the first page,
wherein each page in the plurality of pages is sized to match a page-size configured in the cloud-based account, and
wherein the virtual-disk file in its entirety is bigger than the cache storage area on the computing device;
back up the first data blocks from the first page of the cache storage area into second backed up data blocks; and
add the second backed up data blocks to a secondary copy of the virtual-disk file, wherein the secondary copy is stored in a storage device apart from the computing device;
based on determining that no more of the virtual-disk file remains to be downloaded to the cache storage area and further based on determining that third data blocks in at least one page in the cache storage area have not been backed up, back up the third data blocks from the at least one page in the cache storage area into fourth backed up data blocks, and cause the fourth backed up data blocks to be added to the secondary copy of the virtual-disk file; and
wherein the virtual-disk file is backed up in its entirety from the cloud-based account to the secondary copy, based on downloading data blocks page-by-page from the virtual-disk file into the cache storage area, and without downloading the virtual-disk file in its entirety to the computing device.
2. The system of claim 1 , wherein the computing device is further configured to address read requests to a file system comprising the cache storage area and wherein each read request is served by the driver from the cache storage area.
3. The system of claim 1 , wherein the computing device is further configured to download pages of data blocks, including the first page, sequentially from the virtual-disk file into the cache storage area, based on anticipating a next page of data blocks before receiving a read request for the next page.
4. The system of claim 1 wherein the computing device is further configured to: when one or more data blocks are downloaded from the virtual-disk file to a given page in the cache storage area, update an indication that the given page is most-recently used in the cache storage area.
5. The system of claim 1 , wherein data in the virtual-disk file is in a primary data format that is accessible by the virtual machine in the cloud-based account, and wherein the secondary copy is in a backup format that is distinct from the primary data format.
6. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a computing device, cause the computing device to:
execute a driver associated with a cache storage area configured on the computing device;
download, into a first page of the cache storage area, first data blocks from a snapshot of a virtual-disk file that is associated with a virtual machine hosted by a cloud-based account, wherein the snapshot is stored in its entirety in the cloud-based account,
wherein the cache storage area is configured with a plurality of pages, including the first page,
wherein each page in the plurality of pages is sized to match a page-size configured in the cloud-based account,
wherein the first data blocks form a first page of the cloud-based account, and
wherein the virtual-disk file in its entirety is bigger than the cache storage area on the computing device;
back up the first data blocks from the first page of the cache storage area into second backed up data blocks; and
add the second backed up data blocks to a secondary copy of the virtual-disk file, wherein the secondary copy is stored in a storage device apart from the computing device;
based on determining that no more of the snapshot remains to be downloaded to the cache storage area and further based on determining that third data blocks in at least one page in the cache storage area have not been backed up, back up the third data blocks from the at least one page in the cache storage area into fourth backed up data blocks, and add the fourth backed up data blocks to the secondary copy of the virtual-disk file; and
wherein the snapshot of the virtual-disk file is backed up in its entirety from the cloud-based account to the secondary copy, based on downloading data blocks page-by-page, including the first page of the cloud-based account, from the snapshot into the cache storage area and without downloading either of: the virtual-disk file and the snapshot, in its entirety, to the computing device.
7. The non-transitory computer-readable medium of claim 6 , wherein the computer-executable instructions further cause the computing device to: sequentially download pages of data blocks from the snapshot, including the first page of the cloud-based account, into the cache storage area, based on anticipating a next page of data blocks before receiving a read request for the next page.
8. The non-transitory computer-readable medium of claim 6 , wherein the computer-executable instructions further cause the computing device to: address read requests to a file system comprising the cache storage area and wherein each read request is served by the driver from the cache storage area.
9. The non-transitory computer-readable medium of claim 6 , wherein the virtual-disk file is in a primary data format that is accessible by the virtual machine in the cloud-based account, and wherein the secondary copy is in a backup format that is distinct from the primary data format.