System and method for creating snapshots in openflame environment
A method for creating snapshots and backups in a virtual computing environment is provided. The method includes writing application output of an application spanning one or more virtual machines as an application consistency group to a writeback cache, wherein the one or more virtual machines are implemented using one or more compute nodes and wherein the writeback cache is implemented in direct attached storage in the one or more compute nodes. The method includes pausing I/O (input/output) operations of the application and marking the pausing, in the writeback cache. The method includes resuming the I/O operations of the application, after the marking and dumping data, according to the marking, from the writeback cache to a data node, as a snapshot.
1. A method for creating snapshots and backups in a virtual computing environment, comprising:
writing application output of an application spanning one or more virtual machines as an application consistency group to a writeback cache, wherein the one or more virtual machines are implemented using one or more compute nodes and wherein the writeback cache is implemented in direct attached storage in the one or more compute nodes;
pausing I/O (input/output) operations of the application;
marking the pausing, in the writeback cache;
resuming the I/O operations of the application, after the marking; and
dumping data, according to the marking, from the writeback cache to a data node, as a snapshot.
2. The method of claim 1 , wherein the marking the pausing includes writing an epoch marker into the writeback cache.
3. The method of claim 1 , wherein the snapshot is one of a plurality of snapshots, each having a unique snapshot identifier and each being incremental to a previous snapshot.
4. The method of claim 1 , further comprising:
writing a backup from the data node to a backup storage, while the I/O operations of the application are enabled.
5. The method of claim 1 , further comprising:
representing at least one snapshot image and at least one backup image in a user interface;
receiving a user selection of a snapshot image or a backup image; and
restoring a virtual machine in accordance with the user selection.
6. The method of claim 1 , further comprising:
restoring data from a backup image in a backup storage onto the data node, responsive to a restore operation;
creating a snapshot object on a compute node;
linking the snapshot object to the restored data on the data node; and
attaching the snapshot object to one of: an original virtual machine or a new virtual machine.
7. The method of claim 1 , further comprising:
determining to which granularity of a plurality of supported granularities to perform a backup, wherein the plurality of supported granularities includes a granularity of a virtual machine, a granularity of an application consistency group, and a granularity of a virtual disk; and
writing a backup from the data node to a backup storage, at the determined granularity.
8. A tangible, non-transitory, computer-readable media having instructions thereupon which, when executed by a processor, cause the processor to perform a method comprising:
executing an application spanning an application consistency group of one or more virtual machines, with I/O (input/output) operations to direct attached storage via a writeback cache;
pausing the I/O operations of the application consistency group;
writing an epoch marker to the writeback cache, to mark the pausing;
resuming the I/O operations; and
dumping data of the writeback cache, in accordance with the epoch marker, to a data node as a snapshot.
9. The computer-readable media of claim 8 , wherein the data of the writeback cache is dumped to the data node as an incremental version since a previous snapshot.
10. The computer-readable media of claim 8 , wherein the method further comprises:
writing data from the data node to a backup storage, to a granularity of one of: a virtual machine, a virtual disk, or the application consistency group.
11. The computer-readable media of claim 8 , wherein the method further comprises:
displaying, via a user interface, a plurality of snapshots and one or more backups in a timeline, for user selection to restore to a virtual machine.
12. The computer-readable media of claim 8 , wherein the method further comprises:
restoring data from a backup image on backup storage to the data node;
linking a snapshot object on a compute node to the restored data on the data node; and
attaching the snapshot object to a virtual machine.
13. The computer-readable media of claim 8 , wherein:
dumping the data of the writeback cache to the data node, as the snapshot, occurs with the I/O operations resumed; and
the snapshot is performed to a granularity of the application consistency group.
14. A virtual environment computing system, comprising:
a plurality of compute nodes coupled as a compute plane and configured to host a plurality of virtual machines executing a plurality of applications;
the plurality of compute nodes having direct attached storage (DAS);
a plurality of data nodes coupled as a data plane and configured as virtual storage; and
a controller coupled to the compute plane and the data plane, configured to perform actions comprising:
coordinating the plurality of virtual machines to pause I/O (input/output) operations involving the direct attached storage;
directing one or more snapshots from the direct attached storage to specified ones of the plurality of data nodes with the pause of the I/O operations marked in the direct attached storage; and
coordinating the plurality of virtual machines to resume I/O operations upon completion of marking the pause of the I/O operations.
15. The virtual environment computing system of claim 14 , further comprising:
a storage virtualizer, implemented as a virtual machine in the compute plane; and
a writeback cache, coupled to the storage virtualizer, wherein the I/O operations involving the direct attached storage include writes to the writeback cache.
16. The virtual environment computing system of claim 14 , further comprising:
a storage virtualizer having a writeback cache implemented in the direct attached storage, the storage virtualizer configured to write an epoch marker to the writeback cache to mark the pause of the I/O operations, responsive to the controller initiating a snapshot.
17. The virtual environment computing system of claim 14 , further comprising:
a writeback cache, configured to receive data from the I/O operations involving the direct attached storage and configured to receive an epoch marker to mark the pause of the I/O operations, wherein the one or more snapshots from the direct attached storage to the specified ones of the plurality of data nodes includes the data from the I/O operations as written from the writeback cache to the specified ones of the plurality of data nodes in accordance with the epoch marker.
18. The virtual environment computing system of claim 14 , further comprising:
the controller configured to direct a backup from the plurality of data nodes to backup storage to a granularity of one of: a virtual machine, an application consistency group, or a virtual disk.
19. The virtual environment computing system of claim 14 , further comprising:
the controller configured to show a user interface with a plurality of snapshots and backups for user selection to restore to one or more of the plurality of virtual machines.
20. The virtual environment computing system of claim 14 , further comprising:
the controller configured to direct a restore operation that includes restoring data from a backup image in backup storage to a data node, creation of a snapshot object on a compute node, linking the snapshot object to the restored data on the data node, and attaching the snapshot object to a virtual machine.