Optimizing access to production data
Various systems, methods, and processes for optimizing access to production data in application development and testing environments are disclosed. If an input/output (I/O) operation is a read operation, a storage location on a virtual storage unit at which the read operation is to be performed is determined. Also determined is whether an earlier write operation was performed at the storage location. If an earlier write operation was performed at the storage location, the read operation is performed on one or more virtual data files. However, if the earlier write operation was not performed at the storage location, the read operation is performed on allocated storage space.
1. A method comprising:
in response to determining that a reclamation operation is to be performed on an existing virtual data file maintained by a virtual machine,
fetching an allocation of the existing virtual data file, wherein
the existing virtual data file is stored in an allocated storage space that is allocated within a virtual disk of a virtual storage unit in a storage system,
the storage system comprises one or more storage devices, in which are stored
production data comprising one or more production data files, and
the virtual storage unit,
the one or more production data files and the virtual storage unit are stored separately from one another in the storage system, and
the allocation represents one or more units of storage space of the existing virtual data file in the allocated storage space,
performing the reclamation operation on the one or more units of storage space, using the allocation,
deleting the existing virtual data file, and
performing an allocation operation, wherein
the allocating operation comprises
determining a storage location of a new storage space in the virtual storage unit, such that the new storage space comprises at least one unit of storage space of the one or more units of storage space, and
allocating the new storage space in the virtual storage unit to store at least one new virtual data file.
2. The method of claim 1 , further comprising:
subsequent to the allocation operation,
determining a storage location in the new storage space at which a read operation is to be performed,
determining whether an earlier write operation was performed at the storage location in the new storage space,
in response to determining that the earlier write operation was performed at the storage location in the new storage space, performing the read operation on the new virtual data file, and
in response to determining that the earlier write operation was not performed at the storage location in the new storage space, performing the read operation on the allocated storage space.
3. The method of claim 1 , further comprising:
provisioning at least one new virtual data file in the new storage space.
4. The method of claim 3 , wherein the provisioning comprises:
allocating storage space for the at least one new virtual data file in the new storage space;
fetching an allocation in the new storage space for the at least one new virtual data file; and
mapping the at least one virtual new data file to the new storage space.
5. The method of claim 1 , further comprising:
receiving a request to create the virtual storage unit; and
based on the request, creating the virtual storage unit in the storage system, wherein
the virtual storage unit is used to allocate the allocated storage space, and
the virtual disk in the virtual storage unit is created as a sparse file.
6. The method of claim 1 , further comprising:
allocating the allocated storage space.
7. The method of claim 6 , further comprising:
receiving an allocation map, wherein
the allocation map comprises information regarding the allocated storage space,
the one or more production data files are configured for present use by a production application, and
the allocated storage space corresponds, at least in part, to at least one of one or more production data files.
8. The method of claim 7 , further comprising:
converting information in the allocation map to one or more offsets associated with the one or more production data files, wherein
the one or more offsets are based on a storage structure of the one or more production data files.
9. The method of claim 8 , further comprising:
overlaying the storage structure over the allocation map, wherein
the overlaying facilitates serving data from a location in the one or more production data files, by permitting the storage system to re-map an input/output operation from the allocated storage space to the one or more production data files.
10. The method of claim 1 , wherein
the allocated storage space corresponds, at least in part, to at least one of the one or more production data files,
information regarding the allocated storage space is comprised in an allocation map and is configured to be converted into one or more offsets that are based on a storage structure of the one or more production data files, and
the serving of data from the one or more production data files is facilitated by permitting the storage system to re-map an input/output operation from the allocated storage space to the one or more production data files by virtue of the storage structure being configured to be overlaid over the allocation map.
11. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:
in response to determining that a reclamation operation is to be performed on an existing virtual data file maintained by a virtual machine,
fetching an allocation of the existing virtual data file, wherein
the existing virtual data file is stored in an allocated storage space that is allocated within a virtual disk of a virtual storage unit in a storage system,
the storage system comprises one or more storage devices, in which are stored
production data comprising one or more production data files, and
the virtual storage unit,
the one or more production data files and the virtual storage unit are stored separately from one another in the storage system, and
the allocation represents one or more units of storage space of the existing virtual data file in the allocated storage space,
performing the reclamation operation on the one or more units of storage space, using the allocation,
deleting the existing virtual data file from the virtual disk, and
performing an allocation operation, wherein
the allocating operation comprises
determining a storage location of a new storage space in the virtual storage unit, such that the new storage space comprises at least one unit of storage space of the one or more units of storage space, and
allocating the new storage space in the virtual storage unit to store at least one new virtual data file.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the method further comprises:
subsequent to the allocation operation,
determining a storage location in the new storage space at which a read operation is to be performed,
determining whether an earlier write operation was performed at the storage location in the new storage space,
in response to determining that the earlier write operation was performed at the storage location in the new storage space, performing the read operation on the new virtual data file, and
in response to determining that the earlier write operation was not performed at the storage location in the new storage space, performing the read operation on the allocated storage space.
13. The non-transitory computer-readable storage medium of claim 11 , wherein the method further comprises:
provisioning at least one new virtual data file in the new storage space.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the provisioning comprises:
allocating storage space for the at least one new virtual data file in the new storage space;
fetching an allocation in the new storage space for the at least one new virtual data file; and
mapping the at least one virtual new data file to the new storage space.
15. The non-transitory computer-readable storage medium of claim 11 , wherein the method further comprises:
receiving a request to create the virtual storage unit; and
based on the request, creating the virtual storage unit in the storage system, wherein
the virtual storage unit is used to allocate the allocated storage space, and
the virtual disk in the virtual storage unit is created as a sparse file.
16. A computing system comprising:
one or more processors; and
a computer-readable storage medium coupled to the one or more processors, comprising program instructions, which, when executed by the one or more processors, perform a method comprising
in response to determining that a reclamation operation is to be performed on an existing virtual data file maintained by a virtual machine,
fetching an allocation of the existing virtual data file, wherein
the existing virtual data file is stored in an allocated storage space that is allocated within a virtual disk of a virtual storage unit in a storage system,
the storage system comprises one or more storage devices, in which are stored
production data comprising one or more production data files, and
the virtual storage unit,
the one or more production data files and the virtual storage unit are stored separately from one another in the storage system, and
the allocation represents one or more units of storage space of the existing virtual data file in the allocated storage space,
performing the reclamation operation on the one or more units of storage space, using the allocation,
deleting the existing virtual data file from the virtual disk, and
performing an allocation operation, wherein
the allocating operation comprises
determining a storage location of a new storage space in the virtual storage unit, such that the new storage space comprises at least one unit of storage space of the one or more units of storage space, and
allocating the new storage space in the virtual storage unit to store at least one new virtual data file.
17. The computing system of claim 16 , wherein the method further comprises:
subsequent to the allocation operation,
determining a storage location in the new storage space at which a read operation is to be performed,
determining whether an earlier write operation was performed at the storage location in the new storage space,
in response to determining that the earlier write operation was performed at the storage location in the new storage space, performing the read operation on the new virtual data file, and
in response to determining that the earlier write operation was not performed at the storage location in the new storage space, performing the read operation on the allocated storage space.
18. The computing system of claim 16 , wherein the method further comprises:
provisioning at least one new virtual data file in the new storage space.
19. The computing system of claim 18 , wherein the provisioning comprises:
allocating storage space for the at least one new virtual data file in the new storage space;
fetching an allocation in the new storage space for the at least one new virtual data file; and
mapping the at least one virtual new data file to the new storage space.
20. The computing system of claim 16 , wherein the method further comprises:
receiving a request to create the virtual storage unit; and
based on the request, creating the virtual storage unit in the storage system, wherein
the virtual storage unit is used to allocate the allocated storage space, and
the virtual disk in the virtual storage unit is created as a sparse file.