Storage reservation pools for virtual infrastructure
A method to allocate storage includes assigning a quota on space from a storage reservation pool, allocating an epoch specific storage space to a virtual disk, creating a memory map for the virtual disk to track used space. For every write to the virtual disk during an epoch, the method includes updating the memory map, determining if the used space is greater than a threshold of the epoch specific storage space based on the memory map. When the used space is greater than the threshold, the method includes predicting additional space for future writes to the virtual disk in the epoch, determining if the additional space is available from the storage reservation pool, and, when the additional space is available, increasing the epoch specific storage and proceeding with the write to the virtual disk.
1. A method for a computing system to allocate storage from a storage reservation pool to virtual disks of virtual machines, the method comprising:
assigning a quota on available space from the storage reservation pool;
adding the virtual disks to the storage reservation pool;
for each virtual disk belonging to the storage reservation pool:
allocating a first epoch specific storage space to the virtual disk based on the available space in the storage reservation pool, wherein the first epoch specific storage space corresponds to a first epoch of time, and the first epoch of time occurs prior to a second epoch of time;
creating a memory map for the virtual disk to track space used by the virtual disk;
for every write request to the virtual disk during the first epoch:
updating the memory map with information about the write request, including a write location and an amount of data written;
based on the memory map and without querying the storage reservation pool, determining if the used space consumed by the virtual disk is greater than a threshold of the first epoch specific storage space; and
when the used space is greater than the threshold of the first epoch specific storage space:
predicting additional space that will be consumed by other writes for the remainder of the first epoch to the virtual disk;
determining if the additional space is available from the storage reservation pool; and
when the additional space is available from the storage reservation pool:
increasing the first epoch specific storage space by the additional space; and
proceeding with the write request to the virtual disk.
2. The method of claim 1 , further comprising, at the end of the first epoch:
based on the memory map, determining unused space in the first epoch specific storage space and returning the unused space to the available space from the storage reservation pool; and
determining the available space in the storage reservation pool and allocating a second epoch specific storage space to the virtual disk for the second epoch.
3. The method of claim 1 , wherein the allocating is based on a usage pattern of the virtual disk and the available space in the storage reservation pool.
4. The method of claim 1 , wherein:
the storage reservation pool includes one or more datastores each comprising local hard disks on host computers; and
the allocating is based on input/output throughput of a virtual small computer system interface (vSCSI) controller queue in each host computer that buffers inputs/outputs between virtual machines and local hard disks on each host computer.
5. The method of claim 1 , wherein the predicting is based on a usage pattern of the virtual disk.
6. The method of claim 1 , further comprising suggesting a new quota on the available space from the storage reservation pool based on the available storage in the storage reservation pool.
7. The method of claim 1 , wherein the determining if the additional space is available from the storage reservation pool comprises querying the storage reservation pool.
8. The method of claim 1 , wherein the updating the memory map further includes ignoring overlap in write locations to save overhead so previously written locations are not recorded again.
9. The method of claim 1 , further comprising, when the used space is not greater than the threshold of the first epoch specific storage space, proceeding with the write request.
10. A non-transitory, computer-readable storage medium encoded with instructions executable by a processor to allocate storage from a storage reservation pool to virtual disks of virtual machines, the instructions comprising:
assigning a quota on available space from the storage reservation pool;
adding the virtual disks to the storage reservation pool;
for each virtual disk belonging to the storage reservation pool:
allocating a first epoch specific storage space to the virtual disk based on the available space in the storage reservation pool, wherein the first epoch specific storage space corresponds to a first epoch of time, and the first epoch of time occurs prior to a second epoch of time;
creating a memory map for the virtual disk to track space used by the virtual disk;
for every write request to the virtual disk during the first epoch:
updating the memory map with information about the write request, including a write location and an amount of data written;
based on the memory map and without querying the storage reservation pool, determining if the used space consumed by the virtual disk is greater than a threshold of the first epoch specific storage space; and
when the used space is greater than the threshold of the first epoch specific storage space:
predicting additional space that will be consumed by other writes for the remainder of the first epoch to the virtual disk;
determining if the additional space is available from the storage reservation pool; and
when the additional space is available from the storage reservation pool:
increasing the first epoch specific storage space by the additional space; and
proceeding with the write request to the virtual disk.
11. The non-transitory, computer-readable storage medium of claim 10 , wherein the instructions further comprise, at the end of the first epoch:
based on the memory map, determining unused space in the first epoch specific storage space and returning the unused space to the available space from the storage reservation pool; and
determining the available space in the storage reservation pool and allocating a second epoch specific storage space to the virtual disk for the second epoch.
12. The non-transitory, computer-readable storage medium of claim 10 , wherein the allocating is based on a usage pattern of the virtual disk and the available space in the storage reservation pool.
13. The non-transitory, computer-readable storage medium of claim 10 , wherein:
the storage reservation pool includes one or more datastores each comprising local hard disks on host computers; and
the allocating is based on input/output throughput of a virtual small computer system interface (vSCSI) controller queue in each host computer that buffers inputs/outputs between virtual machines and local hard disks on each host computer.
14. The non-transitory, computer-readable storage medium of claim 10 , wherein the predicting is based on a usage pattern of the virtual disk.
15. The non-transitory, computer-readable storage medium of claim 10 , wherein the instructions further comprise suggesting a new quota on the available space from the storage reservation pool based on the available storage in the storage reservation pool.
16. The non-transitory, computer-readable storage medium of claim 10 , wherein the determining if the additional space is available from the storage reservation pool comprises querying the storage reservation pool.
17. The non-transitory, computer-readable storage medium of claim 10 , wherein the updating the memory map further includes ignoring overlap in write locations to save overhead so previously written locations are not recorded again.
18. The non-transitory, computer-readable storage medium of claim 10 , wherein the instructions further comprise, when the used space is not greater than the threshold of the first epoch specific storage space, proceeding with the write request.
19. A computer system, comprising:
one or more host computers, each host computer running a hypervisor, the hypervisor running one or more virtual machines, each virtual machine having one or more virtual disks and one or more input/output (I/O) filters installed on the one or more virtual disks, respectively,
a processor;
a computer-readable medium encoded with instructions, when executable by the processor, causes the processor configured to:
create a storage reservation pool from one or more datastores;
for each virtual disk belonging to the storage reservation pool:
assign a quota on available space from the storage reservation pool;
add the virtual disk to the storage reservation pool;
create an I/O filter for the virtual disk;
allocate a first epoch specific storage space to the virtual disk based on the available space in the storage reservation pool, wherein the first epoch specific storage space corresponds to a first epoch of time, and the first epoch of time occurs prior to a second epoch of time;
the I/O filter is configured to create a memory map for the virtual disk to track space used by the virtual disk;
each time the I/O filter receives a write request to the virtual disk during the first epoch:
the I/O filter is configured to update the memory map, including recording a write location and an amount of the write data in the memory map;
based on the memory map and without querying the storage reservation pool, the I/O filter is configured to determine if the used space consumed by the virtual disk is greater than a threshold of the first epoch specific storage space; and
when the I/O filter determines the used space is greater than the threshold of the first epoch specific storage space:
the I/O filter is configured to predict additional space that will be consumed by other writes for the remainder of the first epoch to the virtual disk;
the I/O filter is configured to request the additional space;
the processor is configured to determine if the additional space is available from the storage reservation pool; and
when the processor determines the additional space is available from the storage reservation pool:
the processor is configured to increase the first epoch specific storage space by the additional space; and
the I/O filter is configured to proceed with the write request to the virtual disk.
20. The system of claim 19 , wherein, at the end of the first epoch:
the I/O filter is configured to, based on the memory map, determine unused space in the first epoch specific storage space and return the unused space to the available space from the storage reservation pool; and
the processor is configured to determine the available space in the storage reservation pool and allocating a second epoch specific storage space to the virtual disk for the second epoch.
21. The system of claim 19 , wherein the allocating is based on a usage pattern of the virtual disk and the available space in the storage reservation pool.
22. The system of claim 19 , wherein:
the storage reservation pool includes one or more datastores each comprising local hard disks on host computers; and
the allocating is based on input/output throughput of a virtual small computer system interface (vSCSI) controller queue in each host computer that buffers inputs/outputs between virtual machines and local hard disks on each host computer.
23. The system of claim 19 , wherein the predicting is based on a usage pattern of the virtual disk.
24. The system of claim 19 , wherein the instructions further comprise suggesting a new quota on the available space from the storage reservation pool based on the available storage in the storage reservation pool.
25. The system of claim 19 , wherein the determining if the additional space is available from the storage reservation pool comprises querying the storage reservation pool.
26. The system of claim 19 , wherein the updating the memory map further includes ignoring overlap in write locations to save overhead so previously written locations are not recorded again.
27. The system of claim 19 , wherein when the I/O filter determines the used space is not greater than the threshold of the epoch specific storage space, the I/O filter proceeds with the write request.