Memory allocation balancing for storage systems
Described herein are embodiments of a process that can be used to balance the allocation of primary memory between different types of information. In some embodiments, the memory allocation is balanced dynamically based on observed I/O patterns. Related system embodiments are also described.
1. A method comprising:
calculating a relationship of a first number of read operations and a second number of write operations of a plurality of I/O operations made to a storage system having a volatile memory;
selecting a memory allocation scheme based upon the relationship of the first number of read operations and the second number of write operations; and
applying the memory allocation scheme, wherein applying the memory allocation scheme comprises allocating capacity of the volatile memory based upon the selected allocation scheme,
wherein calculating the relationship of the first number of read operations and the second number of write operations comprises calculating a moving average of the relationship of the first number of read operations and the second number of write operations.
2. The method of claim 1 wherein selecting a memory allocation scheme is further based upon moving averages of read I/O latency and write I/O latency.
3. The method of claim 1 further comprising, according to the selected memory allocation scheme:
allocating a first amount of volatile memory as a metadata cache; and
allocating a second amount of volatile memory as a journal.
4. The method of claim 3 further comprising:
mapping logical block addresses (LBA) of the storage system to chunk hashes; and
mapping the chunk hashes to physical storage locations on storage devices of the storage system.
5. The method of claim 1 further comprising de-allocating capacity of the volatile memory based upon the selected allocation scheme.
6. The method of claim 1 wherein applying the memory allocation scheme further comprises determining if the memory allocation scheme should be applied.
7. The method of claim 6 further comprising:
using hysteresis to determine if the memory allocation scheme should be applied.
8. The method of claim 1 wherein calculating the relationship of the first number of read operations and the second number of write operations comprises calculating a ratio of the first number of read operations and the second number of write operations, the method further comprising:
selecting a first memory allocation scheme if the ratio exceeds a first threshold;
selecting a second memory allocation scheme if the ratio exceeds a second threshold; and
selecting a third memory allocation scheme if the ratio exceeds neither the first threshold nor the second threshold.
9. A system comprising:
a processor;
a volatile memory; and
a non-volatile memory storing computer program code that when executed on the processor causes the processor to execute a process operable to perform the operations of:
calculating a relationship between a first number of read operations and a second number of write operations of a plurality of I/O operations to a storage system having a volatile memory;
selecting a memory allocation scheme based on the relationship of the first number of read operations and the second number of write operations; and
applying the selected memory allocation scheme, wherein applying the selected memory allocation scheme comprises allocating capacity of the volatile memory based upon the selected memory allocation scheme,
wherein calculating the relationship of the first number of read operations and the second number of write operations comprises calculating a moving average of the relationship of the first number of read operations and the second number of write operations.
10. The system of claim 9 wherein the computer program code causes the processor to execute a process operable to select the memory allocation scheme further based upon a moving averages of read latency and write latency.
11. The system of claim 9 wherein the computer program code causes the processor to execute a process operable to:
allocate a first amount of volatile memory as a metadata cache according to the selected memory allocation scheme; and
allocate a second amount of volatile memory as a journal according to the selected memory allocation scheme.
12. The system of claim 11 wherein the metadata includes a logical block address (LBA) to chunk hash mapping and a chunk hash to storage location mapping.
13. The system of claim 9 wherein the computer program code causes the processor to execute a process further operable to de-allocate capacity of the volatile memory based upon the selected allocation scheme.
14. The system of claim 9 wherein the computer program code causes the processor to execute a process further operable to determine if the memory allocation scheme should be applied.
15. The system of claim 14 wherein the computer program code causes the processor to execute a process further operable to:
use hysteresis to determine if the memory allocation scheme should be applied.
16. The system of claim 9 wherein the computer program code causes the processor to execute a process operable to calculate the relationship of the first number of read operations and the second number of write operations by calculating a ratio of the first number of read operations and the second number of write operations, the method further comprising and further operable to:
select a first memory allocation scheme if the ratio exceeds a first threshold;
select a second memory allocation scheme if the ratio exceeds a second threshold; and
select a third memory allocation scheme if the ratio exceeds neither the first threshold nor the second threshold.
17. A computer program product tangibly embodied in a non-transitory computer-readable medium, the computer-readable medium storing program instructions that are executable to:
calculate a relationship of a first number of read operations and a second number of write operations of a plurality of I/O operations made to a storage system having a volatile memory;
select a memory allocation scheme based upon the relationship of the first number of read operations and the second number of write operations; and
applying the selected memory allocation scheme, wherein applying the memory allocation scheme comprises allocating capacity of the volatile memory based upon selected allocation scheme,
wherein calculating the relationship of the first number of read operations and the second number of write operations comprises calculating a moving average of the relationship of the first number of read operations and the second number of write operations.