Memory efficient approach to extending cache
A method may use memory efficiently to extend cache. A processor receives a request to write data. The size of the data in the write request is compared to a threshold. When the size of the data exceeds the threshold, the data is stored on a solid state device. Page descriptors for the data on the solid state device are stored in a metadata log, and a reference to a first page descriptor of the page descriptors in the metadata log is stored in a first hash table in memory.
1. A memory efficient method of extending cache, comprising:
receiving, by a processor, a request to write data;
comparing a size of the data in the write request to a threshold;
when the size of the data exceeds the threshold,
storing the data on a solid state device;
storing, in a metadata log, page descriptors for the data on the solid state device; and
storing, in a first hash table in memory, a reference to a first page descriptor of the page descriptors in the metadata log; and
when the size of the data is below the threshold,
storing the data in a non-volatile random access memory;
storing, in the metadata log, at least one page descriptor for the data; and
storing, in a second hash table in memory, a reference to at least one page descriptor for the data in the metadata log.
2. The method of claim 1 , wherein the threshold is 1 MB.
3. The method of claim 1 , further comprising:
receiving, by the processor, a request to read the data;
checking the first and second hash tables for entries associated with the data.
4. The method of claim 3 , further comprising:
when both the first and second hash tables include entries associated with the data,
using the reference, from the first hash table, to the first page descriptor in the metadata log to retrieve the data from the solid state device;
using the reference, in the second hash table, to the at least one page descriptor in the metadata log to retrieve the data from the non-volatile random access memory.
5. The method of claim 4 , further comprising:
replacing at least a portion of the data from the solid state device with the data from the non-volatile random access memory.
6. The method of claim 4 , further comprising:
comparing a sequence number in the first page descriptor associated with the data from the solid state device and a sequence number in the at least one page descriptor associated with the data from the non-volatile random access memory.
7. The method of claim 6 , further comprising:
if the sequence number for the data from the non-volatile random access memory is larger than the sequence number for the data from the solid state device, replacing at least a portion of the data from the solid state device with the data from the non-volatile random access memory.
8. The method of claim 6 , further comprising
if the sequence number for the data from the non-volatile random access memory is smaller than the sequence number for the data from the solid state device, responding to the read request with the data from the solid state device.
9. The method of claim 3 , further comprising:
when the second hash table, but not the first hash table, includes an entry associated with the data, retrieving the data from the non-volatile random access memory.
10. A system comprising at least one processor configured to:
receive a request to write data;
compare a size of the data in the write request to a threshold;
when the size of the data exceeds the threshold,
store the data on a solid state device;
store, in a metadata log, page descriptors for the data on the solid state device; and
store, in a first hash table in memory, a reference to a first page descriptor of the page descriptors in the metadata log; and
when the size of the data is below the threshold,
store the data in a non-volatile random access memory;
store, in the metadata log, at least one page descriptor for the data; and
store, in a second hash table in memory, a reference to at least one page descriptor for the data in the metadata log.
11. The system of claim 10 , wherein the threshold is 1 MB.
12. The system of claim 10 , wherein the at least one processor is further configured to:
receive a request to read the data;
check the first and second hash tables for entries associated with the data.
13. The system of claim 12 , wherein the at least one processor is further configured to:
when both the first and second hash tables include entries associated with the data,
use the reference, from the first hash table, to the first page descriptor in the metadata log to retrieve the data from the solid state device;
use the reference, in the second hash table, to the at least one page descriptor in the metadata log to retrieve the data from the non-volatile random access memory.
14. The system of claim 13 , wherein the at least one processor is further configured to:
replace at least a portion of the data from the solid state device with the data from the non-volatile random access memory.
15. The system of claim 13 , wherein the at least one processor is further configured to:
compare a sequence number in the first page descriptor associated with the data from the solid state device and a sequence number in the at least one page descriptor associated with the data from the non-volatile random access memory.
16. The system of claim 15 , wherein the at least one processor is further configured to:
if the sequence number for the data from the non-volatile random access memory is larger than the sequence number for the data from the solid state device, replace at least a portion of the data from the solid state device with the data from the non-volatile random access memory.
17. The system of claim 15 , wherein the at least one processor is further configured to:
if the sequence number for the data from the non-volatile random access memory is smaller than the sequence number for the data from the solid state device, respond to the read request with the data from the solid state device.
18. The system of claim 12 , wherein the at least one processor is further configured to:
when the second hash table, but not the first hash table, includes an entry associated with the data, retrieve the data from the non-volatile random access memory.
19. A non-transitory computer-readable storage medium having stored thereon program code of one or more software programs, wherein the program code when executed by at least one computing device causes the at least one computing device to perform the following steps:
receiving, by a processor, a request to write data;
comparing a size of the data in the write request to a threshold;
when the size of the data exceeds the threshold,
storing the data on a solid state device;
storing, in a metadata log, page descriptors for the data on the solid state device; and
storing, in a first hash table in memory, a reference to a first page descriptor of the page descriptors in the metadata log; and
when the size of the data is below the threshold,
storing the data in a non-volatile random access memory;
storing, in the metadata log, at least one page descriptor for the data; and
storing, in a second hash table in memory, a reference to at least one page descriptor for the data in the metadata log.
20. The computer-readable storage medium of claim 19 , wherein the threshold is 1 MB.