Hybrid DRAM-SSD memory system for a distributed database node
View Patent ↗In one exemplary aspect, a method includes building a data index from a set of data from a client application in a distributed database system. The data index is optimized for operation in a DRAM data-storage device, and wherein the data index is built to use a set of fix-sized blocks that fit a DRAM device block size. The data index is maintained in the DRAM data-storage device. An SSD data-storage device is used to persistently store the data referenced by the data index in the DRAM data-storage device, and wherein the data and the metadata are stored in the SSD data-storage device in a log-based format.
1. A method of a hybrid dynamic random access memory (DRAM)-solid-state drive (SSD) memory system in a distributed database system comprising:
building a data index from a set of data from a client application in a distributed database system;
optimizing the data index for operation in a DRAM data-storage device, and wherein the data index is built to use a set of fix-sized blocks that fit a DRAM device block size;
maintaining the data index in the DRAM data-storage device; and
utilizing an SSD data-storage device to persistently store the data referenced by the data index in the DRAM data-storage device, and wherein the data and the metadata are stored in the SSD data-storage device in a log-based format.
2. The method of claim 1 , wherein the log-based format creates a copy of the data for each write operation to the SSD data-storage device.
3. The method of claim 2 , wherein the SSD data-storage device can comprise an array of can have an array of flash-based memory devices, and wherein data is fetched in parallel from the flash-based memory devices and shipped back to the client application.
4. The method of claim 2 , wherein the data and the metadata are written sequentially to a circular buffer in the log-based format.
5. The method of claim 4 , wherein the distributed database system comprises a Not Only Structured Query Language (NoSQL) distributed database system.
6. The method of claim 1 , wherein a cryptographic hash function is used to build the data index.
7. The method of claim 1 , wherein an array of SSD data-storage devices are used to persistently store the data referenced by the data index in the DRAM data-storage device.
8. The method of claim 7 , wherein the SSD data-storage device comprises a NAND (Not AND)-based flash memory device.
9. The method of claim 1 , wherein the data index is implemented as a red-black tree data index.
10. The method of claim 1 , wherein the data index is implemented using at least one arena allocation process.
11. A computerized system of a hybrid dynamic random access memory (DRAM)-solid-state drive (SSD) memory system in a distributed database system comprising:
a processor configured to execute instructions;
a memory including instructions when executed on the processor, causes the processor to perform operations that:
build a data index from a set of data from a client application in a distributed database system;
optimize the data index for operation in a DRAM data-storage device, and wherein the data index is built to use a set of fix-sized blocks that fit a DRAM device block size;
maintain the data index in the DRAM data-storage device; and
utilize an SSD data-storage device to persistently store the data referenced by the data index in the DRAM data-storage device, and wherein the data and the metadata are stored in the SSD data-storage device in a log-based format.
12. The computerized system of claim 11 , wherein the log-based format creates a copy of the data for each write operation to the SSD data-storage device.
13. The computerized system of claim 12 , wherein a cryptographic hash function is used to build the data index.
14. The computerized system of claim 12 , wherein an array of SSD data-storage devices are used to persistently store the data referenced by the data index in the DRAM data-storage device.
15. The computerized system of claim 12 , wherein the SSD data-storage device comprises a NAND (Not AND)-based flash memory device.
16. The computerized system of claim 12 , wherein the data index is implemented as a red-black tree data index.
17. The computerized system of claim 12 , wherein the data index is implemented using at least one arena allocation process.
18. The computerized system of claim 12 , wherein the SSD data-storage device can comprise an array of can have an array of flash-based memory devices, and wherein data is fetched in parallel from the flash-based memory devices and shipped back to the client application.
19. The computerized system of claim 12 , wherein the data and the metadata are written sequentially to a circular buffer in the log-based format.
20. The computerized system of claim 12 , wherein the distributed database system comprises a Not Only Structured Query Language (NoSQL) distributed database system.