Self-managed DRAM modules with built-in data compression and tiered caching

1 . A self-managed dynamic random-access memory (DRAM) module, comprising:

a plurality of DRAM; and

a controller chip having;

a plurality of DRAM controllers for managing the plurality of DRAM independently of a central processing unit (CPU) chip;

a CXL/PCIe input/output (I/O) engine for communicatively connecting the controller chip to the CPU chip via a CXL/PCIe channel; and

a cache implemented independently from the CPU chip with a first tier configured for storing uncompressed data and a second tier configured for storing compressed data,

wherein the controller chip is configured to handle data access requests received via the CXL/PCIe channel according to a process that includes:

searching the first tier for an uncompressed cache data block to service the data access request;

in response to a first tier hit, serving the uncompressed cache data block to service the data access request;

in response to a first tier miss, searching the second tier for a compressed cache data block to service the data access request;

in response to a second tier hit, decompressing the compressed cache data block to generate a decompressed cache data block, and inserting the decompressed cache data block into the first tier to service the data access request; and

in response to a second tier miss, reading a compressed data block from DRAM, decompressing the compressed data block to generate a decompressed data block, and inserting the decompressed data block into the first tier to service the data access request.

2 . The self-managed DRAM module of claim 1 , wherein the first tier is implemented using a set-associative cache architecture.

3 . The self-managed DRAM module of claim 2 , wherein the second tier is implemented with a key-value data store architecture, wherein a key is a logical DRAM address and the compressed cache data block is identified with a value associated with the key.

4 . The self-managed DRAM module of claim 1 , wherein prior to inserting either the decompressed cache data block or the compressed data block in the first tier, determining whether the first tier is full.

5 . The self-managed DRAM module of claim 4 , wherein in response to a determination that the first tier is full, choosing an existing cache data block from the first tier, compressing the existing cache data block, and inserting the resulting compressed cache data block into the second tier.

6 . The self-managed DRAM module of claim 5 , wherein in response to a determination that the second tier is full, choosing an existing compressed cache data block from the second tier and determining whether the existing compressed cache data block has been modified since being loaded from DRAM.

7 . The self-managed DRAM module of claim 6 , wherein:

in response to a determination that the existing compressed cache data block has not been modified since being loaded from DRAM, deleting the existing compressed cache data block; and

in response to a determination that the existing compressed cache data block has been modified since being loaded from DRAM, writing the existing compressed cache data block to DRAM.

8 . The self-managed DRAM module of claim 1 , wherein a size of the cache data block is equal to a DRAM compressed block.

9 . A method of handling data access requests with a self-managed dynamic random-access memory (DRAM) module, comprising:

providing a plurality of DRAM;

providing a controller chip having:

a plurality of DRAM controllers for managing the plurality of DRAM independently of a central processing unit (CPU) chip;

a CXL/PCIe input/output (I/O) engine for connecting the controller chip to the CPU chip via a CXL/PCIe channel; and

a cache implemented independently from the CPU chip with a first tier configured for storing uncompressed data and a second tier configured for storing compressed data;

searching the first tier for an uncompressed cache data block to service a data access request received via the CXL/PCIe channel;

in response to a first tier hit, serving the uncompressed cache data block to service the data access request;

in response to a first tier miss, searching the second tier for a compressed cache data block to service the data access request;

in response to a second tier hit, decompressing the compressed cache data block to generate a decompressed cache data block, and inserting the decompressed cache data block into the first tier to service the data access request; and

in response to a second tier miss, reading a compressed data block from DRAM, decompressing the compressed data block to generate a decompressed data block, and inserting the decompressed data block into the first tier to service the data access request.

10 . The method of claim 9 , wherein the first tier is implemented using a set-associative cache architecture and the second tier is implemented with a key-value data store architecture, wherein a key is a logical DRAM address and the compressed cache data block is identified from a value associated with the key.

11 . The method of claim 9 , wherein prior to inserting either the decompressed cache data block or the compressed data block in the first tier, determining whether the first tier is full.

12 . The method of claim 11 , wherein in response to a determination that the first tier is full, choosing an existing cache data block from the first tier, compressing the existing cache data block, and inserting the resulting compressed cache data block into the second tier.

13 . The method of claim 12 , wherein in response to a determination that the second tier is full, choosing an existing compressed cache data block from the second tier and determining whether the existing compressed cache data block has been modified since being loaded from DRAM.

14 . The method of claim 13 , wherein:

in response to a determination that the existing compressed cache data block has not been modified since being loaded from DRAM, deleting the existing compressed cache data block; and

in response to a determination that the existing compressed cache data block has been modified since being loaded from DRAM, writing the existing compressed cache data block to DRAM.

15 . The method of claim 9 , wherein a size of the cache data block is equal to a DRAM compressed block.

16 . A self-managed dynamic random-access memory (DRAM) module, comprising:

a plurality of DRAM; and

a controller chip having:

a plurality of DRAM controllers for managing the plurality of DRAM independently of a central processing unit (CPU) chip;

a CXL/PCIe input/output (I/O) engine for communicatively connecting the controller chip to the CPU chip via a CXL/PCIe channel; and

a cache implemented independently from the CPU chip with a first tier configured for storing uncompressed data and a second tier configured for storing compressed data, wherein a first tier cache data block size is configured to be smaller than a DRAM compressed data block size and wherein a second tier compressed cache data block size is configured to be smaller than the DRAM compressed data block size and equal or larger than the first tier cache data block size, and wherein the controller chip is configured to handle data access requests received via the CXL/PCIe channel according to a process that includes:

searching the first tier for an uncompressed cache data block to service the data access request;

in response to a first tier hit, serving the uncompressed cache data block to service the data access request;

in response to a first tier miss, searching the second tier for a compressed cache data block to service the data access request;

in response to a second tier hit, decompressing the compressed cache data block to generate a plurality of decompressed data chunks, choosing a subset of the decompressed data chunks covering the data access request and inserting the subset of decompressed data chunks into the first tier to service the data access request; and

in response to a second tier miss, reading a compressed data block from DRAM, decompressing the compressed data block to generate a decompressed data block, and inserting a set of data chunks from the decompressed data block into the first tier to service the data access request.

17 . The self-managed DRAM module of claim 16 , further comprising:

choosing and compressing a chosen data block covering the requesting data to generates a compressed chosen data block; and

inserting the compressed chosen data block into the second tier.

18 . The self-managed DRAM module of claim 16 , wherein the first tier is implemented using a set-associative cache architecture and the second tier is implemented with a key-value data store architecture, wherein a key is a logical DRAM address and the compressed cache data block is determined from a value associated with the key.

19 . The self-managed DRAM module of claim 16 , wherein prior to inserting either the subset of decompressed data chunks or the set data chunks into the first tier, determining whether the first tier is full.

20 . The self-managed DRAM module of claim 19 , wherein in response to a determination that the first tier is full, choosing at least one existing cache data block from the first tier to evict, compressing the at least one existing cache data block, and inserting at least one existing compressed cache data block into the second tier.