Systems and methods for performing data compression in memory
An appliance is described. A write engine may process a write request from an application to write a first data into a memory. The write request may including the first data and an address. A compression engine may compress the first data to produce a first compressed data. A write module may store the first compressed data in the memory. The first data may be a first part of a page that may further include a second data as a second part. The first compressed data may be a first part of a compressed page that may further include a second compressed data as a second part.
1. An appliance, comprising:
a write circuit to process a write request from an application executing on a first processor on a first host to write a first data into a memory, the write request including the first data and an address;
a compression circuit to compress the first data to produce a first compressed data; and
a controller to store the first compressed data in a memory page in the memory and to store a metadata identifying where the first compressed data is stored within the memory page in the memory,
wherein the first data is a first part of a page, the page further including a second data as a second part of the page,
wherein the first compressed data is a first part of a compressed page, the compressed page further including a second compressed data as a second part of the compressed page, the second compressed data being a compressed version of the second data,
wherein the appliance is configured to receive a second write request from a second application executing on a second processor on a second host to write a third data into the memory, and
wherein the appliance, the first host, and the second host are separate elements.
2. The appliance according to claim 1 , wherein the memory includes a volatile storage to store the first compressed data and a non-volatile storage as a backup to the volatile storage.
3. An appliance according to claim 1 , wherein the write circuit includes an access pattern detector to detect an access pattern for the first data by the application.
4. An appliance according to claim 3 , wherein a fabric manager is configured to provide a workload hint to the access pattern detector regarding the access pattern for the first data by the application.
5. An appliance according to claim 1 , wherein the compression circuit is configured to determine a compression type for the first data based at least in part on a first compression ratio of the first data using a first compression algorithm and a second compression ratio of the first data using a second compression algorithm.
6. An appliance according to claim 1 , wherein the controller is configured to store the first compressed data at a second address in the memory.
7. An appliance according to claim 6 , further comprising an address map to map the address to the second address.
8. An appliance according to claim 3 , wherein the compression circuit is configured to compress the first data to produce a first compressed data based at least in part on the access pattern for the first data by the application.
9. The appliance according to claim 1 , wherein:
the first data is compressed using a first compression algorithm;
the second data is compressed using a second compression algorithm; and
the first compression algorithm is different from the second compression algorithm.
10. The applicant according to claim 1 , wherein:
the page further includes a fourth data as a third part of the page; and
the appliance is configured to store the fourth data as an uncompressed data in the memory page in the memory.
11. An appliance, comprising:
a read circuit to process a read request from an application executing on a first processor on a first host to read a first data from a memory at an address;
a controller to read a metadata identifying where a first compressed data is stored within a memory page in the memory based on the address and to read the first compressed data from the memory page in the memory based at least in part on the metadata; and
a decompression circuit to decompress the first compressed data to produce a first data,
wherein the read circuit is configured to return the first data to the application,
the first data is a first part of a page, the page further including a second data as a second part of the page,
wherein the first compressed data is a part of a compressed page, the compressed page further including a second compressed data as a second part of the compressed page,
the second compressed data being a compressed version of the second data,
wherein the appliance is configured to receive a second read request from a second application executing on a second processor on a second host to read a third data from the memory, and
wherein the appliance, the first host, and the second host are separate elements.
12. An appliance according to claim 11 , wherein the decompression circuit is configured to determine a compression type for the first compressed data, the compression type associated with the first compressed data.
13. An appliance according to claim 11 , wherein the decompression circuit is configured to decompress the first compressed data to produce the first data inline.
14. An appliance according to claim 11 , wherein:
the appliance further comprises an address map to map the address to a second address; and
the controller is configured to read the first compressed data from the memory at the second address.
15. The appliance according to claim 11 , wherein the read circuit is configured to receive the read request from the application across a network.
16. A method, comprising:
receiving, at an appliance, a write request from an application executing on a first processor on a first host to write a data into a memory, the write request including the data and an address;
determining, at the appliance, an access pattern for the data by the application;
determining, at the appliance, a compression type for the data;
storing, by the appliance, the data in a memory page in the memory based at least in part on the access pattern for the data or the compression type for the data; and
storing, by the appliance, a metadata identifying where the data is stored within the memory page in the memory,
wherein the appliance is configured to receive a second write request from a second application executing on a second processor on a second host to write a third data into the memory, and
wherein the appliance, the first host, and the second host are separate elements.
17. A method according to claim 16 , wherein determining the access pattern for the data by the application includes receiving a workload hint regarding the access pattern for the data by the application.
18. A method according to claim 16 , wherein determining the compression type for the data includes:
determining a first compression ratio for the data for a first compression algorithm;
determining a second compression ratio for the data for a second compression algorithm; and
determining the compression type based at least in part on the first compression ratio and second compression ratio.
19. A method according to claim 16 , wherein storing the data in the memory based at least in part on the access pattern for the data or the compression type for the data includes storing the data in the memory at a second address based at least in part on the access pattern for the data or the compression type for the data.
20. A method according to claim 19 , wherein storing the data in the memory at the second address based at least in part on the access pattern for the data or the compression type for the data includes:
compressing the data to produce a compressed data based at least in part on the access pattern for the data or the compression type for the data;
storing the compressed data in the memory at the second address; and
storing a map from the address to the second address.