Data compression in a data transform accelerator
A method includes obtaining, by a data transform accelerator, a first command and first source data associated with the first command. The method also includes generating, by the data transform accelerator, one or more containers using the first command. Each of the one or more containers may have a container size and may be operable to store compressed source data. The method further includes obtaining a mode of operation for a data compression operation. The method also includes performing the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data. The method further includes storing the compressed source data in a first container of the one or more containers.
1 . A method, comprising:
obtaining, by a data transform accelerator, a first command and first source data associated with the first command;
generating, by the data transform accelerator, one or more containers using the first command, each of the one or more containers having a container size and are operable to store compressed source data;
obtaining a mode of operation for a data compression operation;
performing the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data; and
storing the compressed source data in a first container of the one or more containers.
2 . The method of claim 1 , further comprising determining the container size associated with the one or more containers using metadata included the first command.
3 . The method of claim 1 , wherein the container size is determined based on at least one of an operating system, a platform software, a storage architecture, a performance tuning, and a user input.
4 . The method of claim 3 , wherein the user input comprises an estimate of a number of containers to be used per command.
5 . The method of claim 3 , wherein the performance tuning includes a tuning a data compression parameter, where the data compression parameter is tuned in view of at least one of a container size optimization, a data compression maximization, and a user input.
6 . The method of claim 1 , further comprising:
obtaining a second command and second source data associated with the first command, the first command and the second command included in a session;
performing the data compression operation to a second portion of the first source data and a first portion of the second source data using the mode of operation to obtain second compressed source data; and
storing the second compressed source data in a second container of the one or more containers.
7 . The method of claim 6 , wherein the second container includes one or more dependencies on the first container and the first container includes a frame header comprising compression parameters associated with the data compression operation, such that in response to a request to decompress the second container, the first container is first decompressed and the frame header is used to decompress the second container.
8 . The method of claim 6 , wherein the first command and the second command included in the session one or more share session parameters, the session parameters comprising at least a compression algorithm.
9 . The method of claim 1 , wherein the container size of the one or more containers is uniform.
10 . The method of claim 1 , further comprising generating a report associated with the first source data, the report including an amount of the first source data consumed in the data compression operation.
11 . The method of claim 10 , further comprising transmitting the report to an external device associated with the first source data.
12 . The method of claim 1 , wherein the mode of operation is stateless data compression, and a data decompression operation on the first container may be independent of the data decompression operation on a subsequent container.
13 . The method of claim 1 , further comprising reserving a portion of the first container for a non-volatile memory express (NVMe) protection information (PI) insertion.
14 . A data transform accelerator comprising:
one or more data transform engines including at least a compression engine; and
a processing device operable to:
obtain a first command and first source data associated with the first command;
generate one or more containers using the first command, each of the one or more containers having a container size and are operable to store compressed source data;
obtain a mode of operation for a data compression operation;
perform, using the compression engine, the data compression operation to a first portion of the first source data using the mode of operation to obtain the compressed source data; and
store the compressed source data in a first container of the one or more containers.
15 . The data transform accelerator of claim 14 , wherein the container size of the one or more containers is uniform.
16 . The data transform accelerator of claim 14 , wherein the processing device is further operable to:
obtain a second command and second source data associated with the first command, the first command and the second command included in a session;
perform the data compression operation to a second portion of the first source data and a first portion of the second source data using the mode of operation to obtain second compressed source data; and
store the second compressed source data in a second container of the one or more containers.
17 . The data transform accelerator of claim 14 , wherein the container size is determined based on at least one of an operating system, a platform software, a storage architecture, a performance tuning, and a user input.
18 . The data transform accelerator of claim 17 , wherein the performance tuning includes a tuning a data compression parameter, where tuning the data compression parameter is performed in view of at least one of a container size optimization, a data compression maximization, and a user input.
19 . The data transform accelerator of claim 14 , wherein:
the compression engine is operable to a report associated with the first source data, the report including an amount of the first source data consumed in the data compression operation; and
the processing device is operable to transmit the report to an external device associated with the first source data.
20 . The data transform accelerator of claim 14 , wherein the processing device is further operable to reserve a portion of the first container for subsequent transform operations including at least one of: padding, encryption, authentication tag generation, embedding, and NVMe PI insertion.