Method and apparatus for offloading data processing to hybrid storage devices
Systems and methods for offloading data transformation from a host to a hybrid solid state drive (HSSD) are described. In one such method, the HSSD receives initial data from the host and stores the data at a first non-volatile memory (NVM). The HSSD receives a transformation command from the host to offload data transformation. The HSSD copies the data from the first NVM to a second NVM that is configured to provide a finer granularity of data access than that of the first NVM. Then the HSSD transforms the data at the second NVM utilizing the configured processing circuit. The HSSD may store the result in the first NVM and/or second NVM, and send it to the host.
1. A method of transforming data at a hybrid solid state drive (HSSD) comprising a first non-volatile memory (NVM), a second NVM, and a processing circuit, the method comprising: receiving data from a host and storing the data at the first NVM; receiving a transformation command from the host; storing a copy of the data at the second NVM that is configured to provide a finer granularity of random data access than that of the first NVM, wherein a smallest addressable unit of the second NVM is smaller than a smallest addressable unit of the first NVM; and transforming, based on the transformation command, the data stored at the second NVM utilizing the processing circuit, using the first NVM to store initial data and the second NVM for data processing, to generate transformed data stored in at least one of the first NVM or second NVM.
2. The method of claim 1 , wherein the processing circuit comprises at least one of a processor, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).
3. The method of claim 1 , wherein the transforming comprises:
utilizing a MapReduce model to process the data stored at the second NVM; and
storing temporary values generated by the MapReduce model at the second NVM.
4. The method of claim 1 , wherein the transforming comprises:
processing the data stored at the second NVM to generate intermediate data according to the transformation command; and
storing the intermediate data at the second NVM.
5. The method of claim 1 , further comprising:
receiving metadata of the data from the host, wherein the metadata comprises information on the data stored at the first NVM.
6. The method of claim 1 , wherein the transforming comprises:
processing a first data type of the data using a first data processing method; and
processing a second data type of the data using a second data processing method simultaneously with the processing of the first data type.
7. The method of claim 1 , wherein the transforming comprises:
performing a first data transformation on the data; and
performing a second data transformation, different from the first data transformation, on the data.
8. The method of claim 1 ,
wherein the first NVM comprises a NAND Flash, and
wherein the second NVM comprises at least one of magnetoresistive random-access memory, non-volatile RAM, ferroelectric RAM, phase-change RAM, or resistive random-access memory (ReRAM).
9. The method of claim 8 , wherein the first NVM is page-addressable, and the second NVM is byte-addressable.
10. The method of claim 1 , further comprising at least one of:
storing the transformed data in the first NVM;
storing the transformed data in the second NVM; or
sending the transformed data to the host.
11. A hybrid solid state drive (HSSD) comprising: a first non-volatile memory (NVM); a second NVM that is configured to provide a finer granularity of random data access than that of the first NVM, wherein a smallest addressable unit of the second NVM is smaller than a smallest addressable unit of the first NVM; and a processing circuit operatively coupled to the first NVM and the second NVM, wherein the processing circuit is configured to: receive data from a host and store the data at the first NVM; receive a transformation command from the host; store a copy of the data at the second NVM; and transform, based on the transformation command, the data stored at the second NVM, using the first NVM to store initial data and the second NVM for data processing, to generate transformed data stored in at least one of the first NVM or the second NVM.
12. The HSSD of claim 11 , wherein the processing circuit comprises at least one of a processor, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).
13. The HSSD of claim 11 , wherein the processing circuit is further configured to:
utilize a MapReduce model to process the data stored at the second NVM; and
store temporary values generated by the MapReduce model at the second NVM.
14. The HSSD of claim 11 , wherein the processing circuit is further configured to:
process the data stored at the second NVM to generate intermediate data according to the transformation command; and
store the intermediate data at the second NVM.
15. The HSSD of claim 11 , wherein the processing circuit is further configured to receive metadata of the data from the host, and wherein the metadata comprises information on the data stored at the first NVM.
16. The HSSD of claim 11 , wherein the processing circuit is further configured to transform the data by:
processing a first data type of the data using a first data processing method; and
processing a second data type of the data using a second data processing method simultaneously with the processing of the first data type.
17. The HSSD of claim 11 , wherein the processing circuit is further configured to transform the data by:
performing a first data transformation on the data; and
performing a second data transformation, different from the first data transformation, on the data.
18. The HSSD of claim 11 ,
wherein the first NVM comprises a NAND Flash, and
wherein the second NVM comprises at least one of magnetoresistive random-access memory, non-volatile RAM, ferroelectric RAM, phase-change RAM, or resistive random-access memory (ReRAM).
19. The HSSD of claim 18 , wherein the first NVM is page-addressable, and the second NVM is byte-addressable.
20. The HSSD of claim 11 , wherein the processing circuit is further configured to, at least one of:
store the transformed data in the first NVM;
store the transformed data in the second NVM; or
send the transformed data to the host.
21. A hybrid solid state drive (HSSD) comprising: a first non-volatile memory (NVM); a second NVM that is configured to provide a finer granularity of random data access than that of the first NVM, wherein a smallest addressable unit of the second NVM is smaller than a smallest addressable unit of the first NVM; means for receiving data from a host and storing the data at the first NVM; means for receiving a transformation command from the host; means for storing a copy of the data at the second NVM; and means for transforming, based on the transformation command, the data stored at the second NVM, using the first NVM to store initial data and the second NVM for data processing, to generate transformed data stored in at least one of the first NVM or the second NVM.
22. The HSSD of claim 21 , wherein the means for transforming comprises at least one of a processor, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).
23. The HSSD of claim 21 , wherein the means for transforming is configured to:
utilize a MapReduce model to process the data stored at the second NVM; and
store temporary values generated by the MapReduce model at the second NVM.
24. The HSSD of claim 21 , wherein the means for transforming is configured to:
process the data stored at the second NVM to generate intermediate data according to the transformation command; and
store the intermediate data at the second NVM.
25. The HSSD of claim 21 , further comprising:
means for receiving metadata of the data from the host, wherein the metadata comprises information on the data stored at the first NVM.
26. The HSSD of claim 21 , wherein the means for transforming is further configured to:
process a first data type of the data using a first data processing method; and
process a second data type of the data using a second data processing method simultaneously with the processing of the first data type.
27. The HSSD of claim 21 , wherein the means for transforming is further configured to:
perform a first data transformation on the data; and
perform a second data transformation, different from the first data transformation, on the data.
28. The HSSD of claim 21 ,
wherein the first NVM comprises a NAND Flash, and
wherein the second NVM comprises at least one of magnetoresistive random-access memory, non-volatile RAM, ferroelectric RAM, phase-change RAM, or resistive random-access memory (ReRAM).
29. The HSSD of claim 28 , wherein the first NVM is page-addressable, and the second NVM is byte-addressable.
30. The HSSD of claim 21 , further comprising means for, at least one of:
storing the transformed data in the first NVM;
storing the transformed data in the second NVM; or
sending the transformed data to the host.
31. The method of claim 1 , further comprising:
registering associations between a plurality of data types and a plurality of data processing methods according to the transformation command, wherein the plurality of data processing methods are respectively implemented at a plurality of programmable functional blocks of the processing circuit;
determining a first data type of the plurality of data types corresponding to the data stored at the second NVM;
selecting a first data processing method of the plurality of data processing methods, associated with the first data type; and
transforming the data stored at the second NVM using the first data processing method implemented at the corresponding programmable functional block.
32. The HSSD of claim 11 , wherein the processing circuit is further configured to:
register associations between a plurality of data types and a plurality of data processing methods according to the transformation command, wherein the plurality of data processing methods are respectively implemented at a plurality of programmable functional blocks of the processing circuit;
determine a first data type of the plurality of data types corresponding to the data stored at the second NVM;
select a first data processing method of the plurality of data processing methods, associated with the first data type; and
transform the data stored at the second NVM using the first data processing method implemented at the corresponding programmable functional block.