Recycle optimization in storage engine
When performing a recycling operation on a storage device, a storage system may use or create a data buffer in the storage device, and designate the data buffer to temporarily store data of data blocks to be recycled in the storage device using direct memory access (DMA) operations that are performed internally in the storage device, without the need of reading the data of the data blocks from the storage device and writing the data into a host memory of the storage system, thereby saving or reducing the consumptions of the communication bandwidth of a communication channel between the storage system and the storage device, and the memory bandwidth of the host memory.
1. A method implemented by one or more processors of a storage system, the method comprising:
performing a recycle read for a first storage device, performing the recycle read comprising:
sending a first instruction to the first storage device to transfer record data from a plurality of sectors of the first storage device to a data buffer of the first storage device; and
reading corresponding sector metadata of the plurality of sectors of the first storage device into a host memory of the storage system; and
performing a recycle write for a second storage device, performing the recycle write comprising:
sending a second instruction to the first storage device to transfer valid records of the record data in the data buffer of the first storage device to one or more new sectors of the second storage device;
creating new sector metadata for the one or more sectors in the host memory; and
transferring the new sector metadata from the host memory to the one or more new sectors of the second storage device.
2. The method of claim 1 , wherein the record data is transferred from the plurality of sectors of the first storage device to the data buffer of the first storage device using direct memory access operations.
3. The method of claim 1 , wherein the corresponding sector metadata of the plurality of sectors of the first storage device is read into the host memory of the storage system through a communication channel between the first storage device and the storage system.
4. The method of claim 1 , wherein the recycle read and the recycle write are performed in response to one of: a garbage collection operation, a record merging operation, a wear-leveling operation, a read disturb operation, an error handling operation.
5. The method of claim 1 , wherein the record data in the plurality of sectors of the first storage device comprises the valid records, and records that are labeled as deleted.
6. The method of claim 1 , further comprising: updating a record mapping table in the host memory, the record mapping table comprising at least relationships between identifiers of the valid records and locations of the valid records.
7. The method of claim 1 , wherein the first storage device is identical to the second storage device, and sending the second instruction to the first storage device to transfer the valid records of the record data in the data buffer of the first storage device to the one or more new sectors of the second storage device comprises causing the first storage device to perform direct memory access operations to transfer the valid records of the record data in the data buffer to the one or more new sectors.
8. The method of claim 1 , wherein the first storage device is different from the second storage device, and sending the second instruction to the first storage device to transfer the valid records of the record data in the data buffer of the first storage device to the one or more new sectors of the second storage device comprises causing the first storage device to transfer the valid records of the record data in the data buffer to the one or more new sectors of the second storage device through a communication channel between the first storage device and the second storage device.
9. The method of claim 1 , wherein the corresponding sector metadata of the plurality of sectors comprises location information of record headers of respective first records in the plurality of sectors.
10. One or more processor readable media storing executable instructions that, when executed by one or more processors of a storage system, cause the one or more processors to perform acts comprising:
performing a recycle read for a first storage device, performing the recycle read comprising:
sending a first instruction to the first storage device to transfer record data from a plurality of sectors of the first storage device to a data buffer of the first storage device; and
reading corresponding sector metadata of the plurality of sectors of the first storage device into a host memory of the storage system; and
performing a recycle write for a second storage device, performing the recycle write comprising:
sending a second instruction to the first storage device to transfer valid records of the record data in the data buffer of the first storage device to one or more new sectors of the second storage device;
creating new sector metadata for the one or more sectors in the host memory; and
transferring the new sector metadata from the host memory to the one or more new sectors of the second storage device.
11. The one or more processor readable media of claim 10 , wherein the record data is transferred from the plurality of sectors of the first storage device to the data buffer of the first storage device using direct memory access operations.
12. The one or more processor readable media of claim 10 , wherein the corresponding sector metadata of the plurality of sectors of the first storage device is read into the host memory of the storage system through a communication channel between the first storage device and the storage system.
13. The one or more processor readable media of claim 10 , wherein the recycle read and the recycle write are performed in response to one of: a garbage collection operation, a record merging operation, a wear-leveling operation, a read disturb operation, an error handling operation.
14. The one or more processor readable media of claim 10 , wherein the record data in the plurality of sectors of the first storage device comprises the valid records, and records that are labeled as deleted.
15. The one or more processor readable media of claim 10 , the acts further comprising: updating a record mapping table in the host memory, the record mapping table comprising at least relationships between identifiers of the valid records and locations of the valid records.
16. The one or more processor readable media of claim 10 , wherein the first storage device is identical to the second storage device, and sending the second instruction to the first storage device to transfer the valid records of the record data in the data buffer of the first storage device to the one or more new sectors of the second storage device comprises causing the first storage device to perform direct memory access operations to transfer the valid records of the record data in the data buffer to the one or more new sectors.
17. The one or more processor readable media of claim 10 , wherein the first storage device is different from the second storage device, and sending the second instruction to the first storage device to transfer the valid records of the record data in the data buffer of the first storage device to the one or more new sectors of the second storage device comprises causing the first storage device to transfer the valid records of the record data in the data buffer to the one or more new sectors of the second storage device through a communication channel between the first storage device and the second storage device.
18. The one or more processor readable media of claim 10 , wherein the corresponding sector metadata of the plurality of sectors comprises location information of record headers of respective first records in the plurality of sectors.
19. A storage system comprising:
one or more processors; and
host memory storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
performing a recycle read for a first storage device, performing the recycle read comprising:
sending a first instruction to the first storage device to transfer record data from a plurality of sectors of the first storage device to a data buffer of the first storage device; and
reading corresponding sector metadata of the plurality of sectors of the first storage device into the host memory; and
performing a recycle write for a second storage device, performing the recycle write comprising:
sending a second instruction to the first storage device to transfer valid records of the record data in the data buffer of the first storage device to one or more new sectors of the second storage device;
creating new sector metadata for the one or more sectors in the host memory; and
transferring the new sector metadata from the host memory to the one or more new sectors of the second storage device.
20. The storage system of claim 19 , wherein the record data is transferred from the plurality of sectors of the first storage device to the data buffer of the first storage device using direct memory access operations, and the corresponding sector metadata of the plurality of sectors of the first storage device is read into the host memory through a communication channel between the first storage device and the second storage device.