Nonvolatile memory controller with delegated processing
Processing functions are offloaded to a memory controller for nonvolatile memory by a host in connection with write data. The nonvolatile memory executes these functions, producing processed data that must be written into memory; for example, the offloaded functions can include erasure coding, with the nonvolatile memory controller generating redundancy information that must be written into memory. The memory controller holds this information in internal RAM and then later writes this information into nonvolatile memory according to dynamically determined write time and/or destinations selected by the host, so as to not collide with host data access requests. In one embodiment, the memory is NAND flash memory and the memory controller is a cooperative memory controller that permits the host to schedule concurrent operations in respective, configurable virtual block devices which have been configured by the host out of a pool of structural flash memory structures managed by the memory controller.
1. A storage drive comprising:
flash memory having physical storage locations;
an interface to receive, from a storage manager of a host, write commands and, for each of the write commands, an associated host-provided address and associated data, wherein the write commands are compatible with a Non-Volatile Memory Express (NVMe) command format; and
circuitry, including at least one processor, to
control the flash memory,
for each one of the write commands, map the host-provided address which is associated with the one of the write commands to a respective one of the physical storage locations, and store the associated data in the respective one of the physical storage locations,
store metadata representing each item of data which has been stored in one of the physical storage locations, and
responsive to receipt of an additional command from the storage manager of the host, via the interface, execute a deduplication function, and as part of executing the deduplication function:
detect, using said metadata, duplication of data stored in the flash memory by two different ones of the write commands; and
remap the host-provided addresses associated with at least one the two different ones of the write commands, such that two different host-provided addresses which are respectively associated with the two different ones of the write commands each are thereafter mapped by the storage drive to a single, common one of the physical storage locations in the flash memory.
2. The storage drive of claim 1 wherein the circuitry is able to map each host-provided address associated with one of the write commands to multiple alternative ones of the physical storage locations in the flash memory, at respective times.
3. The storage drive of claim 1 wherein, following the remapping:
the interface is also to receive, via the interface, a first read command which specifies a first one of the two different host-provided addresses and a second read command which specifies a second one of the two different host-provided addresses, wherein the first read command and the second read command are each also compatible with the NVMe command format; and
the circuitry is to service each of the first read command and the second read command by identifying the single, common one of the physical storage locations in the flash memory, causing retrieval of data stored at the single, common one of the physical storage locations in the flash memory, and causing transmission of the retrieved data by the storage drive to the storage manager of the host.
4. The storage drive of claim 1 wherein:
the circuitry is to map a first namespace to a first plurality of the physical storage locations in the flash memory and a second namespace to a second plurality of the physical storage locations in the flash memory;
the second command is specific to the first namespace; and
the circuitry is to execute the deduplication function in a manner unique to the first namespace.
5. The storage drive of claim 1 wherein:
the circuitry is to, responsive to detecting the duplication of data, cause transmission by the storage drive to the storage manager of the host of information dependent on existence of the duplication of data;
the additional command is responsive to the information transmitted to the storage manager of the host from the storage drive.
6. The storage drive of claim 5 wherein the information transmitted to the storage manager of the host corresponds to a subset of logical storage space, wherein the additional command specifies the subset of logical storage space, and wherein the circuitry is to, responsive to the additional command, execute the deduplication function only in a subset of the flash memory which corresponds to the subset of logical storage space.
7. The storage drive of claim 1 wherein:
the circuitry is to, responsive to detecting the duplication of data, cause transmission by the storage drive to the storage manager of the host of information dependent on existence of the duplication of data;
the interface is to receive, via the interface, at least one further command from the storage manager of the host, subsequent to receipt of the additional command and responsive to the information transmitted to the storage manager of the host from the storage drive; and
the circuitry is to, responsive to the at least one further command:
map a third address to a third one of the physical storage locations,
move data stored at the single, common one of the physical storage locations in the flash memory from the single, common one of the physical storage locations in the flash memory to the third one of the physical storage locations, without sending the data being moved to the host,
cause erasure of the single, common one of the physical storage locations in the flash memory, and
thereafter cause the read of data stored at the third one of the physical storage locations in response to a read command, received via the interface, from the storage manager of the host which specifies the third address, wherein the read command is also compatible with the NVMe command format.
8. The storage drive of claim 7 wherein:
the at least one further command specifies the third address, such that the move of data is explicitly commanded by the host as part of the at least one third command.
9. The storage drive of claim 1 wherein the metadata is provided by the storage manager of the host in association with at least one of the write commands.
10. The storage drive of claim 1 wherein the circuitry is to generate the metadata on-board the storage drive.
11. The storage drive of claim 10 wherein the metadata comprises a hash, and wherein the circuitry is to detect the duplication of data by detecting the existence of two corresponding hashes.
12. The storage drive of claim 1 wherein each host-provided address associated with one of the write commands comprises a logical block address.
13. The storage drive of claim 12 wherein the flash memory is logically organized into pages, each of the pages comprising storage for plural logical block addresses.
14. The storage drive of claim 1 wherein:
the circuitry is to map a first namespace to a first plurality of the physical storage locations in the flash memory and a second namespace to a second plurality of the physical storage locations in the flash memory;
the storage drive further comprises a register;
the interface is to receive, via the interface, from the storage manager of the host a further command, and the circuitry is to program the register responsive to receipt of the further command; and
the circuitry is to execute the deduplication function in a manner dependent on programming stored in the register, wherein the deduplication function is selectively different for the first namespace than the second namespace according to the programming stored in the register.
15. The storage drive of claim 1 wherein the circuitry comprises instructions stored on at least physical storage medium that, when executed, are to configure the at least one processor to control the mapping of each host-provided address, the storage of associated data, the storing of the metadata, and the execution of the deduplication function.
16. The storage drive of claim 1 wherein:
the storage drive comprises on-board memory;
the circuitry is to store, in the on-board memory, storage location release information corresponding to the physical memory locations; and
the circuitry is to, in association with execution of the deduplication function, update the storage location release information stored in the on-board memory, so as to indicate that the respective one of the physical storage locations which was mapped to the host-provided address associated first one of the two different ones of the write commands has been released.
17. The storage drive of claim 16 wherein:
the interface is to receive a further command from the storage manager of the host, following the update of the storage location release information; and
the circuitry is to, responsive to the further command, control erasure of at least one of the physical storage locations in the flash memory, including the respective one of the physical storage locations which was mapped to the host-provided address associated first one of the two different ones of the write commands.
18. The storage drive of claim 1 wherein:
the storage drive comprises on-board memory;
the circuitry is to store, in the on-board memory, storage location release information corresponding to the physical memory locations; and
the interface is to receive a further command from the storage manager of the host; and
the circuitry is to, responsive to the further command, update the storage location release information.
19. The storage drive of claim 18 wherein the circuitry is to, responsive to the further command, dynamically control physical erasure of at least one of the physical storage locations in the flash memory.
20. The storage drive of claim 1 wherein the circuitry is further to transmit, via the interface, information, to the storage manager of the host, that explicitly conveys the remapped host-provided address back to the host, such that the host is able to update its logical addressing corresponding to the remapped host-provided address.
21. The storage drive of claim 1 wherein the additional command is interleaved by the host amongst the write commands and wherein the circuitry is operable to execute the deduplication function in a manner timed, relative to storage of the associated data, according to a manner in which the additional command is interleaved amongst the write commands.
22. A storage drive comprising:
flash memory having physical storage locations;
an interface to receive, from a storage manager of a host, write commands and, for each of the write commands, an associated host-provided address and associated data, wherein the write commands are compatible with a Non-Volatile Memory Express (NVMe) command format; and
circuitry, including at least one processor, to
control the flash memory,
for each one of the write commands, map the host-provided address which is associated with the one of the write commands to a respective one of the physical storage locations, and store the associated data in the respective one of the physical storage locations,
store metadata representing each item of data which has been stored in one of the physical storage locations, and
responsive to receipt of an additional command from the storage manager of the host, via the interface, execute a deduplication function, and as part of executing the deduplication function:
detect, using said metadata, duplication of data stored in the flash memory by two different ones of the write commands; and
remap the host-provided address associated with a first one of the two different ones of the write commands to instead map to the respective one of the physical storage locations which has been mapped to the host-provided address associated with a second one of the two different ones of the write commands, such that two different host-provided addresses respectively associated with the two different ones of the write commands each are thereafter mapped by the storage drive to a single, common one of the physical storage locations in the flash memory.
23. The storage drive of claim 22 wherein:
the circuitry is to, responsive to detecting the duplication of data by the two different ones of the write commands, cause transmission to the storage manager of the host, by the storage drive, of information dependent on existence of the duplication of data; and
the interface is to receive the additional command from the storage manager of the host, responsive to the information transmitted to the host by the storage drive.
24. The storage drive of claim 23 wherein the information transmitted to the storage manager of the host corresponds to a subset of logical storage space, wherein the additional command specifies the subset of logical storage space, and wherein the circuitry is to, responsive to the additional command, execute the deduplication function only in a subset of the flash memory which corresponds to the subset of logical storage space.
25. The storage drive of claim 22 wherein the circuitry is further to transmit, via the interface, information, to the storage manager of the host, that explicitly conveys the remapped host-provided address back to the host, such that the host is able to update its logical addressing corresponding to the remapped host-provided address.
26. The storage drive of claim 22 wherein the additional command is interleaved by the host amongst the write commands and wherein the circuitry is operable to execute the deduplication function in a manner timed, relative to storage of the associated data, according to a manner in which the additional command is interleaved amongst the write commands.