Techniques for directed data migration
A host stores “context” metadata for logical block addresses (LBAs) in a manner tied to physical location. Notwithstanding log-structured or copy on write processes, the host is then provided with immediate context when the host is called upon to assist a memory controller with data identified by physical location, for example, for memory reconfiguration, garbage collection, wear leveling or other processes. The metadata for example can provide the host with insight as to which data may be moved to enhance performance optimization and where that data can be placed. In one embodiment, the host writes back one or more references that span multiple layers of indirection in concert with write of the underlying data; in another embodiment, the context can point to other metadata.
1. An apparatus comprising instructions stored on a nontransitory storage medium, the instructions, when executed to cause at least one processor of a host system to:
transmit write requests to a storage drive, including respective logical block addresses and associated data, and respective source identifiers used by the host system in association with the respective data, wherein the storage drive is to store the associated data at respective physical storage locations within the storage drive and is also to store the source identifiers as metadata associated with the respective data;
receive, at a time after initial storage of the respective data in the storage drive, a communication from the storage drive of two or more of the source identifiers; and
execute a consolidation process, using the two or more of the source identifiers, to identify that data corresponding to the two or more of the source identifiers is related, and cause consolidation of the data corresponding to the two or more source identifiers within the storage drive, within a common subset of physical memory within the storage drive.
2. The apparatus of claim 1 , wherein the instructions, when executed, are to cause the at least one processor of the host system to transmit read requests for the data corresponding to the two or more of the source identifiers, at a time after execution of the consolidation process, using the corresponding logical block addresses.
3. The apparatus of claim 1 , wherein the source identifiers comprise at least one of a file identification, a client identification, an owning object identification, or a back reference.
4. The apparatus of claim 1 , wherein the instructions, when executed, and as part of the consolidation process, are to transmit a request to the storage drive for at least one item of data corresponding to the two or more of the source identifiers, to copy the at least one item of data from a source physical memory location within the storage drive to the common subset of physical memory within the storage drive, without sending the data being copied to the host system.
5. The apparatus of claim 4 , wherein:
the storage drive is a flash memory drive having a memory controller and flash memory dies;
the memory controller is to perform a garbage collection operation that will change a mapping between one or more of the logical block addresses and the respective physical storage locations; and
the two or more of the source identifiers are to be received from the storage drive following performance of the garbage collection operation.
6. The apparatus of claim 4 , wherein:
the storage drive is a flash memory drive having a memory controller and flash memory dies;
the memory controller is to perform a garbage collection operation that will change a mapping between one or more of the logical block addresses and the respective physical storage locations;
the two or more of the source identifiers are to be received from the storage drive prior to performance of the garbage collection operation; and
the instructions, when executed, are to cause the at least one processor to specify at least one destination within flash memory as part of the consolidation operation as a destination for the garbage collection operation.
7. The apparatus of claim 1 , wherein the respective source identifiers comprise forward references used by the host system, the forward references encompassing multiple levels of indirection.
8. The apparatus of claim 1 , wherein the respective source identifiers comprise field-delimited references, each field corresponding to a different level of host system indirection.
9. The apparatus of claim 1 , wherein:
the apparatus further comprises instructions that, when executed, are to cause the at least one processor of the host system to transmit a query to the storage drive; and
the communication from the storage drive is to be received in response to receipt by the storage drive of the query.
10. The apparatus of claim 1 , wherein the communication from the storage drive is to be received by the host system as a function of occurrence of a predetermined event.
11. The processor-implemented method of claim 1 , wherein the communication from the storage drive is to be received by the host system as a function of occurrence of a predetermined event.
12. A processor-implemented method of operating a host system, the method comprising:
transmitting write requests to a storage drive, including respective logical block addresses and associated data, and respective source identifiers used by the host system in association with the respective data, wherein the storage drive is to store the associated data at respective physical storage locations within the storage drive and is also to store the source identifiers as metadata associated with the respective data;
receiving, at a time after initial storage of the respective data in the storage drive, a communication from the storage drive of two or more of the source identifiers; and
executing a consolidation process, using the two or more of the source identifiers, to identify that data corresponding to the two or more of the source identifiers is related, and causing consolidation of the data corresponding to the two or more source identifiers within the storage drive, within a common subset of physical memory within the storage drive.
13. The processor-implemented method of claim 12 , further comprising transmitting read requests to the storage drive for the data corresponding to the two or more of the source identifiers, at a time after execution of the consolidation process, using the respective logical block addresses.
14. The processor-implemented method of claim 12 , wherein the source identifiers comprise at least one of a file identification, a client identification, an owning object identification, or a back reference.
15. The processor-implemented method of claim 12 , further comprising, as part of the consolidation process, transmitting a request to the storage drive for at least one item of data corresponding to the two or more of the source identifiers, to copy the at least one item of data from a source physical memory location within the storage drive to the common subset of physical memory within the storage drive, without sending the data being copied to the host system.
16. The processor-implemented method of claim 15 , wherein:
the storage drive is a flash memory drive having a memory controller and flash memory dies;
the memory controller is to perform a garbage collection operation that will change a mapping between one or more of the logical block addresses and the respective physical storage locations; and
the method comprises receiving the two or more of the source identifiers from the storage drive following performance of the garbage collection operation.
17. The processor-implemented method of claim 15 , wherein:
the storage drive is a flash memory drive having a memory controller and flash memory dies;
the memory controller is to perform a garbage collection operation that will change a mapping between one or more of the logical block addresses and the respective physical storage locations; and
the method further comprises receiving the two or more of the source identifiers from the storage drive prior to performance of the garbage collection operation, and specifying at least one destination within flash memory as part of the consolidation operation as a destination for the garbage collection operation.
18. The processor-implemented method of claim 12 , wherein the respective source identifiers comprise forward references used by the host system, the forward references encompassing multiple levels of indirection.
19. The processor-implemented method of claim 12 , wherein the respective source identifiers comprise field-delimited references, each field corresponding to a different level of host system indirection.
20. The processor-implemented method of claim 12 , wherein:
the processor-implemented method further comprises transmitting a query to the storage drive; and
receiving the communication from the storage drive is responsive to receipt by the storage drive of the query.