Techniques for directed data migration
A host stores “context” metadata for logical block addresses (LBAs) in a manner tied to physical location. Notwithstanding log-structured or copy on write processes, the host is then provided with immediate context when the host is called upon to assist a memory controller with data identified by physical location, for example, for memory reconfiguration, garbage collection, wear leveling or other processes. The metadata for example can provide the host with insight as to which data may be moved to enhance performance optimization and where that data can be placed. In one embodiment, the host writes back one or more references that span multiple layers of indirection in concert with write of the underlying data; in another embodiment, the context can point to other metadata.
1 . A storage system comprising:
a host comprising:
a host-side interface, and
a host storage manager; and
a storage drive comprising:
a drive-side interface;
flash memory having one or more namespaces and, for each namespace, subdivisions, each of the subdivisions associated with one or more erase units;
logic operable to cause the storage drive to:
receive, via the drive-side interface, from the storage manager of the host, a query, the query being compatible with a non-volatile memory express (NVMe) command format;
transmit to the host storage manager, via the drive-side interface, and responsive to the query, information representing the one or more namespaces and each associated subdivision;
wherein the host storage manager further comprises logic operable to cause the host storage manager to:
receive, via the host-side interface, the information representing the one or more namespaces and each subdivision;
responsively select, for each instance of write data to be stored in the storage drive, a selected namespace of the one or more namespaces and a selected subdivision of the subdivisions associated with the selected namespace; and
issue write commands to the storage drive, each of the write commands also being compatible with the NVMe format, wherein each of the write commands is accompanied by respective write data and associated context metadata; and
wherein the logic of the storage drive is further operable to cause the storage drive to:
receive the write commands from the host storage manager, via the drive-side interface; and
for each respective one of the write commands, identify the respective selected namespace and the respective selected subdivision, and store the respective write data in the one or more erase units which are associated with the identified subdivision, and store the associated context metadata in memory of the storage drive; and
wherein each said logic comprises at least one of (1) instructional logic stored on a physical storage medium that, when executed, is to control the operation of hardware circuitry of the storage drive, or (2) hardware circuitry.
2 . The storage system of claim 1 , wherein the logic of the storage drive is further operable to cause the storage drive to:
for each one of the subdivisions, track data validity information for the one or more erase units which are associated with the that one of the subdivisions;
dependent on the tracked data validity information, perform garbage collection operations in a manner dedicated to a given one of the subdivisions; and
as part of the garbage collection operations, cause the storage drive to copy write data which is stored in the one or more erase units associated with the given one of the subdivisions to a destination erase unit of the flash memory, the destination erase unit being selected dependent on the context metadata associated with the write data being copied.
3 . The storage system of claim 2 , wherein:
for each given one of the write commands, the logic of the storage drive is also operable to cause the storage drive to store the associated context metadata in the one or more erase units which are associated with the selected subdivision which was identified from the given one of the write commands; and
the logic of the storage drive is further operable to, as part of the garbage collection operation, cause the storage drive, to also copy, to the destination erase unit, the context metadata which is associated with the write data being copied.
4 . The storage system of claim 1 , wherein the logic of the host storage manager is further operable to cause the host storage manager to issue one or more maintenance requests to the storage drive, via the host-side interface, the one or more maintenance requests being delimited to at least one of a given one of the namespaces or one of the subdivisions associated with a given one of the namespaces, and wherein the logic of the storage drive is further operable to cause the storage drive to receive the one or more maintenance requests, via the drive-side interface, and to responsively copy write data stored in physical memory which corresponds to the at least one of the given one of the namespaces or one of the subdivisions associated with the given one of the namespaces to a different erase unit of the flash memory.
5 . The storage system of claim 1 , wherein:
the logic of the storage drive is further operable to cause the storage drive to maintain, for each of the subdivisions of the flash memory, second metadata, the second metadata pertaining to one of:
a status of associated physical memory; or
a time since programming of data which is stored in the associated physical memory;
the logic of the host storage manager is further operable to cause the storage manager to issue a second query command, via the host-side interface, in a manner restricted to a given one of the subdivisions of the flash memory; and
the logic of the storage drive is further operable to receive the second query command, via the drive-side interface, and to cause the storage drive to responsively transmit, to the host storage manager, and via the drive-side interface, information representing at least some of the second metadata which is respective to the given one of the subdivisions.
6 . The storage system of claim 5 , wherein the second metadata represents a number of times associated physical memory has been physically erased.
7 . The storage system of claim 5 , wherein the second metadata comprises information representing release status of write data which is stored in the associated physical memory.
8 . The storage system of claim 1 , wherein the logic of the storage drive is further operable to cause the storage drive to:
identify one of the subdivisions as satisfying a storage drive-monitored condition; and
responsively transmit, to the host storage manager, via the drive-side interface, information representing the one of the subdivisions which was identified as satisfying the storage drive-monitored condition.
9 . The storage system of claim 1 , wherein the logic of the storage drive is further operable to cause the storage drive to, in association with each of the write commands:
receive a respective, host-provided address;
derive, from the host-provided address, a first address portion, a second address portion and a third address portion;
identify the selected namespace from the first address portion;
identify the selected subdivision from the second address portion; and
identify a logical block address (LBA) from the third address portion.
10 . The storage system of claim 1 , wherein:
the logic of the storage drive is further operable to cause the storage drive to generate and maintain, for each of the subdivisions of the flash memory that has received write data, at least one value representing a time;
the logic of the host storage manager is further operable to cause the host storage manager to transmit, via the host-side interface, a second query; and
the logic of the storage drive is further operable to cause the storage drive to receive the second query, via the drive-side interface, to process the second query, dependent on the generated and maintained values for of the subdivisions of flash memory that have received write data, and to responsively transmit to the host storage manager, via the drive-side interface, information which is specific to a given one of the subdivisions of the flash memory.
11 . The storage drive of claim 10 , wherein the time is associated with storage, by the storage drive, of associated write data into the flash memory.
12 . The storage system of claim 1 , wherein the logic of the storage drive is further operable to cause the storage drive to store, for each given one of the subdivisions of the flash memory which has received write data, second metadata, the second metadata representing an extent to which additional write data can be programmed, into the one or more erase units which are associated with the given one of the subdivisions, without first erasing the one or more erase units which are associated with the given one of the subdivisions.
13 . The storage system of claim 12 , wherein the logic of the storage drive is further operable to cause the storage drive to transmit information representing the second metadata to the storage manager, via the drive-side interface.
14 . The storage system of claim 1 , wherein the logic of the storage drive is further operable to cause the storage drive to transmit, to the storage manager, in response to the query command, and via the drive-side interface, a size of each subdivision associated with the one or more namespaces.
15 . The storage system of claim 1 , wherein the flash memory of the storage drive comprises planes, and wherein at least one of the subdivisions of the flash memory is associated with physical erase units on different ones of the planes.
16 . The storage system of claim 15 , wherein, within the storage drive, the erase units on different ones of the planes are also on at least two different dies.
17 . The storage system of claim 1 , wherein the logic of the host storage manager is further operable to cause the host storage manager to issue, via the host-side interface, at least one maintenance command to the storage drive, in a manner delimited to a given one of the subdivisions of the flash memory, and wherein the logic of the storage drive is further operable to cause the storage drive to, responsive to receipt of the at least one maintenance command, dynamically control physical erasure of the one or more erase units associated with the given one of the subdivisions.
18 . The storage system of claim 1 , wherein each erase unit associated with one of the subdivisions of the flash memory corresponds to a physical erase unit of the flash memory, wherein the logic of the storage drive is further operable to cause the storage drive to detect a failure condition of one of the physical erase units and to responsively remap the corresponding one of the subdivisions to a different physical erase unit.
19 . The storage system of claim 1 , wherein the context metadata corresponds to a file reference.
20 . The storage system of claim 1 , wherein:
within the storage drive, at least one die of flash memory comprises multiple planes;
a given one of the subdivisions is associated with at least one physical erase unit from each of the multiple planes;
the logic of the storage drive is operable to cause the storage drive to identify, for each one of a set of the write commands, a commonly addressed page, wherein each of the write commands in the set is directed to the given one of the subdivisions and wherein the write commands in the set have sequential addressing; and
the logic of the storage drive is operable to cause the storage drive to program the write data associated with the incoming write commands in the set concurrently into respective ones of the multiple planes, using the identified commonly addressed page.
21 . The storage system of claim 1 , wherein the logic of the host storage manager is operable to cause the host storage manager to transmit, via the host-side interface, a second query to the storage drive, and wherein the logic of the storage drive is operable to cause the storage drive to receive the second query, and to responsively transmit information to the host storage manager indicating at least one of a number of dies of the flash memory or a number of channels of the flash memory.
22 . The storage system of claim 21 , wherein:
the flash memory further supports two block devices;
the logic of the host storage manager is further operable to transmit a command to the storage drive, via the host-side interface, to program the storage drive to associate each of the two block devices with a selected number of the at least one of a number of dies of the flash memory or a number of channels of the flash memory, in a manner such that the selected number is free to be different for each of the two block devices; and
for each of the respective one of the write commands, the logic of the storage drive is operable to identify a selected one of the two block devices and to store the respective write data in a flash memory die on a flash memory channel that is uniquely associated with the selected one of the two block devices.
23 . The storage system of claim 1 , wherein:
the logic of the host storage manager is further operable to cause the host storage manager to transmit, to the storage drive, via the host-side interface, a second query;
the logic of the storage drive is further operable to cause the storage drive to:
track second metadata, the second metadata comprising physical storage location release information;
receive the second query, via the drive-side interface; and
responsive to the second query, transmit information, to the host storage manager, via the drive-side interface, which represents the storage location release information; and
the logic of the host storage manager is further operable to cause the host storage manager to issue, via the host-side interface, one or more maintenance requests to the storage drive, wherein the one or more maintenance requests are dependent on the information representing the storage location release information transmitted to the host storage manager which represents the storage location release information.
24 . The storage system of claim 1 , wherein the logic of the host storage manager is further operable to cause the host storage manager to issue, via the host-side interface, a storage location release request which specifies at least one memory location, and wherein the logic of the storage drive is further operable to cause the storage drive to receive the release request from the host storage manager, via the drive-side interface, and to responsively mark each specified memory location as released.
25 . The storage system of claim 1 , wherein:
the logic of the host storage manager is further operable to cause the host storage manager to transmit, to the storage drive, via the host-side interface, a second query;
the logic of the storage drive is further operable to cause the storage drive to:
track second metadata, the second metadata comprising hot-cold information;
receive the second query, via the drive-side interface; and
responsive to the second query, transmit information, to the host storage manager, via the drive-side interface, which is dependent on the hot/cold information; and
the logic of the host storage manager is further operable to cause the host storage manager to issue, via the host-side interface, one or more maintenance requests to the storage drive, wherein the one or more maintenance requests are dependent on the information which is dependent on the hot/cold information which was transmitted to the host storage manager.
26 . The storage system of claim 1 wherein, within the storage drive, each particular one of the subdivisions corresponds to an integer number of physical erase units and a sequential range of logical block addresses (LBAs) mapped to the integer number of physical erase units, wherein each of a number of read commands directed to the particular one of the subdivisions is accompanied by an LBA address value, and wherein the logic of the storage drive is further operable to access data within the integer number of physical erase units according to the LBA address value.
27 . The storage system of claim 1 , wherein:
the logic of the host storage manager is further operable to cause the host storage manager to transmit, to the storage drive, via the host-side interface, a second query;
the logic of the storage drive is further operable to cause the storage drive to:
track second metadata, the second metadata comprising physical storage location release information;
identify one or more subdivisions that are garbage collection candidates dependent on the second metadata;
receive the second query, via the drive-side interface; and
responsive to the second query, transmit information, to the host storage manager, via the drive-side interface, which represents the identified one or more subdivisions; and
the logic of the host storage manager is further operable to cause the host storage manager to issue, via the host-side interface, one or more maintenance requests to the storage drive, wherein the one or more maintenance requests are dependent on the information representing the identified one or more subdivisions transmitted to the host storage manager.