Storage system with division based addressing and cooperative flash management
This disclosure provides techniques hierarchical address virtualization within a memory controller and configurable block device allocation. By performing address translation only at select hierarchical levels, a memory controller can be designed to have predictable I/O latency, with brief or otherwise negligible logical-to-physical address translation time. In one embodiment, address transition may be implemented entirely with logical gates and look-up tables of a memory controller integrated circuit, without requiring processor cycles. The disclosed virtualization scheme also provides for flexibility in customizing the configuration of virtual storage devices, to present nearly any desired configuration to a host or client.
1. A storage apparatus comprising:
at least one host processor; and
one or more storage drives, each storage drive comprising flash memory having segments, each of the segments corresponding to one or more erase units, and each storage drive comprising circuitry to receive incoming read addresses in association with read requests received from the at least one host processor, wherein said circuitry, for each one of the incoming read addresses which is addressed to an associated storage drive, is to
identify from the one of the incoming read addresses an addressed block device within the associated storage drive and an addressed segment within the addressed block device,
perform a division operation to identify from the one of the incoming addresses a first bit field and a second bit field, and
identify one of the erase units which corresponds to the addressed segment according to a value of the first bit field and an addressed storage location in the identified erase unit according to a value of the second bit field;
wherein each of the one or more storage drives is to store metadata respective to its erase units;
wherein each given drive of the one or more storage drives of the storage apparatus is to
identify logical address information that identifies a specific one of the segments of the given drive, by identifying a physical address corresponding to one of the erase units of the given drive for which respective metadata is stored and by performing, on the identified physical address, an inverse of the division operation performed by the given drive, and
transmit to the at least one host processor the logical address information and additional information dependent on the stored metadata of the one of the erase units corresponding to the logical address information, dependent on the respective metadata; and
wherein, dependent on receipt of the logical address information, the at least one host processor is to issue one or more requests to responsively copy data from a first physical storage location in the flash memory to a second physical storage location in the flash memory and is to issue an address-delimited erase request to perform a targeted erasure of one or more erase units corresponding to the logical address information.
2. The storage apparatus of claim 1 wherein the storage apparatus further comprises instructions stored on at least one physical non-transitory storage medium, said instructions when executed to control at least one host processor so as to operate as special purpose circuitry.
3. The storage apparatus of claim 2 wherein each of the one or more storage drives comprises associated nonvolatile memory and is to store its metadata in the associated nonvolatile memory.
4. The storage apparatus of claim 1 wherein for each drive of the one or more storage drives, the metadata comprises information dependent on at least one of a number of times that the respective erase unit has been erased or existence of read error detected for the respective erase unit.
5. The storage apparatus of claim 1 wherein for each drive of the one or more storage drives, the metadata comprises information dependent on operability of the respective erase unit.
6. The storage apparatus of claim 1 wherein for each drive of the one or more storage drives, the metadata comprises information dependent on the time since data was programmed into the respective erase unit.
7. The storage apparatus of claim 1 wherein:
each drive of the one or more storage drives comprises circuitry to identify a maintenance condition dependent on a comparison of the metadata stored by that storage drive with at least one threshold; and
the transmission of the logical address information that identifies a specific one of the segments corresponding to the one of the physical erase units is dependent on the identified maintenance condition.
8. The storage apparatus of claim 1 wherein:
the given drive is to receive a query from the at least one host processor;
responsive to said query, the given drive is to compare the metadata stored by the given drive, as updated, with at least one threshold, to detect a wear condition; and
the transmission of the logical address information is responsive to the detection of the wear condition.
9. The storage apparatus of claim 1 wherein:
the given drive is to receive, from the at least one host processor, a query accompanied by address information, from which the logical address information is identified; and
responsive to said query, the given drive is to derive from the respective metadata stored by the given drive, as updated, the additional information; and
the additional information is dependent on an erase count of the one of the erase units corresponding to the logical address.
10. The storage apparatus of claim 1 wherein:
the given drive is to receive, from the at least one host processor, a query accompanied by address information, from which the logical address information is identified; and
responsive to said query, the given drive is to derive from the respective metadata stored by the given drive, as updated, the additional information; and
the additional information is dependent on a time since the programming of the one of the erase units corresponding to the logical address.
11. The storage apparatus of claim 1 wherein:
the one or more requests to responsively copy comprise a first read request and a first write request; and
the at least one host processor is to
issue the first read request to read the data from the first physical location in the flash memory, in a manner dependent on receipt of the logical address information, and
subsequently issue the first write request to write the data into the second physical storage location in the flash memory.
12. The storage apparatus of claim 1 wherein:
the one or more requests to responsively copy comprise a first read request and a first write request;
the first physical storage location and the second physical storage location are each in the given storage drive; and
the at least one host processor is to
issue the first read request to the given storage drive, to read the data from the first physical location in the flash memory dependent on receipt of the logical address information from the given storage drive, and
subsequently issue a write request to the given storage drive to write the data into the second physical storage location in the flash memory within the given storage drive.
13. The storage apparatus of claim 1 wherein:
the one or more requests to responsively copy comprise a first read request and a first write request;
the one or more storage drives comprise the given storage drive and a second storage drive;
the second physical storage location is in flash memory of the second storage drive; and
the at least one host processor is to
issue the first read request to the given storage drive, to read the data from the first physical location in the flash memory dependent on receipt of the logical address information from the given storage drive, and
subsequently issue the first write request to the second storage drive to write the data into the second physical storage location.
14. The storage apparatus of claim 1 wherein the one or more requests to responsively copy are to cause the one or more storage drives to read the data from the first physical location in the flash memory dependent on receipt of the logical address information and to write the data into the second physical storage location in the flash memory without sending the data to the at least one host processor.
15. The storage apparatus of claim 1 wherein:
the one or more storage drives comprise the given storage drive and a second storage drive; and
the one or more requests to responsively copy are to cause the given storage drive to
read the data from the first physical location in flash memory of the given storage drive and
directly send the data to the second storage drive, without sending the data to the at least one host processor, in a manner so as to cause the second storage drive to write the data into flash memory of the second storage drive, the second physical storage location in the flash memory being in the flash memory of the second storage drive.
16. The storage apparatus of claim 1 wherein:
the flash memory of the given drive comprises multiple planes; and
the addressed storage drive is to, in association with the execution of the address-delimited erase request, concurrently erase a physical erase unit in each of two or more of the multiple planes.
17. The storage apparatus of claim 1 wherein:
the one or more storage drives are each to receive from the at least one host processor a request to erase; and
each request to erase is to address a block device within an addressed storage drive and a segment within the addressed block device of the addressed storage drive.
18. The storage apparatus of claim 1 wherein each item of metadata represents a status of corresponding physical memory space, and wherein each drive of the one or more storage drives further comprise circuitry to:
receive a query issued by the at least one host processor, wherein the query specifies an addressed block device;
identify from the query a specific segment within the addressed block device within that drive depending on the status of associated physical memory space;
identify a segment address within the block device which corresponds to the specific segment; and
responsively transmit to the at least one host processor logical address information for the identified segment address.
19. The storage apparatus of claim 1 wherein:
each one of the one or more storage drives is to store information identifying a number of flash memory structures in that one of the one or more storage drives and is to transmit to the at least one host processor the information identifying the number of flash memory structures in that one of the one or more storage drives;
the at least one host processor is to issue data access requests to the one or more storage drives which each are dependent, for an addressed drive of the one or more storage drives, on the information transmitted to the at least one host processor identifying the number of flash memory structures for the addressed drive;
each addressed drive comprises circuitry to fulfill a corresponding one of the data access requests by issuing commands to the flash memory of the addressed drive for which a quantum of flash memory selected by the commands is dependent on the information transmitted to the at least one host processor identifying the number of flash memory structures for the addressed drive; and
for each one of the one or more storage drives, the information transmitted to the at least one host processor by that one of the one or more storage drives identifying the number of flash memory structures is to identify an amount of flash memory in that one of the one or more storage drives that is to be erased in a common operation.
20. The storage apparatus of claim 1 wherein:
each drive of the one or more storage drives comprises physical erase units; and
each drive of the one or more storage drives comprises circuitry to
detect a failure condition of one of the physical erase units of that drive, and
transmit information to the at least one host processor that identifies a segment corresponding to the one of the physical erase units for which the failure condition was detected.
21. The storage apparatus of claim 1 wherein:
the flash memory comprises multiple planes; and
each of the one or more storage drives is to, for each one of a set of the incoming read requests having sequential addressing,
identify a commonly addressed page, and
concurrently retrieve read data corresponding to the commonly address page from respective ones of the multiple planes.
22. The storage apparatus of claim 1 wherein each drive of the one or more storage drives comprises circuitry to, for an incoming write request addressed to that drive, accompanied by an incoming write address:
identify a block device addressed by the incoming write request and a segment addressed by the incoming write request within the block device addressed by the incoming write request;
perform a division operation to identify a first write address bit field and second write address bit field;
identify one of the erase units of that drive which corresponds to the segment addressed by the incoming write request according to a value of the first write address bit field and a write storage location in the one of the erase units identified from the first write address bit field according to a value of the second write address bit field; and
program associated data into the identified write storage location.
23. The storage apparatus of claim 22 wherein:
each drive of the one or more storage drives comprises circuitry to detect a failure condition of one of the erase units of that drive; and
each drive of the one or more storage drives is further to, for an incoming write request addressed to that drive which is directed to a segment corresponding to an erase unit for which a failure condition has been detected, transmit to the at least one host processor a notification of a write failure and logical address information identifying the segment corresponding to the one of the erase units for which the failure condition has been detected.
24. The storage apparatus of claim 22 wherein:
each of the one or more storage drives is to track a time since data was programmed into the respective erase unit and is to transmit to the at least one host processor, responsive to a determination that the time since data was programmed for a given erase unit meets a predetermined condition, logical address information corresponding to the given erase unit.
25. The storage apparatus of claim 22 wherein:
for the incoming write request, the circuitry to identify the write storage location is to generate an address value corresponding to the identified write storage location and is to transmit the address value to the at least one host processor; and
for each one of read requests which is directed to the associated data, the associated incoming read address is dependent on the address value which was transmitted to the at least one host processor.
26. The storage apparatus of claim 1 wherein each drive of the one or more storage drives is to:
track additional metadata respective to the segments of that drive, where said additional metadata indicates an extent to which associated memory can continue to receive additional write data without being erased; and
transmit information to the at least one host processor dependent on said additional metadata.
27. The storage apparatus of claim 1 wherein each drive of the one or more storage drives comprises circuitry to receive incoming write requests and associated write addresses, and for each associated write address, to:
identify from the associated write addresses, a block device within the drive which is addressed by the associated write address and a segment within the block device addressed by the associated write address;
identify an address offset into the segment identified from the associated write address;
perform a division operation to identify from the address offset a first write address bit field and second write address bit field;
identify one of the erase units which corresponds to the segment addressed by the associated write address according to a value of the first write address bit field and a write storage location in the one of the erase units which was identified from the first write address bit field, according to a value of the second write address bit field;
program associated data into the identified one of the erase units which was identified from the first write address bit field, at the write storage location; and
transmit address information to the at least one host processor dependent on the segment addressed by the associated write address and the identified address offset.
28. The storage apparatus of claim 27 wherein the segment identified from the associated write address corresponds to an integer number of physical erase units for which a failure condition has not been detected, and wherein the address offset corresponds to a location within a sequential range of locations that correspond to the integer number of physical erase units.
29. The storage apparatus of claim 27 wherein each drive of the one or more storage drives is to:
track additional metadata which indicates an extent to which memory comprising at least one physical erase unit corresponding to a given segment can continue to receive additional write data without being erased; and
for a given one of the incoming write requests which is addressed to that drive and for which the associated write address designates the given segment in that drive, update the corresponding additional metadata.
30. A storage apparatus comprising:
at least one host processor; and
one or more storage drives, each of the one or more storage drives comprising flash memory having block devices at a hierarchically-superior level, segments within each of the block devices at a hierarchically-intermediate level, each segment corresponding to a mutually exclusive set of one or more physical erase units, each segment being associated with only one of the block devices, each physical erase unit being associated with only one of the segments, and storage locations within each physical erase unit at a hierarchically-inferior level;
wherein each storage drive comprises circuitry to receive incoming read addresses in association with read requests received from the at least one host processor, wherein said circuitry, for each one of the incoming read addresses which is addressed to an associated storage drive, is to
identify from the one of the incoming read addresses an addressed one of the block devices within the associated storage drive and an addressed one of segments which is within the addressed one of the block devices,
perform a division operation to identify from the one of the incoming addresses a first bit field and a second bit field, and
identify one of the physical erase units which corresponds to the addressed one of the segments according to a value of the first bit field and an addressed storage location in the identified physical erase unit according to a value of the second bit field;
wherein each of the one or more storage drives is to store metadata respective to its physical erase units;
wherein each given drive of the one or more storage drives of the storage apparatus is to
identify logical address information that identifies a specific one of the segments of the given drive, by identifying a physical address corresponding to one of the physical erase units of the given drive for which respective metadata is stored and by performing, on the identified physical address, an inverse of the division operation performed by the given drive, and
transmit to the at least one host processor the logical address information and additional information dependent on the stored metadata of the one of the physical erase units corresponding to the logical address information, dependent on the respective metadata; and
wherein, dependent on receipt of the logical address information, the at least one host processor is to issue one or more requests to responsively copy data from a first physical storage location in the flash memory to a second physical storage location in the flash memory and is to issue an address-delimited erase request to perform a targeted erasure of one or more physical erase units corresponding to the logical address information.