Non-volatile memory with optimized read
A non-volatile storage system that is implementing a storage region (e.g., a persistent memory region) which is accessible to a host (e.g., via a PCIe connection) and a cache for the storage region shares details of the structure of the storage region and/or the cache (e.g., cache segment size). With awareness of the shared details of the structure of the storage region and/or the cache, the host arranges and sends out requests to read data from the persistent memory region in a manner that takes advantage of parallelism within the non-volatile storage system. For example, the host may initially send out one read request per cache segment to cause the non-volatile storage system to load the cache. Subsequently, additional read requests are made to the non-volatile storage system, with the data already loaded (or starting to load) in the cache, thereby increasing performance.
1. A method comprising:
a non-volatile storage system, that is implementing a persistent memory region (“PMR”) and a PMR cache comprising a plurality of cache segments that are each a cache segment size, informing a host connected to the storage system of the cache segment size;
the host determining that a set of data needs to be read from the PMR;
the host using the cache segment size to determine a set of the cache segments that will be used by the storage system for reading the set of data;
the host sending a plurality of read requests including at least one read request for each cache segment of the set of the cache segments that will be used by the storage system for reading the set of data;
in response to the plurality of read requests, the storage system reading groups of data from the PMR, loading the groups of data into the set of the cache segments, and transmitting a unit of data from each cache segment of the set of cache segments to the host such that each group of data is at a cache segment size and the unit of data is smaller than the cache segment size such that multiple units of data fit within one cache segment;
after the host sends the at least one read request for each cache segment of the set of cache segments, the host sending additional read requests for additional data of the set of data; and
the storage system transmitting the additional data to the host in response to the additional read requests by reading the additional data from the set of the cache segments and transmitting the additional data read to the host.
2. The method of claim 1 , further comprising:
the storage system sending first data back to the host in response to one or more of the at least one read request for each cache segment of the set of cache segments, the first data is a unit of data from the PMR cache, the host sends the additional read requests in response to receiving the first data.
3. The method of claim 1 , wherein:
the plurality of cache segments includes a first cache segment and a second cache segment;
the at least one read request for each cache segment of the set of the cache segments that will be used by the storage system for reading the set of data includes a first read request for a first unit of data from the first cache segment and a second read request for a second unit of data from the second cache segment;
the storage system reading groups of data from the PMR and loading the groups of data into the set of the cache segments and transmitting a unit of data from each cache segment of the set of cache segments comprises the storage system reading a first group of data from the PMR, loading the first group of data into the first cache segment, transmitting the first unit of data from the first cache segment to the host, reading a second group of data from the PMR, loading the second group of data into the second cache segment, and transmitting the second unit of data from the second cache segment to the host;
the first group of data includes the first unit of data, the second group of data includes the second unit of data, the first group of data and the second group of data are at the cache segment size, the unit of data is smaller than the cache segment size such that multiple units of data fit within one cache segment; and
the additional read requests include additional read requests for additional data from the first cache segment and additional read requests for additional data from the second cache segment, the host sends the additional read requests for additional data from the first cache segment in response to the storage system transmitting the first unit of data from the first cache segment.
4. The method of claim 3 , wherein:
the storage system reading the first group of data from the PMR and loading the first group of data into the first cache segment is performed concurrently with the storage system reading the second group of data from the PMR and loading the second group of data into the second cache segment.
5. The method of claim 1 , further comprising:
the host waiting a predetermined period of time after the host sends at least one read request for each cache segment of the set of the cache segments, the host sends the additional read requests after waiting the predetermined period of time.
6. The method of claim 1 , wherein the storage system reading groups of data from the PMR, loading the groups of data into the set of the cache segments, transmitting a unit of data from each cache segment and transmitting the additional data read to the host comprises:
the storage system receiving a TLP from the host;
the storage system determining whether a unit of data requested by the TLP is in the PMR cache;
transmitting the unit of data to the host in response to determining that the unit of data is in the PMR cache;
determining if the TLP is a first read request for a PMR cache segment that corresponds to an address in the TLP in response to determining that the unit of data is not in the PMR cache; and
in response to determining that the TLP is a first read request for the PMR cache segment that corresponds to the address in the TLP, reading PMR data from the PMR, loading the PMR data into the PMR cache and transmitting the unit of data to the host from the PMR cache segment that corresponds to the address in the TLP.
7. The method of claim 1 , wherein:
the storage system comprises a memory controller connected to volatile memory and non-volatile memory, the PMR resides in the non-volatile memory, the PMR cache resides in the volatile memory, the non-volatile memory comprises multiple memory dies;
the plurality of cache segments includes a first cache segment and a second cache segment;
the at least one read request for each cache segment of the set of the cache segments includes a first read request for a first unit of data and a second read request for a second unit of data; and
the storage system reading groups of data from the PMR and loading the groups of data into the set of the cache segments includes the memory controller reading a page of data from each of the multiple memory dies, aggregating the pages of data to form a cache segment of data and storing the cache segment of data in the first cache segment.
8. The method of claim 1 , wherein:
the host is connected to the storage system by a Peripheral Component Interconnect Express (“PCIe”) connection; and
each read request of the at least one read request for each cache segment of the set of the cache segments is a PCIe Transaction Layer Packet (“TLP”) that requests a unit of data.
9. A non-transitory processor readable storage medium storing processor readable code that when executed on a processor causes the processor to perform a method comprising:
accessing an indication of a cache segment size for a non-volatile storage system implementing a storage region and a cache for the storage region, the cache comprises a plurality of cache segments that are each sized at the cache segment size;
based on the indication of the cache segment size, determining a set of cache segments of the plurality of cache segments that will be used by the storage system for reading a set of data; and
sending read requests for the host data to the non-volatile storage system including sending an initial read request for each cache segment of the set of cache segments, waiting until receiving one or more completion responses for one or more of the initial read requests after sending the initial read request for each cache segment of the set of cache segments and sending additional read requests for data that is already stored in the set of cache segments in response to receiving the one or more completion responses, each of the read requests is for a unit of data, the unit of data is smaller than the cache segment size such that multiple units of data fit within one cache segment.
10. The non-transitory processor readable storage medium of claim 9 , wherein:
the storage region is a persistent memory region (“PMR”); and
the cache is a PMR cache.
11. The non-transitory processor readable storage medium of claim 9 , wherein:
the plurality of cache segments includes a first cache segment and a second cache segment;
the initial read request for each cache segment includes a first read request for a first unit of data from the first cache segment and a second read request for a second unit of data from the second cache segment;
the method further comprises receiving the first unit of data from the non-volatile storage system; and
the additional read requests include additional read requests for additional data from the first cache segment and additional read requests for additional data from the second cache segment, the additional read requests for additional data from the first cache segment are sent to the non-volatile storage system in response to the receiving the first unit of data from the non-volatile storage system.
12. The non-transitory processor readable storage medium of claim 9 , wherein:
the plurality of cache segments includes a first cache segment and a second cache segment;
the initial read request for each cache segment includes a first read request for a first unit of data from the first cache segment and a second read request for a second unit of data from the second cache segment; and
the additional read requests include additional read requests for additional data from the first cache segment and additional read requests for additional data from the second cache segment.
13. The non-transitory processor readable storage medium of claim 9 , wherein:
the set of data is stored in the storage region at a set of addresses;
the determining the set of cache segments of the plurality of cache segments that will be used by the storage system for reading the set of data comprises determining which subset of the plurality of cache segments corresponds to the set of addresses.
14. The non-transitory processor readable storage medium of claim 9 , wherein:
the accessing the indication of the cache segment size includes receiving the indication of the cache segment size from the non-volatile storage system.
15. The non-transitory processor readable storage medium of claim 9 , wherein:
the storage region is a persistent memory region (“PMR”);
the cache is a PMR cache;
the accessing, determining, sending the initial read request for each cache segment and the sending additional read requests are performed by a host computer connected to the non-volatile storage system by a Peripheral Component Interconnect Express (“PCIe”) connection; and
each read request of the initial read request for each cache segment is a PCIe Transaction Layer Packet (“TLP”).
16. An apparatus comprising:
non-volatile memory configured to implement a persistent memory region in the non-volatile memory that is accessible by a host;
a persistent memory region cache comprising a plurality of cache segments that are each a cache segment size, the set of cache segments includes a first cache segment, a second cache segment and a third cache segment; and
a processor connected to the non-volatile memory and the persistent memory region cache, the processor is configured to communicate with a host, the processor is configured to transmit the cache segment size to the host, the processor is further configured to:
receive an initial set of read requests from the host including one read request for each cache segment of a set of cache segments of the plurality of cache segments, each read request is for a request unit of data, the request unit of data is smaller than the cache segment size such that multiple request units of data fit within on cache segment, the receiving the initial set of read requests comprises receiving one read request for the first cache segment as well as receiving one read request for the second cache segment and receiving one read request for the third cache segment,
read data from the persistent memory region for each read request of the initial set of read requests,
store the data read into the cache segments of the set of cache segments,
send a completion response with requested data for each of the read requests of the initial set of read requests,
after receiving the initial set of read requests, receive additional read requests for data that is already stored in the set of cache segments in response to the initial set of read requests, the receiving additional read requests for data that is already stored in the set of cache segments comprises receiving a plurality of read requests for the first cache segment followed by receiving a plurality of read requests for the second cache segment followed by receiving a plurality of read requests for the third cache segment, and
send a completion response with requested data for each of the additional read requests such that the requested data that is sent was obtained from one or more of cache segments of the set of cache segments.
17. The apparatus of claim 16 , further comprising:
a volatile memory, the persistent memory region cache is resident in the volatile memory, the processor comprises a memory controller, the memory controller is connected to the volatile memory, the request unit of data is smaller than the cache segment size such that multiple request units of data fit within on cache segment.
18. The apparatus of claim 16 , further comprising:
a computer that is external to the non-volatile memory and the processor, the computer implements the host, the processor includes a host interface for communicating with the host, the computer is connected to the host interface, the host is configured to:
receive the cache segment size from the processor;
based on the cache segment size and addresses for a set of host data, identify the set of cache segments of the plurality of cache segments because they will be used by the processor for reading the set of host data, the set of host data comprises the requested data for each of the read requests of the initial set of read requests and the requested data for the for each of the additional read requests,
send the initial set of read requests;
wait until receiving one or more completion responses for one or more of the initial set of read requests; and
in response to receiving the one or more completion responses, send the additional read requests for data that is already stored in the set of cache segments.