Write congestion aware bypass for non-volatile memory, last level cache (LLC) dropping from write queue responsive to write queue being full and read queue threshold wherein the threshold is derived from latency of write to LLC and main memory retrieval time
A processor includes a processing core and a cache controller including a read queue and a separate write queue. The read queue is to buffer read requests of the processing core to a non-volatile memory, last level cache (NVM-LLC), and the write queue is to buffer write requests to the NVM-LLC. The cache controller is to detect whether the write queue is full. The cache controller further prioritizes a first order of sending requests to the NVM-LLC when the write queue contains an empty slot, the first order specifying a first pattern of sending the read requests before the write requests, and prioritizes a second order of sending requests to the NVM-LLC in response to a determination that the write queue is full, the second order specifying a second pattern of alternating between sending a write request from the write queue and a read request from the read queue.
1. A processor comprising:
a processing core;
a read queue to buffer read requests of the processing core to a non-volatile memory, last level cache (NVM-LLC);
a write queue to buffer write requests of the processing core to the NVM-LLC; and
a cache controller coupled to the processing core, and to include the read queue and the write queue, the cache controller further to:
buffer a liveness score with each write request in the write queue;
detect write congestion comprising the write queue being full and the read queue having over a threshold number of read requests, the threshold number being derived from a combination of write latency to the NVM-LLC and a turn-around time of fetching a cache line from main memory; and
drop, from the write queue, one of the write requests having a lowest liveness score in response to detection of the write congestion.
2. The processor of claim 1 , wherein the cache controller is further to prioritize an order of sending requests to the NVM-LLC in response to a determination that the write queue is full, the order specifying a pattern of alternating between sending a write request from the write queue and a read request from the read queue.
3. The processor of claim 2 , wherein the cache controller is further to:
determine that a most-recent request sent to the NVM-LLC was a read request; and
send, to the NVM-LLC, a subsequent request from the write queue in response to a determination that the write queue is full.
4. The processor of claim 2 , wherein the cache controller is further to:
determine a most-recent request sent to the NVM-LLC was a write request; and
send, to the NVM-LLC, a subsequent request from the read queue in response to a determination that the write queue is full.
5. The processor of claim 1 , wherein the cache controller is further to:
determine whether a cache line of the one of the write requests is dirty; and
write the cache line to memory when the one of the write requests is dirty.
6. An integrated circuit comprising:
a cache controller comprising:
a read queue to buffer read requests to a non-volatile memory, last level cache (NVM-LLC); and
a write queue to buffer write requests to the NVM-LLC; and
wherein the cache controller is to:
buffer, in the write queue, a liveness score with each respective write request;
detect write congestion comprising the write queue being full and the read queue having over a threshold number of read requests, the threshold number being derived from a combination of write latency to the NVM-LLC and a turn-around time of fetching a cache line from main memory; and
drop, from the write queue, a write request having a lowest liveness score in response to detection of the write congestion.
7. The integrated circuit of claim 6 , wherein the cache controller is further to:
determine whether a cache line of the write request is dirty; and
write the cache line to memory when the write request is dirty.
8. The integrated circuit of claim 6 , wherein the cache controller is further to:
detect, after the drop of the write request, an empty slot in the write queue; and
prioritize an order of sending requests to the NVM-LLC, the order specifying a pattern of sending the read requests before the write requests.
9. The integrated circuit of claim 6 , wherein the cache controller is further to prioritize an order of sending requests to the NVM-LLC in response to a determination that the write queue is full, the order specifying a pattern of alternating between sending a write request from the write queue and a read request from the read queue.
10. The integrated circuit of claim 9 , wherein the cache controller is further to:
determine a most-recent request sent to the NVM-LLC was a read request; and
send, to the NVM-LLC, a subsequent request from the write queue in response to the determination that the write queue is full.
11. The integrated circuit of claim 9 , wherein the cache controller is further to:
determine a most-recent request sent to the NVM-LLC was a write request; and
send, to the NVM-LLC, a subsequent request from the read queue in response to the determination that the write queue is full.
12. The integrated circuit of claim 6 , wherein the liveness score comprises a probability that a cache line will be recalled.
13. A method comprising:
buffering, by a cache controller, read requests to a non-volatile memory, last level cache (NVM-LLC) in a read queue;
buffering, by the cache controller, write requests to the NVM-LLC in a write queue;
buffer, by the cache controller, a liveness score with each write request in the write queue;
detecting, by the cache controller, write congestion comprising the write queue being full and the read queue having over a threshold number of read requests, the threshold number being derived from a combination of write latency to the NVM-LLC and a turn-around time of fetching a cache line from main memory; and
dropping, from the write queue by the cache controller, one of the write requests having a lowest liveness score in response to detection of the write congestion.
14. The method of claim 13 , further comprising:
prioritizing, by the cache controller, sending the read requests, before the write requests, to the NVM-LLC; and
responsive to detecting the write queue being full, alternately sending, to the NVM-LLC, a write request from the write queue and a read request from the read queue.
15. The method of claim 14 , further comprising:
determining a most-recent request sent to the NVM-LLC was a read request; and
sending, to the NVM-LLC, a subsequent request from the write queue in response to detecting that the write queue is full.
16. The method of claim 14 , further comprising:
determining a most-recent request sent to the NVM-LLC was a write request; and
sending next, to the LLC, a subsequent request from the read queue in response to detecting that the write queue is full.
17. The method of claim 14 , further comprising:
determining whether a cache line of the one of the write requests is dirty; and
writing the cache line to memory when the one of the write requests is dirty.
18. A method comprising:
buffering, by a cache controller, read requests to a non-volatile memory, last level cache (NVM-LLC) in a read queue;
buffering, by the cache controller, write requests to the NVM-LLC in a write queue with a corresponding liveness score;
detecting, by the cache controller, write congestion comprising the write queue being full and the read queue having over a threshold number of read requests, the threshold number being derived from a combination of write latency to the NVM-LLC and a turn-around time of fetching a cache line from main memory; and
dropping, by the cache controller from the write queue, a write request having a lowest liveness score in response to detection of the write congestion.
19. The method of claim 18 , further comprising:
detecting, after the dropping of the write request, an empty slot in the write queue; and
prioritizing an order of sending requests to the NVM-LLC, the order specifying a pattern of sending the read requests before the write requests.
20. The method of claim 18 , further comprising prioritizing an order of sending requests to the NVM-LLC in response to a determination that the write queue is full, the order specifying a pattern of alternating between sending a write request from the write queue and a read request from the read queue.