Speculative memory accesses in a proximity communication-based off-chip cache memory architecture
View Patent ↗A proximity interconnect module includes a plurality of processors operatively connected to a plurality of off-chip cache memories by proximity communication. Due to the high bandwidth capability of proximity interconnect, when an off-chip cache memory is searched for requested data, either the requested data is at the same time searched for in on-chip cache memories of the proximity interconnect module or the requested data is at the same time retrieved from main memory. This reduces latency by reducing serial operations.
1. A system, comprising:
a first processor having a first on-chip cache memory;
a second processor having a second on-chip cache memory, wherein the second processor is operatively connected to the first processor and a third processor by proximity communication, the third processor having a third on-chip cache memory, wherein the third processor is operatively connected to the first processor and the second processor by proximity communication; and
a first off-chip cache memory operatively connected to the first processor and the second processor by proximity communication,
wherein, in response to a cache miss for requested data in the first on-chip cache memory, the system is configured to search for the requested data in the first off-chip cache memory in parallel with searching for the requested data in the second on-chip cache memory and the third on-chip cache memory.
2. The system of claim 1 , wherein, in response to finding the requested data in the first off-chip cache memory, the system is configured to return the requested data from the first off-chip cache memory.
3. The system of claim 1 , wherein, in response to finding the requested data in the second on-chip cache memory and not finding the requested data in the first off-chip cache memory, the system is configured to return the requested data from the second on-chip cache memory.
4. The system of claim 1 , wherein the proximity communication is performed using at least one of capacitive coupling, inductive coupling, magnetic coupling, and optical coupling.
5. The system of claim 1 , wherein the requested data is searched for in the second on-chip cache memory and the third on-chip cache memory based on a snooping-based cache-coherence protocol.
6. The system of claim 1 , further comprising:
a second off-chip cache memory operatively connected to the second processor and the first off-chip cache memory by proximity communication; and
a third off-chip cache memory operatively connected to the third processor and the first off-chip cache memory by proximity communication,
wherein the system is further configured to search for the requested data in the second on-chip cache memory by searching the second off-chip cache memory, and to search for the requested data in the third on-chip cache memory by searching the third off-chip cache memory.
7. A method of performing computer system operations, comprising:
searching for requested data in a first on-chip cache memory of a first processor; and
when the requested data is not found in the first on-chip cache memory, searching for the requested data in a first off-chip cache memory in parallel with searching for the requested data in a second on-chip cache memory and a third on-chip cache memory of a third processor,
wherein the first processor, the second processor, and the third processor are operatively connected by proximity communication, and the first off-chip cache memory are is operatively connected to the first processor and the second processor by proximity communication.
8. The method of claim 7 , further comprising:
when the requested data is found in the second on-chip cache memory, returning the requested data from the second on-chip cache memory.
9. The method of claim 7 , further comprising:
when the requested data is found in the first off-chip cache memory, returning the requested data from the first off-chip cache memory.
10. The method of claim 7 , further comprising:
when the requested data is not found in the first off-chip cache memory and the second on-chip cache memory but is found in the third on-chip cache memory, returning the requested data from the third on-chip cache memory.
11. The method of claim 7 , wherein the proximity communication is performed using at least one of capacitive coupling, inductive coupling, magnetic coupling, and optical coupling.
12. The method of claim 7 , wherein searching for the requested data in the second on-chip cache memory and the third on-chip cache memory further comprises:
searching a second off-chip cache memory operatively connected to the second processor and the first off-chip cache memory by proximity communication; and
searching a third off-chip cache memory operatively connected to the third processor and the first off-chip cache memory by proximity communication.
13. A system, comprising:
a proximity interconnect module;
a plurality of processors disposed on the proximity interconnect module, the plurality of processors each having an on-chip cache memory; and
a plurality of off-chip cache memories disposed on the proximity interconnect module and operatively connected to the plurality of processors by proximity communication,
wherein, in response to a cache miss for requested data in a first on-chip cache memory of a first processor of the plurality of processors, the system is configured to search for the requested data in a first off-chip cache memory of the plurality of off-chip cache memories in parallel with searching for the requested data in all other on-chip cache memories of the on-chip cache memories.
14. The system of claim 13 , wherein, in response to finding the requested data in the first off-chip cache memory, the system is configured to return the requested data from the first off-chip cache memory.
15. The system of claim 13 , wherein, in response to not finding the requested data in the first off-chip cache memory and finding the requested data in a second on-chip cache memory of the on-chip cache memories, the system is configured to return the requested data from the second on-chip cache memory.
16. The system of claim 13 , wherein the system is configured to search for the requested data in all other on-chip cache memories of the on-chip cache memories using a snooping-based cache-coherence protocol.
17. A system, comprising:
a proximity interconnect module;
a plurality of processors disposed on the proximity interconnect module, the plurality of processors each having an on-chip cache memory;
a plurality of off-chip cache memories disposed on the proximity interconnect module and operatively connected to the plurality of processors by proximity communication; and
a main memory operatively connected to the plurality of processors and the proximity interconnect module,
wherein, in response to a cache miss for requested data in one of the on-chip cache memories, the system is configured to search for the requested data in one off-chip cache memory of the plurality of off-chip cache memories in parallel with searching for the requested data in all of the other on-chip cache memories of the plurality of other processors in the system.
18. The system of claim 17 , wherein, in response to not finding the requested data in the off-chip cache memory, the system is configured to retrieve the requested data from the main memory.