High bandwidth memory system with distributed request broadcasting masters
A system comprises a processor and a plurality of memory units. The processor is coupled to each of the plurality of memory units by a plurality of network connections. The processor includes a plurality of processing elements arranged in a two-dimensional array and a corresponding two-dimensional communication network communicatively connecting each of the plurality of processing elements to other processing elements on same axes of the two-dimensional array. Each processing element that is located along a diagonal of the two-dimensional array is configured as a request broadcasting master for a respective group of processing elements located along a same axis of the two-dimensional array.
1. A method comprising:
receiving a first memory request provided from a first request broadcasting master at a first connection to a memory unit;
receiving a second memory request provided from a second request broadcasting master at a second connection to the memory unit, wherein the first request broadcasting master and the second request broadcasting master are located on a diagonal of a two-dimensional array of processing elements;
decomposing the first memory request into a first plurality of partial requests and the second memory request into a second plurality of partial requests;
determining for each partial request of the first plurality of partial requests and the second plurality of partial requests whether the partial request is to be served from associated memory banks;
discarding a first group of partial requests from the first plurality of partial requests and the second plurality of partial requests that is not to be served from the associated memory banks;
for each partial request of a second group of partial requests from the first plurality of partial requests that is to be served from the associated memory banks, retrieving data of the partial request and providing the retrieved data to the first request broadcasting master using the first connection to the memory unit; and
for each partial request of a third group of partial requests from the second plurality of partial requests that is to be served from the associated memory banks, retrieving data of the is partial request and providing the retrieved data to the second request broadcasting master using the second connection of the memory unit.
2. A system, comprising:
a memory unit configured to:
receive a first memory request provided from a first request broadcasting master at a first connection to the memory unit;
receive a second memory request provided from a second request broadcasting master at a second connection to the memory unit, wherein the first request broadcasting master and the second request broadcasting master are located on a diagonal of a two-dimensional array of processing elements;
decompose the first memory request into a first plurality of partial requests and the second memory request into a second plurality of partial requests;
determine for each partial request of the first plurality of partial requests and the second plurality of partial requests whether the partial request is to be served from associated memory banks;
discard a first group of partial requests from the first plurality of partial requests and the second plurality of partial requests that is not to be served from the associated memory banks;
for each partial request of a second group of partial requests from the first plurality of partial requests that is to be served from the associated memory banks, retrieve data of the partial request and providing the retrieved data to the first request broadcasting master using the first connection to the memory unit; and
for each partial request of a third group of partial requests from the second plurality of partial requests that is to be served from the associated memory banks, retrieve data of the partial request and providing the retrieved data to the second request broadcasting master using the second connection of the memory unit.