PLACEMENT OF COMPUTE AND MEMORY FOR ACCELERATED DEEP LEARNING
Techniques in placement of compute and memory for accelerated deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element comprises a compute element to execute programmed instructions using the data and a router to route the wavelets. The routing is in accordance with virtual channel specifiers of the wavelets and controlled by routing configuration information of the router. A software stack determines placement of compute resources and memory resources based on a description of a neural network. The determined placement is used to configure the routers including usage of the respective colors. The determined placement is used to configure the compute elements including the respective programmed instructions each is configured to execute.
1 - 45 . (canceled)
46 . A method comprising:
determining one or more costs associated with processing computations of one or more model layers of a model;
identifying, based on the one or more costs, a compute fabric region of an integrated circuit comprising a plurality of processing elements used to implement the one or more model layers based on the one or more costs; and
allocating the processing computations of the one or more model layers to the compute fabric region.
47 . The method of claim 46 , wherein the plurality of processing elements comprises physically contiguous processing elements, each having a respective location on a wafer, and wherein the allocating the processing computations of the one or more model layers comprises allocating, based on the respective locations, the processing computations associated with respective model layers of the model to respective portions of the physically contiguous processing elements.
48 . The method of claim 46 , wherein the identifying, based on the one or more costs, the compute fabric region comprises:
subdividing the compute fabric region to form a plurality of non-overlapping sub-regions;
evaluating, based on the one or more costs, each of the plurality of non-overlapping sub-regions; and
allocating, based on the evaluating, the processing computations of each of the one or more model layers to one of the plurality of non-overlapping sub-regions.
49 . The method of claim 48 , wherein subdividing the compute fabric region comprises:
determining, based on respective costs associated with the processing computations of each model layer, respective region areas for each model layer;
determining, based on binary partitioning of the respective region areas, parameters of the plurality of non-overlapping sub-regions; and
allocating the respective region areas to the one or more model layers.
50 . The method of claim 49 , further comprising:
altering, based on distances between the one or more model layers, a placement layout of the respective region areas by swapping allocation of the one or more model layers between two or more of the respective region areas.
51 . The method of claim 46 , wherein the identifying, based on the one or more costs, the compute fabric region comprises:
determining a number of computations associated with the one or more model layers; and
determining that utilization of the plurality of processing elements used to implement the one or more model layers meets the number of computations.
52 . The method of claim 46 , wherein the determining the one or more costs associated with the processing computations comprises determining one or more of a compute cost, a memory cost, a kernel cost, a placement cost, and a routing cost.
53 . The method of claim 46 , further comprising:
determining one or more parameters of the compute fabric region, wherein the one or more parameters comprises any of a number of processing elements, a region size, a region shape, a number of routes between processing elements, and a fabric color indicating one or more virtual channels between processing elements.
54 . The method of claim 53 , further comprising:
training the model using the compute fabric region;
receiving performance data associated with training the model; and
altering, based at least in part on the performance data, the one or more parameters of the compute fabric region.
55 . The method of claim 46 , further comprising:
determining processing times associated with the processing computations of each model layer; and
adding one or more delay buffers between sub-regions of the compute fabric region such that output timing of the processing computations of the one or more model layers is concurrent.
56 . A system comprising:
a compute fabric; and
processing circuitry configured to:
determine one or more costs associated with processing computations of one or more model layers of a model;
identify, based on the one or more costs, a compute fabric region, of the compute fabric, comprising a plurality of processing elements used to implement the one or more model layers based on the one or more costs; and
allocate the processing computations of the one or more model layers to the compute fabric region.
57 . The system of claim 56 , wherein the plurality of processing elements comprises physically contiguous processing elements, each having a respective location on a wafer, and wherein the processing circuitry, when allocating the processing computations of the one or more model layers, is configured to allocate, based on the respective locations, the processing computations associated with respective model layers of the model to respective portions of the physically contiguous processing elements.
58 . The system of claim 56 , wherein the processing circuitry, when identifying, based on the one or more costs, the compute fabric region, is configured to:
subdivide the compute fabric region to form a plurality of non-overlapping sub-regions;
evaluate, based on the one or more costs, each of the plurality of non-overlapping sub-regions; and
allocate, based on the evaluating, the processing computations of each of the one or more model layers to one of the plurality of non-overlapping sub-regions.
59 . The system of claim 58 , wherein the processing circuitry, when subdividing the compute fabric region, is configured to:
determine, based on respective costs associated with the processing computations of each model layer, respective region areas for each model layer;
determine, based on binary partitioning of the respective region areas, parameters of the plurality of non-overlapping sub-regions; and
allocate the respective region areas to the one or more model layers.
60 . The system of claim 59 , wherein the processing circuitry is further configured to:
alter, based on distances between the one or more model layers, a placement layout of the respective region areas by swapping allocation of the one or more model layers between two or more of the respective region areas.
61 . The system of claim 56 , wherein the processing circuitry, when identifying, based on the one or more costs, the compute fabric region, is configured to:
determine a number of computations associated with the one or more model layers; and
determine that utilization of the plurality of processing elements used to implement the one or more model layers meets the number of computations.
62 . The system of claim 56 , wherein the processing circuitry, when determining the one or more costs associated with the processing computations, is configured to determine one or more of a compute cost, a memory cost, a kernel cost, a placement cost, and a routing cost.
63 . The system of claim 56 , wherein the processing circuitry is further configured to:
determine one or more parameters of the compute fabric region, wherein the one or more parameters comprises any of a number of processing elements, a region size, a region shape, a number of routes between processing elements, and a fabric color indicating one or more virtual channels between processing elements.
64 . The system of claim 63 , wherein the processing circuitry is further configured to:
train the model using the compute fabric region;
receive performance data associated with training the model; and
alter, based at least in part on the performance data, the one or more parameters of the compute fabric region.
65 . The system of claim 56 , wherein the processing circuitry is further configured to:
determine processing times associated with the processing computations of each model layer; and
add one or more delay buffers between sub-regions of the compute fabric region such that output timing of the processing computations of the one or more model layers is concurrent.