Distributed placement of linear operators for accelerated deep learning
Techniques in distributed placement of linear operators for accelerated deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element comprises a compute element to execute programmed instructions using the data and a router to route the wavelets. The routing is in accordance with virtual channel specifiers of the wavelets and controlled by routing configuration information of the router. A software stack determines distributed placement of linear operators based on a description of a neural network. The determined placement is used to configure the routers including usage of the respective colors. The determined placement is used to configure the compute elements including the respective programmed instructions each is configured to execute.
1 . A method comprising:
extracting a model from a neural network description, wherein the extracted model comprises a plurality of nodes;
identifying, based on the extracted model, an element from a library of executable kernel modules, the identified element corresponding to at least one node of the plurality of nodes in the extracted model, wherein the identified element is associated with executable code;
determining accelerator configuration information usable to configure a deep learning accelerator to provide a trained model, wherein:
the deep learning accelerator comprises a fabric and a plurality of processing elements to communicate packets with each other via the fabric in accordance with a plurality of communication pathways identifiable by respective virtual channel identifiers;
each of the plurality of processing elements comprises a respective compute element to execute programmed instructions based at least in part on respective compute element configuration information;
the accelerator configuration information comprises the respective compute element configuration information; and
the respective compute element configuration information comprises the executable code; and
configuring the deep learning accelerator based on the accelerator configuration information.
2 . The method of claim 1 , wherein the plurality of processing elements is a plurality of logical processing elements, a target wafer comprises a plurality of physical processing elements each having a respective physical location in a context of the target wafer, and each of the plurality of logical processing elements has a correspondence to a respective one of the plurality of physical processing elements.
3 . The method of claim 2 , wherein the determining comprises assigning computations associated with respective nodes of the extracted model to respective portions of the plurality of logical processing elements in accordance with the respective physical locations.
4 . The method of claim 3 , wherein the determining comprises identifying a region of physically contiguous ones of the plurality of physical processing elements, cutting the identified region orthogonal to a boundary of the identified region into two sub-regions, evaluating each of the sub-regions with respect to a placement of a delay buffer, and responsive to the evaluating ascertaining that the placement is a better one for the delay buffer, indicating that the placement is a best placement for the delay buffer.
5 . The method of claim 3 , wherein the determining further comprises performing a first routing of all communication paths between a plurality of regions of the plurality of physical processing elements, evaluating a heatmap in accordance with the first routing, inserting obstacles responsive to the heatmap, and performing a second routing of all the communication paths.
6 . The method of claim 3 , wherein the determining further comprises evaluating a wire cost based on Manhattan distance.
7 . The method of claim 6 , wherein the wire cost accounts for bandwidth of communication between the computations.
8 . The method of claim 1 , wherein each of the plurality of compute elements comprises respective one or more registers and the respective compute element configuration information comprises respective settings for at least a portion of the respective registers.
9 . The method of claim 1 , wherein each of the plurality of compute elements is to store programmed instructions for execution and the respective compute element configuration information comprises respective instruction code corresponding to the stored programmed instructions of the respective compute element.
10 . The method of claim 1 , wherein each of the executable kernel modules is associated with a respective template code generator enabled to generate the executable code associated with the respective executable kernel module.
11 . The method of claim 10 , wherein at least one of the template code generators is to accept arguments specifying dimensions, measured in numbers of the plurality of processing elements, to generate the executable code for.
12 . The method of claim 1 , wherein each of the executable kernel modules is associated with a respective cost model indicating any one or more of memory, bandwidth, and compute utilization used by the respective executable kernel module.
13 . The method of claim 1 , wherein one or more of the executable kernel modules comprise a hand-written microcode element.
14 . The method of claim 1 , wherein one or more of the executable kernel modules are associated with a respective utilization function that monotonically decreases with larger areas.
15 . The method of claim 1 , wherein at least one of the executable kernel modules is associated with a performance model that is usable to determine a shape of a compute region for the at least one executable kernel module.
16 . The method of claim 1 , further comprising evaluating one or more results of the determining in accordance with one or more predetermined cost criteria to produce one or more goal-evaluation metrics, conditionally altering one or more meta-parameters that the determining is based at least in part on wherein the conditionally altering is dependent on at least one of the one or more goal-evaluation metrics being less than a respective predetermined threshold, and repeating at least a portion of the determining in accordance with the altered meta-parameters.
17 . A non-transitory computer-readable medium comprising one or more instructions that, when executed by one or more processors, cause the one or more processors to perform actions comprising:
extracting a model from a neural network description, wherein the extracted model comprises a plurality of nodes;
identifying, based on the extracted model, an element from a library of executable kernel modules, the identified element corresponding to at least one node of the plurality of nodes in the extracted model, wherein the identified element is associated with executable code;
determining accelerator configuration information usable to configure a deep learning accelerator to provide a trained model, wherein:
the deep learning accelerator comprises a fabric and a plurality of processing elements to communicate packets with each other via the fabric in accordance with a plurality of communication pathways identifiable by respective virtual channel identifiers;
each of the plurality of processing elements comprises a respective compute element to execute programmed instructions based at least in part on respective compute element configuration information;
the accelerator configuration information comprises the respective compute element configuration information; and
the respective compute element configuration information comprises the executable code; and
configuring the deep learning accelerator based on the accelerator configuration information.
18 . The non-transitory computer-readable medium of claim 17 , wherein the plurality of processing elements is a plurality of logical processing elements, a target wafer comprises a plurality of physical processing elements each having a respective physical location in a context of the target wafer, and each of the plurality of logical processing elements has a correspondence to a respective one of the plurality of physical processing elements.
19 . The non-transitory computer-readable medium of claim 18 , wherein the determining comprises assigning computations associated with respective nodes of the extracted model to respective portions of the plurality of logical processing elements in accordance with the respective physical locations.
20 . The non-transitory computer-readable medium of claim 19 , wherein the determining comprises identifying a region of physically contiguous ones of the plurality of physical processing elements, cutting the identified region orthogonal to a boundary of the identified region into two sub-regions, evaluating each of the sub-regions with respect to a placement of a delay buffer, and responsive to the evaluating ascertaining that the placement is a better one for the delay buffer, indicating that the placement is a best placement for the delay buffer.
21 . The non-transitory computer-readable medium of claim 19 , wherein the determining further comprises performing a first routing of all communication paths between a plurality of regions of the plurality of physical processing elements, evaluating a heatmap in accordance with the first routing, inserting obstacles responsive to the heatmap, and performing a second routing of all the communication paths.
22 . The non-transitory computer-readable medium of claim 19 , wherein the determining further comprises evaluating a wire cost based on Manhattan distance.
23 . The non-transitory computer-readable medium of claim 22 , wherein the wire cost accounts for bandwidth of communication between the computations.
24 . The non-transitory computer-readable medium of claim 17 , wherein each of the plurality of compute elements comprises a respective one or more registers and the respective compute element configuration information comprises respective settings for at least a portion of the respective registers.
25 . The non-transitory computer-readable medium of claim 17 , wherein each of the plurality of compute elements is to store programmed instructions for execution and the respective compute element configuration information comprises respective instruction code corresponding to the stored programmed instructions of the respective compute element.
26 . The non-transitory computer-readable medium of claim 17 , wherein each of the executable kernel modules is associated with a respective template code generator enabled to generate the executable code associated with the respective executable kernel module.
27 . The non-transitory computer-readable medium of claim 26 , wherein at least one of the template code generators is enabled to accept arguments specifying dimensions, measured in numbers of the plurality of processing elements, to generate the executable code for.
28 . The non-transitory computer-readable medium of claim 17 , wherein each of the executable kernel modules is associated with a respective cost model indicating any one or more of memory, bandwidth, and compute utilization used by the respective executable kernel module.
29 . The non-transitory computer-readable medium of claim 17 , wherein one or more of the executable kernel modules comprise a hand-written microcode element.
30 . The non-transitory computer-readable medium of claim 17 , wherein one or more of the executable kernel modules are associated with a respective utilization function that monotonically decreases with larger areas.
31 . The non-transitory computer-readable medium of claim 17 , wherein at least one of the executable kernel modules is associated with a performance model that is usable to determine a shape of a compute region for the at least one executable kernel module.
32 . The non-transitory computer-readable medium of claim 17 , further comprising evaluating one or more results of the determining in accordance with one or more predetermined cost criteria to produce one or more goal-evaluation metrics, conditionally altering one or more meta-parameters that the determining is based at least in part on wherein the conditionally altering is dependent on at least one of the one or more goal-evaluation metrics being less than a respective predetermined threshold, and repeating at least a portion of the determining in accordance with the altered meta-parameters.