Mixed-precision processing elements, systems, and methods for computational models
The disclosed method may include (1) receiving a precision level of each weight associated with each input of a node of a computational model, (2) identifying, for each weight, one of a plurality of multiplier groups, where each multiplier group may include a plurality of hardware multipliers of a corresponding bit width, and where the corresponding bit width of the plurality of hardware multipliers of the one of the plurality of multiplier groups may be sufficient to multiply the weight by the associated input, and (3) multiplying each weight by its associated input using an available hardware multiplier of the one of the plurality of multiplier groups identified for the weight. Various other processing elements, methods, and systems are also disclosed.
1. A method comprising:
receiving a precision level of each weight associated with each input of a node of a computational model;
identifying, for each weight, one of a plurality of multiplier groups, wherein each multiplier group comprises a plurality of hardware multipliers of a corresponding bit width, and wherein the corresponding bit width of the plurality of hardware multipliers of the one of the plurality of multiplier groups is sufficient to multiply the weight by the associated input based on the precision level of the weight; and
multiplying each weight by its associated input using an available hardware multiplier of the one of the plurality of multiplier groups identified for the weight.
2. The method of claim 1 , wherein receiving the precision level of the weight associated with each input comprises receiving, for each weight, a sideband signal indicating the precision level of the weight.
3. The method of claim 1 , wherein receiving the precision level of the weight associated with each input comprises:
quantizing, prior to identifying, for each weight, one of a plurality of multiplier groups, the weight associated with each input of the node; and
detecting, after the quantizing of the weight associated with each input and prior to identifying, for each weight, one of a plurality of multiplier groups, the precision level of the weight associated with each input.
4. The method of claim 1 , wherein the weights range from a maximum negative integer to a maximum positive integer.
5. The method of claim 1 , wherein the one of the plurality of multiplier groups for each weight comprises the multiplier group corresponding to the smallest corresponding bit width sufficient to multiply the weight by the associated input.
6. The method of claim 1 , wherein:
identifying the one of the plurality of multiplier groups for each weight comprises determining, for one of the weights, that the multiplier group corresponding to the smallest corresponding bit width sufficient to multiply the weight by the associated input has no hardware multipliers currently available; and
the corresponding bit width of the plurality of hardware multipliers of the one of the plurality of multiplier groups is the smallest available bit width larger than the smallest corresponding bit width sufficient to multiply the weight by the associated input.
7. The method of claim 1 , wherein identifying one of the plurality of multiplier groups for each weight facilitates the multiplying of the weights and the associated inputs of the node in parallel.
8. The method of claim 1 , wherein:
each hardware multiplier of a first multiplier group is an n-by-n hardware multiplier;
each hardware multiplier of a second multiplier group is an m-by-n hardware multiplier;
n is a maximum number of significant bits of the weights of the node; and
m is less than n.
9. The method of claim 8 , wherein m and n are different powers of two.
10. The method of claim 1 , wherein multiplying each weight of the node by its associated input is performed within a single hardware processing cycle.
11. The method of claim 1 , further comprising adding together the products of multiplying each weight of the node by its associated input.
12. The method of claim 11 , further comprising:
applying an activation function to the sum of the products of multiplying each weight of the node by its associated input; and
forwarding the result of the activation function as an input for at least one other node of the computational model.
13. The method of claim 1 , wherein:
a first number of hardware multipliers in a first multiplier group are of a first corresponding bit width; and
a second number of hardware multipliers in a second multiplier group are of a second corresponding bit width, wherein the first number is less than the second number and the first corresponding bit width is greater than the second corresponding bit width.
14. A processing element comprising:
a plurality of multiplier groups, wherein each multiplier group comprises a plurality of hardware multipliers of a corresponding bit width; and
a multiplier group identification unit that:
receives a precision level of each weight associated with each input of a node of a computational model; and
identifies, for each weight, one of the plurality of multiplier groups, wherein the corresponding bit width of the plurality of hardware multipliers of the one of the plurality of multiplier groups is sufficient to multiply the weight by the associated input based on the precision level of the weight; and
wherein each weight is multiplied by its associated input using an available hardware multiplier of the one of the plurality of multiplier groups identified for the weight.
15. The processing element of claim 14 , wherein the multiplier group identification unit receives a sideband signal indicating the precision level of the weight associated with each input of the node.
16. The processing element of claim 14 , further comprising a precision level determination unit that:
receives the weight associated with each input of the node;
quantizes the weight associated with each input of the node; and
detects, after quantizing the weight associated with each input, the precision level of the weight associated with each input.
17. The processing element of claim 14 , further comprising a hardware accumulator that sums products provided by the hardware multipliers of the processing element.
18. The processing element of claim 14 , wherein the one of the plurality of multiplier groups for each weight comprises the multiplier group corresponding to the smallest corresponding bit width sufficient to multiply the weight by the associated input.
19. A system comprising:
a plurality of hardware processing elements that operate as nodes of a computational model, wherein at least one of the hardware processing elements comprises:
a plurality of multiplier groups, wherein each multiplier group comprises a plurality of hardware multipliers of a corresponding bit width; and
a multiplier group identification unit that:
receives a precision level of each weight associated with each input of a node of the computational model; and
identifies, for each weight, one of the plurality of multiplier groups, wherein the corresponding bit width of the plurality of hardware multipliers of the one of the plurality of multiplier groups is sufficient to multiply the weight by the associated input based on the precision level of the weight; and
wherein each weight is multiplied by its associated input using an available hardware multiplier of the one of the plurality of multiplier groups identified for the weight.