SYSTEM AND METHOD FOR EFFICIENT UTILIZATION OF MULTIPLIERS IN NEURAL-NETWORK COMPUTATIONS
A system and method for performing neural network calculations may include selecting a size in bits for representing a plurality of weight elements of the neural network based on a value of the weight elements. In each computational cycle: if the size in bits of a weight element of the plurality of weight elements is N, configuring an N*K multiply accumulator to perform one multiply-accumulate operation of a K-bit data element and the N-bit weight element; and if the size in bits of at least two N/M-bit weight elements of the plurality of weight elements is N/M, configuring the N*K multiply accumulator to perform up to N/M multiply-accumulate operations, each of a K-bit, data element and an N/M-bit weight element, where N, K and M are integers bigger than one, N is a power of 2, M is even and N≥M.
1 . A method for performing multiplications in a computer system, the method comprising:
determining a size in bits of weight elements:
configuring an N*K multiply accumulator to perform at least two multiply operations in parallel, if the size in bits of at least two weight elements is not bigger than N/M, where K is an integer bigger than one, each of N and M is a power of 2 and N≥M.
2 . The method of claim 1 , comprising:
configuring the N*K multiply accumulator to perform N/M multiply operations in parallel, if the size in bits of M weight elements is N/M.
3 . The method of claim 1 , comprising:
configuring the N*K multiply accumulator to perform one multiply operation, if the size in bits of a weight element is N.
4 . The method of claim 1 , comprising:
obtaining a weight packet, the weight packet including a header indicative of the size in bits of weight elements in the weight packet, wherein the size in bits of the weight elements in the weight packet is determined based on the header.
5 . The method of claim 4 , comprising selecting the size in bits for representing the weight elements in the weight packet based on a value of the weight elements.
6 . The method of claim 1 , wherein the weight elements pertain to a neural network.
7 . The method of claim 1 , comprising accumulating the results of the at least two multiply operations with the results of previous multiplications performed by the N*K multiply accumulator.
8 . The method of claim 7 , wherein N=16, and the value of M is selectable from 1, 2 and 4.
9 . A method for performing neural network calculations, the method comprising:
selecting a size in bits for representing a plurality of weight elements of the neural network based on a value of the weight elements;
in each computational cycle:
if the size in bits of a weight element of the plurality of weight elements is N, configuring an N*K multiply accumulator to perform one multiply-accumulate operation of a K-bit data element and the N-bit weight element; and
if the size in bits of at least two N/M-bit weight elements of the plurality of weight elements is N/M, configuring the N*K multiply accumulator to perform up to N/M multiply-accumulate operations, each of a K-bit data element and an N/M-bit weight element,
wherein N, K and M are integers bigger one, N is a power of 2, M is even and N≥M.
10 . The method of claim 9 , wherein N=16, and the value of M is selectable from 2 and 4.
11 . A neural network hardware accelerator comprising:
a weight packet buffer configured to store at least one weight packet;
a data queue configured to store at least M data elements;
an N*K multiplier-accumulator comprising:
an N*K multiplier;
an adder; and
an accumulator;
wherein the neural network hardware accelerator is configured to:
determine a size in bits of weight elements in the at least one weight packet;
configure the N*K multiply accumulator to perform at least two multiply operations in parallel, if the size in bits of at least two of the weight elements is not bigger than N/M, where N, K and M are integers bigger than one, N is a power of 2, M is even and N≥M.
12 . The neural network hardware accelerator of claim 11 , wherein the neural network hardware accelerator is configured to:
configure the N*K multiply accumulator to perform N/M multiply operations in parallel, if the size in bits of M weight elements is N/M.
13 . The neural network hardware accelerator of claim 11 , wherein the neural network hardware accelerator is configured to:
configure the N*K multiply accumulator to perform one multiply operation, if the size in bits of a weight elements is N.
14 . The neural network hardware accelerator of claim 11 , wherein the neural network hardware accelerator is configured to:
obtain a weight packet, the weight packet including a header indicative of the size in bits of weight elements in the weight packet, wherein the size in bits of the weight elements in the weight packet is determined based on the header.
13 . The neural network hardware accelerator of claim 14 , wherein the neural network hardware accelerator is configured to select the size in bits for representing the weight elements in the weight packet based on a value of the weight elements.
16 . The neural network hardware accelerator of claim 11 , wherein the weight elements pertain to a neural network.
17 . The neural network hardware accelerator of claim 11 , wherein the neural network hardware accelerator is configured to accumulate the results of the at least two multiply operations with the results of previous multiplications performed by the N*K multiply accumulator.
18 . The neural network hardware accelerator of claim 11 , wherein N=16, and the value of M is selectable from 1, 2 and 4.