Neural network chip for ear-worn device
A hearing aid may include a neural network chip having tiles arranged in an array, each tile including memory, 16-128 multiplier-accumulator circuits (MACs), and routing circuitry. The memory of each tile may be configured to store a portion of elements of a matrix A comprising weights of a recurrent neural network. Each tile may be configured to receive and store elements of an activation vector X, and all tiles in a column of the array may be configured to receive the same elements of X. The plurality of tiles may be configured to perform a matrix-vector multiplication A*X by performing multiply-and-accumulate sub-operations in parallel among the plurality of tiles. The routing circuitry from the tiles in each respective row of tiles may be configured to combine results of the multiply-and-accumulate sub-operations.
1. A hearing aid, comprising:
a neural network chip, comprising:
a plurality of tiles arranged in a tile array, each tile comprising memory, a number of multiplier-accumulator circuits (MACS) in a range of 16-128 MACs, and routing circuitry;
wherein:
the memory of each tile is configured to store a portion of elements of a matrix A comprising weights of a recurrent neural network;
each tile is configured to receive and store elements of an activation vector X, wherein the activation vector X is derived from an input audio signal;
all or a subset of the plurality of tiles are configured to perform a matrix-vector multiplication A*X by performing multiply-and-accumulate sub-operations in parallel among all or the subset of the plurality of tiles;
routing circuitries of tiles in each respective row of tiles in the tile array are configured to combine results of the multiply-and-accumulate sub-operations; and
all tiles in a column of the tile array are configured to receive same elements of the activation vector X.
2. The hearing aid of claim 1 , wherein the memory and the MACs of any given tile are disposed within an area no larger than 0.25 mm{circumflex over ( )}2.
3. The hearing aid of claim 1 , wherein a tile of the plurality of tiles is configured to reuse an element of the activation vector X across all calculations performed by the MACs in the tile on a given clock cycle.
4. The hearing aid of claim 1 , wherein a tile of the plurality of tiles is configured to simultaneously fan out a single element of the activation vector X from the memory to each of the MACs in the tile.
5. The hearing aid of claim 1 , wherein all tiles in a column of the tile array are coupled to a vector memory only by a single, shared bus.
6. The hearing aid of claim 1 , wherein the tile array lacks independent connections between adjacent tiles in a column of the tile array.
7. The hearing aid of claim 6 , wherein a tile in the column lacks capability to output data to another tile in the column.
8. The hearing aid of claim 1 , wherein the neural network chip lacks capability to transmit different elements of the activation vector X to different tiles in a column.
9. The hearing aid of claim 1 , wherein all memory on the neural network chip together comprises no more than approximately 40 Mbits of memory for the weights of the recurrent neural network.
10. The hearing aid of claim 1 , wherein an area of the neural network chip is in a range of approximately 9 mm{circumflex over ( )}2 to approximately 14 mm{circumflex over ( )}2.
11. The hearing aid of claim 1 , wherein the neural network chip is approximately equal to or less than 20 mm{circumflex over ( )}2 in area.
12. The hearing aid of claim 1 , wherein:
the neural network chip further comprises a plurality of bias circuits, each bias circuit electrically coupled with one row of the plurality of tiles in the tile array and comprising bias memory and routing circuitry;
each bias circuit of the plurality of bias circuits is configured to receive and store one or more biases in the bias memory; and
routing circuitries of tiles in a respective row of tiles in the tile array and routing circuitry of a bias circuit electrically coupled with the respective row of tiles are configured to combine the results of the multiply-and-accumulate sub-operations with the one or more biases.
13. The hearing aid of claim 1 , wherein the neural network chip further comprises:
short-time Fourier transform (STFT) circuitry and inverse short-time Fourier transform (iSTFT) circuitry configured to perform STFT on audio signals coming from off-chip and iSTFT on audio signals going off-chip, respectively.
14. The hearing aid of claim 13 , wherein the activation vector X for a first layer of the recurrent neural network is a result of processing the input audio signal coming from off-chip with the STFT circuitry.
15. The hearing aid of claim 1 , wherein the recurrent neural network is configured to perform de-noising of audio signals.
16. The heating aid of claim 1 , wherein the neural network chip is configured to disable a subset of the plurality of tiles within the tile array.
17. The hearing aid of claim 16 , wherein the neural network chip is configured to disable the subset of the plurality of tiles within the tile array based on sizes of the matrix A and/or the activation vector X.
18. The hearing aid of claim 16 , wherein the neural network chip is further configured to disable the subset of the plurality of tiles within the tile array when estimating a signal-to-noise ratio (SNR) of the input audio signal.
19. The hearing aid of claim 18 , wherein the neural network chip is further configured to estimate the SNR of the input audio signal with one tile.
20. The hearing aid of claim 16 , wherein the neural network chip is further configured to select the subset of the plurality of tiles within the tile array to disable based on a target amount of de-noising to be provided by the recurrent neural network.