Compression of kernel data for neural network operations
Embodiments relate to a neural processor circuit that includes a kernel access circuit and multiple neural engine circuits. The kernel access circuit reads compressed kernel data from memory external to the neural processor circuit. Each neural engine circuit receives compressed kernel data from the kernel access circuit. Each neural engine circuit includes a kernel extract circuit and a kernel multiply-add (MAD) circuit. The kernel extract circuit extracts uncompressed kernel data from the compressed kernel data. The kernel MAD circuit receives the uncompressed kernel data from the kernel extract circuit and performs neural network operations on a portion of input data using the uncompressed kernel data.
1 . A processor circuit, comprising:
an access circuit coupled to memory external to the processor circuit, the access circuit configured to read compressed kernel data from the memory, wherein the kernel data comprises kernel coefficients of a matrix, wherein the compressed kernel data comprises:
indices of a look up table (LUT), wherein the LUT is configured to store representative values having a first precision approximating the kernel coefficients having a second precision, wherein the LUT has fewer distinct representative values than a number of kernel coefficients of the kernel data, and
a mask value that indicates one or more locations of non-zero coefficients of the matrix in the kernel data;
an extract circuit coupled to the access circuit, the extract circuit configured to extract uncompressed kernel data from the compressed kernel data, wherein each of the uncompressed kernel data comprises a representative value selected from the representative values having the first precision stored in the LUT whose index is used as the compressed kernel data; and
a computation circuit coupled to the extract circuit and configured to receive the uncompressed kernel data, the computation circuit further configured to perform computation operations using the uncompressed kernel data.
2 . The processor circuit of claim 1 , wherein each of the computation operations is part of a convolution operation.
3 . The processor circuit of claim 1 , wherein the computation circuit skips one or more of the computation operations associated with coefficients that are zero.
4 . The processor circuit of claim 1 , further comprising additional computation circuits coupled to the extract circuit, each of the additional computation circuits configured to perform computation operations using the uncompressed kernel data.
5 . The processor circuit of claim 4 , wherein the extract circuit comprises:
a reconstruction circuit configured to reconstruct uncompressed kernel data corresponding to each of the computation circuit and the additional computation circuits, and
a look-ahead buffer configured to store the uncompressed kernel data.
6 . The processor circuit of claim 5 , wherein the look-ahead buffer is configured to send information on locations of coefficients that are zero in the uncompressed kernel data to the computation circuit before sending coefficients that are non-zero so that the computation circuit skips one or more operations associated with the coefficients that are zero.
7 . The processor circuit of claim 1 , wherein the extract circuit is further configured to extract, from the compressed kernel data, a parameter to configure the computation circuit.
8 . The processor circuit of claim 7 , wherein the extract circuit is further configured to extract, from the compressed kernel data, another parameter for configuring a post-processor circuit that performs an operation on a result of the computation operations.
9 . The processor circuit of claim 1 , wherein the extract circuit further comprises a look-up table storage configured to store the look-up table comprising the representative values identified by the indices and the representative values determined during a compilation process based on coefficients of original kernel data before compression.
10 . The processor circuit of claim 1 , wherein the compressed kernel data comprises a plurality of matrix groups, wherein each of the matrix groups includes a mask indicating non-zero coefficients of a single matrix.
11 . A method of operating a processor circuit, comprising:
reading, by an access circuit, compressed kernel data from a memory external to the processor circuit, wherein the kernel data comprises kernel coefficients of a matrix, wherein the compressed kernel data includes:
indices of a look up table (LUT), wherein the LUT is configured to store representative values having a first precision approximating kernel coefficients having a second precision, wherein the LUT has fewer distinct representative values than a number of kernel coefficients of the kernel data, and
a mask value for the matrix indicative of one or more locations in the matrix where the kernel coefficients are non-zero;
extracting, by an extract circuit, uncompressed kernel data from the compressed kernel data using the mask value, wherein each of the uncompressed kernel data comprises a representative value selected from the representative values having the first precision stored in the LUT whose index is used as the compressed kernel data; and
performing, by a computation circuit, computation operations using the uncompressed kernel data.
12 . The method of claim 11 , wherein each of the computation operations is part of a convolution operation.
13 . The method of claim 11 , further comprising skipping one or more of the computation operations associated with coefficients that are zero.
14 . The method of claim 11 , further comprising performing additional computation operations using the uncompressed kernel data by additional computation circuits.
15 . The method of claim 14 , further comprising:
reconstructing the uncompressed kernel data corresponding to each of the computation circuit and the additional computation circuits, and
storing the uncompressed kernel data in a look-ahead buffer.
16 . The method of claim 15 , further comprising:
sending information on locations of coefficients that are zero in the uncompressed kernel data by the extract circuit to the computation circuit before sending coefficients that are non-zero so that the computation circuit skips one or more operations associated with the coefficients that are zero.
17 . The method of claim 11 , further comprising:
extracting, from the compressed kernel data, a parameter to configure the computation operations of the computation circuit.
18 . The method of claim 17 , further comprising:
extracting, from the compressed kernel data, another parameter for configuring a post-processor circuit that performs an operation on a result of the computation operations.
19 . The method of claim 11 , further comprising:
storing in the look-up tables the representative values identified by the indices, wherein the representative values are determined during a compilation process based on coefficients of original kernel data before compression.
20 . An electronic device, comprising:
memory storing compressed kernel data; and
a processor circuit coupled to the memory, the processor circuit comprising:
an access circuit coupled to the memory, the access circuit configured to read compressed kernel data from the memory, wherein the kernel data comprises kernel coefficients of a matrix, the compressed kernel data comprises:
indices of a look up table (LUT), wherein the LUT is configured to store representative values having a first precision approximating the kernel coefficients having a second precision, wherein the LUT has fewer distinct representative values than a number of kernel coefficients of the kernel data, and
a mask value that indicates one or more locations of non-zero coefficients of the matrix in the kernel data;
an extract circuit coupled to the access circuit, the extract circuit configured to extract uncompressed kernel data from the compressed kernel data, wherein each of the uncompressed kernel data comprises a representative value selected from the representative values having the first precision stored in the LUT whose index is used as the compressed kernel data; and
a computation circuit coupled to the extract circuit to receive the uncompressed kernel data, the computation circuit further configured to perform computation operations using the uncompressed kernel data.