Neural network processor with on-chip convolution kernel storage
A neural network processor architecture provides decompression circuitry that can exploit patterns of data in kernel weights of a convolutional neural net as flattened to a vector, the compression allowing reduced kernel data storage costs including on-chip-storage.
1 . A neural network processor operating to receive data and to classify that data comprising:
an input register holding received data for classification according to a kernel of multiple kernel weight values of a neural network trained to provide a set of classifications;
a codebook storage memory holding data permitting a mapping of individual codeword values each, to a pattern of multiple kernel weight values;
a codeword memory holding codeword data to provide generation of a reconstructed kernel reconstructing the kernel by indexing the codebook storage memory with codeword data;
arithmetic circuitry communicating with the input register, the codebook storage memory, and the codeword memory to generate output representing a dot product between the received data and the reconstructed kernel to classify the received data according to the set of classifications using the reconstructed kernel; and
wherein the input register, codebook storage memory, codeword memory, and arithmetic circuit are held on a single integrated circuit substrate.
2 . The neural network processor of claim 1 wherein the patterns of multiple kernel weight values are contiguous weight values within a matrix of the kernel.
3 . The neural network processor of claim 2 wherein the patterns of multiple kernel weight values are contiguous weight values along a predetermined dimension of the kernel data.
4 . The neural network processor of claim 3 wherein the patterns of multiple kernel weight values have beginning and end values aligned within the matrix of the kernel.
5 . The neural network processor of claim 4 wherein the beginning and end values extend at least a full extent of a dimension of the matrix of the kernel.
6 . The neural network processor of claim 1 wherein the reconstructed kernel is a scalar compression of the kernel of the neural network trained to provide the set of classifications.
7 . The neural network processor of claim 6 wherein the scalar compression replaces a range of kernel data values with a predetermined scalar value.
8 . The neural network processor of claim 1 wherein the patterns of multiple kernel weight values are a product quantization of vectors of the kernel of the neural network trained to provide a set of classifications.
9 . The neural network processor of claim 8 wherein the product quantization provides multiple individual codeword values associated with different but contiguous patterns of multiple kernel weights.
10 . The neural network processor of claim 1 wherein the arithmetic circuitry employs the data of the codebook storage memory to precompute a set of multiplications between the received data and data of the codebook storage memory to populate a first lookup table and repeatedly uses the precomputed set of multiplications according to data of the codeword memory in generation of the output representing a dot product between the received data and the reconstructed kernel.
11 . The neural network processor of claim 10 wherein the arithmetic circuit further includes a second lookup table and further operates to populate the second lookup table with sums formed using the data of the codeword storage memory and the first lookup table and uses the second lookup table to form the dot product.
12 . The neural network processor of claim 11 wherein the codebook storage memory holds a scalar-quantized codebook that can be reconstructed into a product-quantized codebook using scalar-quantized codewords of the codeword memory, and wherein the product-quantized codebook can be reconstructed into a kernel using product-quantized codewords stored in the codeword memory.
13 . The neural network processor of claim 11 wherein the data of the first and second lookup tables are stored for reuse between successive received data separated by a convolution of the kernel on an input data set.