Lossless compression of neural network weights
View Patent ↗A system and a method provide compression and decompression of weights of a layer of a neural network. For compression, the values of the weights are pruned and the weights of a layer are configured as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor. The tensor is formatted into at least one block of values. Each block is encoded independently from other blocks of the tensor using at least one lossless compression mode. For decoding, each block is decoded independently from other blocks using at least one decompression mode corresponding to the at least one compression mode used to compress the block; and deformatted into a tensor having the size of H×W×C.
1. A system to compress weights of a layer of a neural network, the system comprising:
a processor programmed to initiate executable operations comprising:
configuring, using the processor, the weights of a layer as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor;
formatting the tensor into at least one block of values; and
encoding the at least one block independently from other blocks of the tensor using Sparse-Exponential-Golomb-RemoveMin lossless compression encoding.
2. The system of claim 1 ,
wherein the at least one block is encoded independently from other blocks of the tensor using the lossless compression mode.
3. The system of claim 1 , wherein the processor is further programmed to initiate executable operations comprising pruning the weights to form pruned weights before configuring the weights as a tensor.
4. The system of claim 1 , wherein the processor is further programmed to initiate executable operations comprising quantizing values of the weights to form quantized weights before configuring the weights as a tensor.
5. The system of claim 1 , wherein the processor is further programmed to initiate executable operations comprising transforming values of the weights to a smaller domain before configuring the weights as a tensor.
6. The system of claim 1 , wherein encoding the at least one block further comprises generating a metadata structure indicating an address in memory of each encoded block.
7. The system of claim 1 , wherein the executable operations further comprise outputting the at least one block encoded as a bit stream.
8. The system of claim 7 , wherein executable operations further comprise:
decoding the at least one block independently from other blocks of the tensor using a decompression mode corresponding to the compression mode used to compress the at least one block; and
deformatting the at least one block into a tensor having the size of H×W×C.
9. A method to compress weights of a neural network, the method comprising:
configuring, using a processor, the weights of a layer as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor;
formatting the tensor into at least one block of values; and
encoding the at least one block independently from other blocks of the tensor using Sparse-Exponential-Golomb-RemoveMin lossless compression encoding.
10. The method of claim 9 ,
wherein encoding the at least one block further comprises encoding the at least one block independently from other blocks of the tensor using the lossless compression mode.
11. The method of claim 9 , further comprising pruning the weights to form pruned weights before configuring the weights as a tensor.
12. The method of claim 9 , further comprising quantizing values of the weights to form quantized weights before configuring the weights as a tensor.
13. The method of claim 9 , wherein encoding the at least one block further comprises generating a metadata structure indicating an address in memory of each encoded block.
14. The method of claim 9 , further comprising outputting the at least one block encoded as a bit stream.
15. The method of claim 14 , further comprising:
decompressing, using the processor, the at least one block independently from other blocks of the tensor using a decompression mode corresponding to the compression mode used to compress the at least one block; and
deformatting the at least one block into a tensor have the size of H×W×C.
16. A method to decompress encoded weights of a neural network, the method comprising:
decompressing, using a processor, an encoded block of values of a bitstream representing values of weights to form at least one decompressed block of values, the decompressed block of values being independently decompressed from other blocks of the weights using Sparse-Exponential-Golomb-RemoveMin lossless compression encoding; and
deformatting the decompressed block to be part of a tensor having a size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor, the tensor being the decompressed weights.
17. The method of claim 16 , further comprising:
configuring, using the processor, the weights as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor;
formatting the tensor into at least one block of values; and
encoding the at least one block independently from other blocks of the tensor using the lossless compression mode,
wherein compressing the at least one block further comprises compressing the at least one block independently from other blocks of the tensor using the lossless compression mode.