Systems and methods for compression and distribution of machine learning models

1. A computer-implemented method to compress machine-learned models, the method comprising:

obtaining, by one or more computing devices, a machine-learned model;

selecting, by the one or more computing devices, a weight to be quantized;

quantizing, by the one or more computing devices, the weight;

propagating, by the one or more computing devices, at least a part of a quantization error to one or more non-quantized weights;

quantizing, by the one or more computing devices, one or more of the non-quantized weights; and

providing, by the one or more computing devices, a quantized machine-learned model.

2. The computer-implemented method of claim 1 wherein the step of propagating the quantization error comprises:

determining, by the one or more computing devices, the quantization error;

determining, by the one or more computing devices, one or more non-quantized weights associated with the weight; and

allocating, by the one or more computing devices, a fractional part of the quantization error to each of the associated non-quantized weights.

3. The computer-implemented method of claim 2 wherein the fractional part of the quantization error is allocated to each of the associated non-quantized weights based in part on a predetermined multiplier.

4. The computer-implemented method of claim 2 wherein:

determining, by the one or more computing devices, the quantization error comprises determining, by the one or more computing devices, the quantization error associated with a first node in a first layer of the machine-learned model; and

determining, by the one or more computing devices, one or more non-quantized weights associated with the weight comprises identifying a second node in the first layer, wherein the first and second nodes each have a connection to a third node in a second layer.

5. The computer-implemented method of claim 2 wherein allocating, by the one or more computing devices, a fractional part of the quantization error to each of the associated non-quantized weights comprises dithering, by the one or more computing devices, the quantization error within a node of the model.

6. The computer-implemented method of claim 5 further comprising dithering, by the one or more computing devices, the quantization error within a node of the model based in part on a correlation of activity between nodes of the model.

7. The computer-implemented method of claim 6 wherein when there is an inverse correlation of activity between nodes of the model, the quantization error is dithered in an opposite direction.

8. The computer-implemented method of claim 1 wherein the step of propagating the quantization error comprises:

locking, by the one or more computing devices, the quantized weight; and

performing, by the one or more computing devices, additional training iterations of the machine-learned model, wherein performing the additional training iterations comprises updating a non-quantized weight.

9. The computer-implemented method of claim 8 further comprising locking, by the one or more computing devices, a number of quantized best matching coefficient weights.

10. The computer-implemented method of claim 1 wherein selecting the weight to be quantized comprises selecting a best matching coefficient.

11. A computing device comprising:

one or more processors; and

one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing device to:

obtain a machine-learned model;

select a weight to be quantized;

quantize the weight;

propagate at least a part of a quantization error to one or more non-quantized weights;

quantize one or more of the non-quantized weights; and

provide a quantized machine-learned model.

12. The computing device of claim 11 wherein propagating the quantization error comprises:

determining the quantization error;

determining one or more non-quantized weights associated with the weight; and

allocating a fractional part of the error to each of the associated non-quantized weights.

13. The computing device of claim 12 wherein the fractional part of the quantization error is allocated to each of the associated non-quantized weights based in part on a predetermined multiplier.

14. The computing device of claim 12 wherein:

determining the quantization error comprises determining the quantization error associated with a first node in a first layer of the machine-learned model; and

determining one or more non-quantized weights associated with the weight comprises identifying a second node in the first layer, wherein the first and second nodes each have a connection to a third node in a second layer.

15. The computing device of claim 12 wherein allocating a fractional part of the quantization error to each of the associated non-quantized weights comprises dithering the quantization error within a node of the model.

16. The computing device of claim 15 further comprising dithering the quantization error within a node of the model based in part on a correlation of activity between nodes of the model.

17. The computing device of claim 16 wherein when there is an inverse correlation of activity between nodes of the model, the quantization error is dithered in an opposite direction.

18. The computing device of claim 11 wherein propagating the quantization error comprises:

locking the quantized weight; and

performing additional training iterations of the machine-learned model, wherein performing the additional training iterations comprises updating a non-quantized weight.

19. The computing device of claim 18 further comprising locking a number of the quantized best matching coefficient weights.

20. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations, the operations comprising:

obtaining a machine-learned model;

for one or more weights of the machine-learned model:

selecting a weight to be quantized;

quantizing the weight;

propagating at least a part of a quantization error to one or more non-quantized weights; and

providing a quantized machine-learned model.