METHOD FOR AUTOMATIC HYBRID QUANTIZATION OF DEEP ARTIFICIAL NEURAL NETWORKS
A method includes, for each floating-point layer in a set of floating-point layers: calculating a set of input activations and a set of output activations of the floating-point layer; converting the floating-point layer to a low-bit-width layer; calculating a set of low-bit-width output activations based on the set of input activations; and calculating a per-layer deviation statistic of the low-bit-width layer. The method also includes ordering the set of low-bit-width layers based on the per-layer deviation statistic of each low-bit-width layer. The method additionally includes, while a loss-of-accuracy threshold exceeds the accuracy of the quantized network: converting a floating-point layer represented by the low-bit-width layer to a high-bit-width layer; replacing the low-bit-width layer with the high-bit-width layer in the quantized network; updating the accuracy of the quantized network; and, in response to the accuracy of the quantized network exceeding the loss-of-accuracy threshold, returning the quantized network.
1 . A method for quantizing an artificial neural network, the method comprising:
converting a set of floating-point layers in a floating-point network to a set of low-bit-width layers;
for each low-bit-width layer in the set of low-bit-width layers:
calculating a set of low-bit-width output activations of the low-bit-width layer based on a set of example input activations;
calculating a per-layer deviation statistic of the low-bit-width layer based on a set of error metrics between the set of low-bit-width output activations of the low-bit-width layer and a set of example output activations of a floating-point layer corresponding to the low-bit-width layer, each error metric in the set of error metrics characterized by a difference between an example output activation in the set of example output activations and a corresponding low-bit-width output activation in the set of low bit-width activations; and
sorting the low-bit-width layer in the set of low-bit-width layers based on the per-layer deviation statistic as a set of ordered low-bit-width layers;
generating a quantized network representing the floating-point network and comprising the set of low-bit-width layers; and
in response to an accuracy of the quantized network falling below a loss-of-accuracy threshold, sequentially, according to the set of ordered low-bit-width layers:
converting a floating-point layer, represented by a low-bit-width layer in the set of ordered low-bit-width layers, to a high-bit-width layer; and
replacing the low-bit-width layer with the high-bit-width layer in the quantized network.
2 . The method of claim 1 , further comprising calculating the set of example output activations of the floating-point layer corresponding to the low-bit-width layer based on the set of example input activations.
3 . The inventions as shown and/or described herein.