Quantizing autoencoders in a neural network
The performance of a neural network is improved by applying quantization to data at various points in the network. In an embodiment, a neural network includes two paths. A quantization is applied to each path, such that when an output from each path is combined, further quantization is not required. In an embodiment, the neural network is an autoencoder that includes at least one skip connection. In an embodiment, the system determines a set of quantization parameters based on the characteristics of the data in the primary path and in the skip connection, such that both network paths produce output data in the same fixed point format. As a result, the data from both network paths can be combined without requiring an additional quantization.
1 . A computer-implemented method, comprising:
identifying a first range of values input to a first path of one or more neural networks;
identifying a second range of values input to a second path of the one or more neural networks;
selecting, based at least in part on the first range of values and the second range of values, a set of quantization parameters to be used by both of the first path of the one or more neural networks, and the second path of the one or more neural networks; and
computing output of the first path of the one or more neural networks and the second path of the one or more neural networks based, at least in part, on the selected set of quantization parameters.
2 . The computer-implemented method of claim 1 ,
further comprising adjusting the selected set of quantization parameters based, at least in part, on variation in the first and the second ranges of values.
3 . The computer-implemented method of claim 1 , further comprising normalizing the first range of values and the second range of values.
4 . The computer-implemented method of claim 1 , further comprising using a machine learning model to determine one or more quantization ranges to use as one or more of the selected set of quantization parameters.
5 . The computer-implemented method of claim 1 , further comprising determining whether outputs of the first path and the second path are compatible with a third path of the one or more neural networks.
6 . The computer-implemented method of claim 1 , further comprising concatenating output of one or more paths of the one or more neural networks with one or more other paths.
7 . A system, comprising:
one or more processors to respond to one or more application programming interface (API) calls by at least causing a parallel processing unit to:
identify a first range of values to be input to a first path of one or more neural networks and a second range of values to be input to a second path of the one or more neural networks;
determine, based at least in part on the first range of values and the second range of values, a set of quantization parameters to be shared between the first path of the one or more neural networks and the second path of the one or more neural networks; and
cause output of the first path of the one or more neural networks and the second path of the one or more neural networks to both be computed using the shared set of quantization parameters.
8 . The system of claim 7 , wherein the system adjusts the shared set of quantization parameters based, at least in part, on variations in the first range of values and the second range of values.
9 . The system of claim 7 , wherein the system normalizes values of the first and the second ranges of values.
10 . The system of claim 7 , wherein the system determines the shared set of quantization parameters using a machine learning model whose output comprises one or more quantization ranges.
11 . The system of claim 7 , wherein the system quantizes outputs within portions of at least one of the first path or the second path using the shared set of quantization parameters.
12 . The system of claim 7 , wherein the set of quantization parameters comprise ranges and resolutions of integer values.
13 . One or more processors, comprising:
circuitry to:
identify a first range of values input to a first path of one or more neural networks;
identify a second range of values input to a second path of the one or more neural networks;
select, based at least in part on the first range of values and the second range of values, a set of quantization parameters to be shared between the first path of the one or more neural networks and the second path of the one or more neural networks; and
cause output of the first path of the one or more neural networks and the second path of the one or more neural networks to both be computed using the selected shared set of quantization parameters.
14 . The one or more processors of claim 13 , wherein the circuitry is further to select the shared set of quantization parameters based, at least in part, on variations in the first range of values and the second range of values.
15 . The one or more processors of claim 13 , wherein the circuitry is further to perform normalization on the first range of values and the second range of values prior to selecting the set of quantization parameters to be shared.
16 . The one or more processors of claim 13 , wherein the circuitry is further to select the shared set of quantization parameters using a machine learning model whose output comprises one or more quantization ranges for one or more neural network paths.
17 . The one or more processors of claim 13 , wherein the circuitry is further to select the shared set of quantization parameters based, at least in part, on whether outputs of the first path and the second path are compatible with a third path of the one or more neural networks.
18 . The one or more processors of claim 13 , wherein the circuitry is to concatenate output of one or more paths of the one or more neural networks with output of one or more other paths of the one or more neural networks.
19 . The one or more processors of claim 13 , wherein an application programming interface (API) is to schedule operations on a parallel processing unit (“PPU”) to compute output of the first path of the one or more neural networks and the second path of the one or more neural networks.
20 . The one or more processors of claim 13 , wherein the set of quantization parameters comprises a range and resolution applied to integer values.