Neural network system and training method
Neural network systems and methods are provided. One method for processing a neural network includes, for at least one neural network layer that includes a plurality of weights, applying an offset function to each of a plurality of weight values in the plurality of weights to generate an offset weight value, and quantizing the offset weight values to form quantized offset weight values. The plurality of weights are pruned. One method for executing a neural network includes reading, from a memory, at least one neural network layer that includes quantized offset weight values and an offset value α, and performing a neural network layer operation on an input feature map, based on the quantized offset weight values and the offset value α, to generate an output feature map. The quantized offset weight values are signed integer numbers.
1 . A method comprising:
processing, by a computer, an initial neural network, said processing including, for at least one neural network layer that includes a plurality of weights W* of the initial neural network:
for each weight value of the plurality of weights W*:
applying an offset value α to the weight value to generate an offset weight value;
quantizing the offset weight value to form a quantized offset weight value of a plurality of quantized offset weight values W* q ; and
pruning the quantized offset weight based on a range of weight values to produce a pruned quantized weight value of a plurality of pruned quantized weight values W* qp ;
where said processing is based, at least in part, on a hardware requirement of an inference system to which a processed neural network is to be deployed; and
storing the plurality of pruned quantized offset weight values W* qp and the offset value α in a memory, the plurality of pruned quantized offset weight values forming at least part of the processed neural network,
where the processed neural network has reduced storage cost, inference latency, or power consumption compared to the initial neural network when deployed to the inference system.
2 . The method according to claim 1 , further comprising:
training the processed neural network based, at least in part, on the pruned quantized offset weight values.
3 . The method according to claim 2 , where said training includes iteratively quantizing the offset weight values based on learned quantization parameters.
4 . The method according to claim 1 , where said applying the offset value includes applying an offset value α to each weight value.
5 . The method according to claim 4 , where the offset value α is equal to a largest value of a range of weight values, the range of weight values having a central weight value.
6 . The method according to claim 1 , further comprising deploying the processed neural network to the inference system.
7 . The method according to claim 1 , where said applying the offset value α includes:
when the weight value is greater than the central weight value, subtracting the offset value α from the weight value; and
when the weight value is less than the central weight value, adding the offset value α to the weight value.
8 . The method according to claim 7 , where the central weight value is zero.
9 . The method according to claim 1 , where:
the plurality of weights W* are pruned by setting each weight value within a range of weight values to be pruned to zero; or
the plurality of quantized offset weights W* q are pruned by setting each quantized offset weight value within the range of weight values to be pruned to zero.
10 . The method according to claim 1 , where the plurality of weights W* are pruned based on a pruning mask.
11 . An apparatus for processing an initial neural network, comprising:
a processor configured to process the initial neural network including:
for at least one neural network layer that includes a plurality of weights W* of the initial neural network:
apply an offset value α to each of a plurality of weight values in the plurality of weights W* to generate an offset weight value; and
quantize the offset weight values to form quantized offset weight values W* q ;
prune the plurality of quantized offset weights to produce pruned quantized weight values W* qp ; and
store the pruned quantized offset weight values W* qp and the offset value α in a memory, the pruned quantized offset weight values forming at least part of the processed neural network;
the processor further configured to train the processed neural network based, at least in part, on the pruned quantized offset weight values W* qp , including iteratively quantize the offset weight values based on learned quantization parameters; where:
said processing of the initial neural network is based, at least in part, on a hardware requirement of the inference system; and
the processed neural network has reduced storage cost, inference latency, or power consumption compared to the initial neural network when deployed to the inference system.
12 . The apparatus according to claim 11 , where:
a range of weight values has a central weight value and a largest weight value; and
said apply the offset value α includes:
when the weight value is greater than the central weight value, subtract the offset value α from the weight value, and
when the weight value is less than the central weight value, add the offset value α to the weight value.
13 . The apparatus according to claim 11 , where:
the weight values W* are pruned by setting each weight value within a range of weight values to be pruned to zero; or
the quantized offset weight values W* q are pruned by setting each quantized offset weight value within the range of weight values to be pruned to zero.