Method and apparatus for training a quantized classifier
A computer-implemented method for training a classifier is disclosed. The classifier is designed to determine an output (y) for an input data point (x). The output (y) characterizes a classification of the input data point (x). The classifier comprises a multiplicity of weights on the basis of which the output (y) is determined. At least one weight of the multiplicity of weights is quantized to a predefined first number of first values. Each two consecutive first values differ by a distance value. The distance value is also adjusted for training and the multiplicity of weights is not adjusted.
1 . A method for training a classifier, the method being computer-implemented, the classifier being configured to determine an output for an input data point based on a multiplicity of weights of the classifier, the output characterizing a classification of the input data point, the method comprising:
pre-training at least a portion of the multiplicity of weights with a second quantization type in which the at least the portion of the multiplicity of weights are quantized with a predefined second quantization resolution and a predefined second quantization step size; and
training the classifier by (i) quantizing at least one weight of the multiplicity of weights with a first quantization type in which the at least one weight is quantized with a predefined first quantization resolution and a first quantization step size and (ii) training the classifier by adjusting the first quantization step size without adjusting the multiplicity of weights.
2 . The method according to claim 1 further comprising:
storing the multiplicity of weights with the second quantization type,
wherein the predefined second quantization resolution is higher than the predefined first quantization resolution.
3 . The method according to claim 1 , wherein the predefined second quantization resolution differs from the predefined first quantization resolution.
4 . The method according to claim 3 further comprising:
determining a respective desired output for a respective input data point using the classifier based on the multiplicity of weights determined during the pre-training.
5 . The method according to claim 1 , the training further comprising:
determining, with the classifier, a respective output for a respective input data point from a dataset of input data; and
adjusting the first quantization step size based on a difference between the determined respective output and a respective desired output for the respective input data point from the dataset of input data.
6 . The method according to claim 1 , wherein the method is carried out by executing a computer program.
7 . A method for operating a classifier, the method being computer-implemented, the classifier being configured to determine an output for an input data point based on a multiplicity of weights of the classifier, the output characterizing a classification of the input data point, the method comprising:
pre-training at least a portion of the multiplicity of weights with a second quantization type in which the at least the portion of the multiplicity of weights are quantized to a predefined second number of values;
training the classifier by (i) quantizing at least one weight of the multiplicity of weights with a first quantization type in which the at least one weight is quantized with a predefined first quantization resolution and a first quantization step size and (ii) training the classifier by adjusting the first quantization step size without adjusting the multiplicity of weights;
determining a first input data point;
selecting a quantization type after the training, the selected quantization type being one of the first quantization type or the second quantization type;
determine a first output for the first input data point using the classifier using the multiplicity of weights with the selected quantization type; and
activating at least one of (i) an actuator and (ii) a display device based on the output.
8 . A non-transitory machine-readable storage medium configured to store on a computer program for operating a classifier, the classifier being configured to determine an output for an input data point based on a multiplicity of weights of the classifier, the output characterizing a classification of the input data point, the computer program, when executed by a computer, causing the computer to:
receive an input data point;
select a quantization type, the selected quantization type being one of a first quantization type or a second quantization type;
determine an output for the input data point using the classifier using the multiplicity of weights with the selected quantization type; and
activate at least one of (i) an actuator and (ii) a display device based on the output,
wherein, prior to the selection of the quantization type, at least a portion of the multiplicity of weights were pre-trained with the second quantization type in which the at least the portion of the multiplicity of weights are quantized with a predefined second quantization resolution and a predefined second quantization step size, and
wherein, prior to the selection of the quantization type, the classifier was trained by (i) quantizing at least one weight of the multiplicity of weights with a first quantization type in which the at least one weight is quantized with a predefined first quantization resolution and a first quantization step size and (ii) training the classifier by adjusting the first quantization step size without adjusting the multiplicity of weights.