Electronic device and control method therefor
An electronic device and a control method therefor are provided. The electronic device may comprise: a memory for storing at least one instruction; and a processor connected to the memory so as to control the electronic device, wherein the processor: by executing the at least one instruction, appends a second layer including a learnable function to a first layer in an artificial neural network including a plurality of layers; updates a parameter value included in the second layer by learning of the artificial neural network; acquires a function value by inputting the updated parameter value to the learnable function; and eliminates at least one channel among a plurality of channels included in the first layer on the basis of the acquired function value so as to achieve update to a third layer.
1. An electronic device comprising:
a memory configured to store at least one instruction; and
a processor connected to the memory and configured to control the electronic device,
wherein the processor, by executing the at least one instruction, is further configured to:
append a second layer including a trainable function to a first layer in an artificial neural network including a plurality of layers, wherein the trainable function is obtained by adding a first function outputting 0 or 1 to a differentiable second function and the differentiable second function is obtained by multiplying a differentiable function by a function having a predetermined gradient,
obtain a first function value by inputting parameter value of the second layer to the trainable function,
obtain output data of the second layer by multiplying output data of the first layer by the first function value channel-wise,
generate a loss function based on the output data of the first layer and the output data of the second layer,
obtain parameter value that outputs minimum function value of the loss function,
update the parameter value of the second layer to the obtained parameter value that outputs the minimum function value of the loss function,
obtain a second function value by inputting the updated parameter value to the trainable function, and
update the first layer to a third layer by eliminating at least one channel among a plurality of channels included in the first layer based on the obtained second function value,
wherein the loss function is a function of adding an additional loss function indicating a size or computational complexity of a layer of the artificial neural network to be obtained after compression of the artificial neural network to a loss function using a sum of output values of the trainable function.
2. The electronic device of claim 1 , wherein the processor is further configured to:
based on the updated parameter value being negative, obtain a function value of 0 by inputting a negative parameter value to the trainable function, and
based on the updated parameter value being positive, obtain a function value of 1 by inputting a positive parameter value to the trainable function.
3. The electronic device of claim 2 , wherein the processor is further configured to:
based on the obtained function value of 0, eliminate a channel of the first layer corresponding to the parameter value input to the trainable function; and
based on the obtained function value of 1, maintain the channel of the first layer corresponding to the parameter value input to the trainable function.
4. The electronic device of claim 1 , wherein the processor is further configured to update the first layer to a third layer by eliminating the second layer and eliminating the channel of the first layer based on an obtained function value.
5. The electronic device of claim 1 , wherein the processor is further configured to:
obtain a function value by inputting the updated parameter value to the trainable function;
change a weight of a first kernel of the first layer based on the obtained function value; and
update the first kernel of the first layer to a second kernel including the changed weight.
6. The electronic device of claim 5 , wherein the processor is further configured to:
based on the obtained function value being 0, change a weight of the first kernel of the first layer corresponding to the parameter value input to the trainable function to 0; and
based on the obtained function value being 1, maintain the weight of the first kernel of the first layer corresponding to the parameter value input to the trainable function.
7. A control method of an electronic device, the method comprising:
appending a second layer including a trainable function to a first layer in an artificial neural network including a plurality of layers, wherein the trainable function is obtained by adding a first function outputting 0 or 1 to a differentiable second function and the differentiable second function is obtained by multiplying a differentiable function by a function having a predetermined gradient;
obtaining a first function value by inputting parameter value of the second layer to the trainable function;
obtaining output data of the second layer by multiplying output data of the first layer by the first function value channel-wise;
generating a loss function based on the output data of the first layer and the output data of the second layer;
obtaining parameter value that outputs minimum function value of the loss function;
updating the parameter value of the second layer to the obtained parameter value that outputs the minimum function value of the loss function;
obtaining a second function value by inputting the updated parameter value to the trainable function; and
updating the first layer to a third layer by eliminating at least one channel among a plurality of channels included in the first layer based on the obtained second function value,
wherein the loss function is a function of adding an additional loss function indicating a size or computational complexity of a layer of the artificial neural network to be obtained after compression of the artificial neural network to a loss function using a sum of output values of the trainable function.