Method and device for optimizing neural network model
A method of optimizing a neural network model, performed by a neural network model optimization device, may include removing an activation function included in one of a consecutive first block and second block, fusing batch normalization function and a fully connected layer included in a block included in the neural network model, and fusing the first block and the second block.
1 . A method of optimizing a neural network model, performed by a neural network model optimization device, the method comprising:
removing an activation function included in one of a consecutive first block and second block;
fusing batch normalization function and a fully connected layer included in a block included in the neural network model; and
fusing the first block and the second block,
wherein the fusing the first block and the second block comprises fusing the first block and the second block if an activation function is absent in the first block and an activation function is present in the second block,
wherein the fusing of the first block and the second block comprises fusing a first layer included in the first block and a second layer included in the second block, and
a third block generated by fusing the first block and the second block includes a third layer in which the first layer and the second layer are fused and an activation function included in the second block,
wherein the method further comprising:
reducing the number of hidden layers included in the neural network model based on a hidden layer order,
wherein the hidden layer order is numerical value assigned to each hidden layer with higher value assigned as it gets closer to input of the neural network model and lower value assigned as it gets closer to output of the neural network model,
wherein the number of hidden layers is adjusted such that the number of hidden layers corresponds to square of the hidden layer order.
2 . The method of claim 1 , wherein the fusing of the fully connected layer and the batch normalization comprises generating a fusion layer by fusing the fully connected layer and the batch normalization function.
3 . A neural network model optimization device comprising:
a memory configured to store a neural network model optimization program for optimizing a neural network model; and
a processor configured to control the memory,
wherein the processor is configured to,
remove an activation function included in one of a consecutive first block and second block,
fuse batch normalization function and a fully connected layer included in a block included in the neural network model, and
fuse the first block and the second block if an activation function is absent in the first block and an activation function is present in the second block,
wherein the processor is further configured to,
fuse a first layer included in the first block and a second layer included in the second block, and
a third block generated by fusing the first block and the second block includes a third layer in which the first layer and the second layer are fused and an activation function included in the second block,
wherein the processor is further configured to,
reduce the number of hidden layers included in the neural network model based on a hidden layer order,
wherein the hidden layer order is numerical value assigned to each hidden layer with higher value assigned as it gets closer to input of the neural network model and lower value assigned as it gets closer to output of the neural network model,
wherein the number of hidden layers is adjusted such that the number of hidden layers corresponds to square of the hidden layer order.