Feature map compression method and apparatus
A neural network-based image processing method and apparatus according to an embodiment of the present invention may: acquire a feature tensor from an input image by using a first neural network including a plurality of neural network layers; acquire a symbol tensor by performing quantization on the acquired feature tensor; and generate a bitstream by performing entropy encoding on the basis of the symbol tensor.
1 . A neural network-based image processing method, comprising:
obtaining a feature tensor from an input image using a first neural network including a plurality of neural network layers;
obtaining a symbol tensor by performing quantization on the obtained feature tensor; and
generating a bitstream by performing entropy encoding based on the symbol tensor,
wherein generating the bitstream comprises selectively performing the entropy encoding by applying skip coding to the symbol tensor.
2 . The method of claim 1 , wherein when the skip coding is applied to the symbol tensor, the entropy encoding is skipped on all or part of a plurality of symbols included in the symbol tensor.
3 . The method of claim 1 , wherein generating the bitstream further comprises:
determining whether to apply the skip coding to the symbol tensor.
4 . The method of claim 3 , wherein whether to apply the skip coding is determined based on a structure of the first neural network, a type of a first layer from which the feature tensor is output, a type of a second layer indicating a next layer of the first layer based on an encoding order, or a symbol included in the symbol tensor.
5 . The method of claim 4 , wherein when the type of the second layer is a summation layer, it is determined to apply the skip coding to the symbol tensor.
6 . The method of claim 4 , wherein when the type of the first layer is not an upsampling layer and the type of the second layer is a summation layer, it is determined to apply the skip coding to the symbol tensor.
7 . The method of claim 3 , wherein whether to apply the skip coding is determined using a second neural network including a plurality of neural network layers.
8 . The method of claim 1 , wherein generating the bitstream further comprises:
determining whether to apply the skip coding to each of a plurality of channels included in the symbol tensor.
9 . The method of claim 8 , wherein when it is determined that the skip coding is applied to a current channel among the plurality of channels included in the symbol tensor, entropy encoding is performed on a skip coding flag indicating that the skip coding is applied to the current channel.
10 . The method of claim 9 , wherein generating the bitstream further comprises:
when it is determined that the skip coding is not applied to the current channel, determining whether to apply the skip coding for each of a plurality of blocks included in the current channel.
11 . The method of claim 10 , wherein the plurality of blocks included in the current channel has a predefined fixed size.
12 . The method of claim 1 , wherein the feature tensor represents data expressed in floating-point, and the symbol tensor represents data expressed in fixed-point.
13 . The method of claim 1 , wherein the feature tensor is partitioned into a plurality of processing units based on a predefined partitioning structure, and
wherein the predefined partitioning structure includes at least one of an octree partitioning structure, a point unit partitioning structure, a channel unit partitioning structure, or a hierarchical partitioning structure.
14 . A neural network-based image processing device, comprising:
a processor configured to control the image processing device; and
a memory, coupled to the processor, configured to store data,
wherein the processor is configured to:
obtain a feature tensor from an input image using a first neural network including a plurality of neural network layers,
obtain a symbol tensor by performing quantization on the obtained feature tensor, and
generate a bitstream by performing entropy encoding based on the symbol tensor,
wherein to generate the bitstream, the processor is configured to selectively perform the entropy encoding by applying skip coding to the symbol tensor.