Learning method of neural codec, encoding and decoding method of neural codec

1 . An encoding method of a neural codec performed by at least one processor, the method comprising:

encoding input data to a first latent vector of a quantized representation;

generating, after encoding the input data, a bitstream corresponding to the input data by channel-wise grouping and entropy encoding the first latent vector; and

transmitting the bitstream,

wherein the generating the bitstream further comprises:

grouping channels of the first latent vector;

predicting a distribution of probability mass functions (PMFs) of frequencies of symbols respectively corresponding to the grouped channels of the first latent vector by an entropy layer of the neural codec; and

generating the bitstream by entropy encoding the channels of the first latent vector based on the distribution of PMFs.

2 . The method of claim 1 , wherein the grouping of the channels of the first latent vector further comprises:

sequentially grouping the channels of the first latent vector according to an adjacent position.

3 . The method of claim 1 , wherein the neural codec comprises the entropy layer, and

the predicting of the distribution of PMFs further comprises estimating the distribution of PMFs respectively corresponding to the grouped channels of the first latent vector in the entropy layer.

4 . The method of claim 1 , wherein the generating of the bitstream by entropy encoding the channels of the first latent vector further comprises:

performing the entropy encoding by grouping channels having a predetermined similarity of the distribution of the PMFs among the grouped channels of the first latent vector.

5 . The method of claim 1 , wherein the bitstream is generated by entropy encoding the grouped channels of the first latent vector using a look-up table (LUT) generated based on the distribution of PMFs.

6 . The method of claim 5 , wherein the LUT is generated by assigning, using Huffman coding, bits corresponding to the frequencies of the symbols to nodes of a binary tree and generating bit information corresponding to the nodes in which the symbols are positioned in the binary tree.

7 . The method of claim 1 , wherein the generating of the bitstream by entropy encoding the channels of the first latent vector further comprises:

generating the bitstream by differently adjusting a length of code generated according to the frequencies of the symbols by the entropy encoding.

8 . The method of claim 1 , wherein the encoding to the first latent vector further comprises:

generating an initial latent vector by encoding the input data; and

generating the first latent vector by quantizing the initial latent vector.

9 . The method of claim 8 , wherein the generating of the first latent vector comprises:

clamping the initial latent vector in a predetermined range; and

generating the first latent vector by quantizing the clamped initial latent vector.

10 . The method of claim 1 , wherein the neural codec is trained by a first loss between the input data and restored input data and a second loss based on a latency of the first latent vector.

11 . The method of claim 10 , wherein the neural codec is trained by adjusting at least one of a first weight corresponding to the first loss and a second weight corresponding to the second loss.

12 . A decoding method of a neural codec performed by at least one processor, the method comprising:

receiving a bitstream transmitted by a neural encoder;

generating a second latent vector by entropy decoding the bitstream using a preset look-up table (LUT); and

restoring the input data based on the second latent vector,

wherein the preset LUT is generated based on channels of a first latent vector grouped by a distribution of symbols respectively corresponding to channels of the first latent vector of a quantized representation in which the input data is encoded by the neural encoder,

wherein the bitstream is generated by entropy encoding grouped channels of the first latent vector using an LUT generated based on a distribution of probability mass functions (PMFs) for frequencies of the symbols.

13 . The method of claim 12 , wherein the distribution of symbols further comprises

a distribution of symbol histograms based on the symbols.

14 . The method of claim 12 , wherein the LUT is generated by assigning, using Huffman coding, bits corresponding to the frequencies of the symbols to nodes of a binary tree and generating bit information corresponding to the nodes in which the symbols are positioned in the binary tree.

15 . A learning method of a neural codec comprising a neural encoder and a neural decoder, the method comprising:

encoding input data to a first latent vector of a quantized representation by the neural encoder;

generating, after encoding the input data, a bitstream corresponding to the input data by channel-wise grouping and entropy encoding the first latent vector;

obtaining a second latent vector by entropy decoding the bitstream;

restoring the second latent vector to the input data by the neural decoder; and

learning the neural codec based on a difference between the input data and the restored input data,

wherein the generating of the bitstream comprises:

channel-wise grouping channels of the first latent vector;

predicting a distribution of probability mass functions (PMFs) indicating frequencies of symbols respectively corresponding to the grouped channels of the first latent vector by an entropy layer of the neural codec; and

generating the bitstream by entropy encoding the channels of the first latent vector based on the distribution of PMFs.

16 . The method of claim 15 , wherein the generating of the bitstream by entropy encoding the channels of the first latent vector comprises:

generating a look-up table (LUT) based on the distribution of PMFs; and

generating the bitstream by entropy encoding the grouped channels of the first latent vector using the LUT.

17 . The method of claim 16 , wherein the generating of the LUT comprises:

assigning, by using Huffman coding, bits matching the frequencies of the symbols to nodes of a binary tree; and

generating the LUT by bit information corresponding to the nodes in which the symbols are positioned in the binary tree.