Methods for encoding and decoding feature data, and decoder
Embodiments of the disclosure provide a method for encoding feature data, a method for decoding feature data, and a decoder. The method for encoding feature data includes the following. Feature data of multiple channels corresponding to a picture to-be-processed is obtained. Feature data of a reference channel in the feature data of the multiple channels is determined. Sorted feature data of the multiple channels is obtained by sorting, starting from the feature data of the reference channel, the feature data of the multiple channels in a descending order of similarity among the feature data of the multiple channels. The sorted feature data of the multiple channels is spliced to obtain a target feature frame sequence. The target feature frame sequence is encoded to generate a bitstream.
1 . A method for encoding feature data, comprising:
obtaining first feature data of a plurality of channels corresponding to a picture to-be-processed;
determining second feature data of a reference channel in the first feature data of the plurality of channels;
obtaining sorted feature data of the plurality of channels by sorting, starting from the second feature data of the reference channel, the first feature data of the plurality of channels in a descending order of similarity among the first feature data of the plurality of channels;
splicing the sorted feature data of the plurality of channels to obtain a target feature frame sequence; and
encoding the target feature frame sequence to generate a bitstream;
wherein determining the second feature data of the reference channel in the first feature data of the plurality of channels comprises:
determining, among the first feature data of the plurality of channels, feature data of a channel as the second feature data of the reference channel when a cumulative sum of values of the feature data of the channel satisfies a target condition.
2 . The method of claim 1 , wherein the target condition comprises:
the cumulative sum of the values of the feature data being greatest among the first feature data of the plurality of channels; or
the cumulative sum of the values of the feature data being smallest among the first feature data of the plurality of channels.
3 . The method of claim 1 , wherein after obtaining the sorted feature data of the plurality of channels by sorting, starting from the second feature data of the reference channel, the first feature data of the plurality of channels in the descending order of similarity among the first feature data of the plurality of channels, the method further comprises:
obtaining a channel-order correspondence between an original channel-order of the first feature data of the plurality of channels in the picture to-be-processed and an encoding channel-order of the sorted feature data of the plurality of channels; and
signalling the channel-order correspondence into the bitstream.
4 . The method of claim 3 , wherein the channel-order correspondence comprises:
when an amount of spliced time-domain frame is one, the original channel-order is an X-th channel, and the encoding channel-order corresponding to the original channel-order is an I-th channel; and
when an amount of the spliced time-domain frames is at least two, the original channel-order is the X-th channel, and the encoding channel-order corresponding to the original channel-order is an N_I-th channel.
5 . The method of claim 1 , wherein splicing the sorted feature data to obtain the target feature frame sequence comprises:
determining that an amount of spliced time-domain frames is greater than one, and splicing the sorted feature data of the plurality of channels according to a splicing strategy in time and spatial domains to obtain the target feature frame sequence.
6 . The method of claim 5 , wherein determining that the amount of spliced time-domain frames is greater than one, and splicing the sorted feature data of the plurality of channels according to the splicing strategy in time and spatial domains to obtain the target feature frame sequence comprises:
determining that the amount of spliced time-domain frames is greater than one, and splicing the sorted feature data of the plurality of channels according to the splicing strategy in time and spatial domains to obtain spliced feature data;
determining a product of a row amount of the spliced feature data, a column amount of the spliced feature data, and the amount of the spliced time-domain frames; and
determining that a channel amount of the first feature data of the plurality of channels is less than the product, and padding a region without a channel of feature data in a spliced frame to obtain the target feature frame sequence.
7 . The method of claim 5 , wherein determining that the amount of spliced time-domain frames is greater than one, and splicing the sorted feature data of the plurality of channels according to the splicing strategy in time and spatial domains comprises:
determining that the amount of spliced time-domain frames is greater than one; and
according to a splicing strategy of first time domain and then spatial domain in time and spatial domains:
in time domain, splicing at same positions in different frames according to a raster scan order; and
in spatial domain, splicing at adjacent positions according to the raster scan order or splicing at adjacent positions according to a zigzag scan order.
8 . The method of claim 5 , wherein determining that the amount of spliced time-domain frames is greater than one, and splicing the sorted feature data of the plurality of channels according to the splicing strategy in time and spatial domains comprises:
determining that the amount of spliced time-domain frames is greater than one, and according to a splicing strategy of first spatial domain and then time domain in time and spatial domains:
in spatial domain, splicing at adjacent positions according to a raster scan order, or splicing at adjacent positions according to a zigzag scan order; and
in time domain, splicing at same positions in different frames according to the raster scan order.
9 . The method of claim 1 , wherein splicing the sorted feature data to obtain the target feature frame sequence comprises:
determining that an amount of spliced time-domain frame is one, and splicing the sorted feature data according to a splicing strategy in spatial domain to obtain the target feature frame sequence.
10 . The method of claim 1 , wherein splicing the sorted feature data comprises:
splicing the sorted feature data according to a strategy of first padding and then splicing in spatial domain.
11 . The method of claim 10 , wherein splicing the sorted feature data according to the strategy of first padding and then splicing in spatial domain comprises:
padding each of the sorted feature data in spatial domain, and splicing padded feature data in spatial domain;
wherein there is a gap between feature data of adjacent channels after padding.
12 . The method of claim 11 , wherein after splicing padded feature data in spatial domain, the method further comprises:
signalling a height and a width corresponding to the padded feature data into the bitstream.
13 . The method of claim 1 , further comprising:
signalling a channel amount corresponding to the first feature data of the plurality of channels, a height corresponding to feature data of a channel, and a width corresponding to the feature data of the channel into the bitstream.
14 . The method of claim 4 , further comprising:
signalling an amount of spliced time-domain frame into the bitstream.
15 . The method of claim 1 , further comprising:
obtaining the picture to-be-processed; and
performing feature extraction on the picture to-be-processed with a neural network model to obtain the first feature data of the plurality of channels.
16 . A method for decoding feature data, comprising:
parsing a bitstream to obtain a reconstructed feature frame sequence and sorted feature data of a plurality of channels, wherein the sorted feature data of the plurality of channels is in an order obtained by sorting, starting from second feature data of a reference channel, first feature data of the plurality of channels in a descending order of similarity among the first feature data of the plurality of channels; and
sorting inversely the sorted feature data of the plurality of channels to obtain reconstructed feature data of the plurality of channels;
wherein the second feature data of the reference channel is determined from the first feature data of the plurality of channels when a cumulative sum of values of the second feature data of the reference channel satisfies a target condition.
17 . The method of claim 16 , further comprising:
parsing the bitstream to obtain a channel-order correspondence, a channel amount, an amount of spliced time-domain frames, a height corresponding to feature data of a channel, and a width corresponding to the feature data of the channel; and
determining a position of feature data of each channel in the reconstructed feature frame sequence according to the channel amount, an amount of spliced time-domain frames, the height corresponding to the feature data of the channel, and the width corresponding to the feature data of the channel;
wherein correspondingly, sorting inversely the sorted feature data of the plurality of channels to obtain the reconstructed feature data of the plurality of channels comprises:
determining an original channel-order of feature data at different positions in the reconstructed feature frame sequence based on the channel-order correspondence; and
sorting inversely the feature data at different positions in the reconstructed feature frame sequence based on the original channel-order to obtain the reconstructed feature data of the plurality of channels.
18 . The method of claim 17 , further comprising:
parsing the bitstream to obtain a height corresponding to padded feature data and a width corresponding to the padded feature data; wherein
wherein correspondingly, determining the position of feature data of each channel in the reconstructed feature frame sequence according to the channel amount, the amount of spliced time-domain frames, the height corresponding to the feature data of the channel and the width corresponding to the feature data of the channel comprises:
determining the position of feature data of each channel in the reconstructed feature frame sequence according to the channel amount, the amount of spliced time-domain frames, the height corresponding to the padded feature data, the width corresponding to the padded feature data, the height corresponding to the feature data of the channel and the width corresponding to the feature data of the channel.
19 . A decoder, comprising:
a processor and a memory storing a computer program which, when executed by the processor, causes the processor to:
parse a bitstream to obtain a reconstructed feature frame sequence and sorted feature data of a plurality of channels, wherein the sorted feature data of the plurality of channels is in an order obtained by sorting, starting from second feature data of a reference channel, first feature data of the plurality of channels in a descending order of similarity among the first feature data of the plurality of channels; and
sort inversely the sorted feature data of the plurality of channels to obtain reconstructed feature data of the plurality of channels;
wherein the second feature data of the reference channel is determined from the first feature data of the plurality of channels when a cumulative sum of values of the second feature data of the reference channel satisfies a target condition.