Image decoding apparatus and method and image encoding apparatus and method
An image decoding method and apparatus obtain intra prediction feature data of a current block from a bitstream, determine an intra flow indicating a reference pixel of a current pixel in the current block, by applying the intra prediction feature data, neighboring pixels of the current block, and coding context information of the current block to a neural network, obtain a predicted pixel of the current pixel based on the intra flow of the current block, and reconstruct the current block based on the predicted pixel.
1. A neural network based image decoding method comprising:
obtaining intra prediction feature data of a current block from a bitstream;
obtaining intra flow information indicating a reference pixel corresponding to each pixel included in the current block, based on the intra prediction feature data, first neighboring pixels of the current block, and coding context information of the current block;
obtaining a predicted pixel of the current block based on the intra flow information of the current block; and
reconstructing the current block based on the predicted pixel.
2. The image decoding method of claim 1 , further comprising:
applying the intra prediction feature data, the first neighboring pixels, and the coding context information of the current block to the neural network;
obtaining, from the neural network, the intra flow information along with weight information about the reference pixel indicated by the intra flow information and a plurality of second neighboring pixels adjacent to the reference pixel; and
obtaining the predicted pixel based on a weighted average obtained based on the weight information, the reference pixel, and the plurality of second neighboring pixels.
3. The image decoding method of claim 1 , wherein the neural network outputs the intra flow information based on the intra prediction feature data, the first neighboring pixels of the current block, and the coding context information provided as an input into the neural network.
4. The image decoding method of claim 1 , wherein the intra flow information comprises one of a first vector comprising an x component and a y component, a second vector comprising a size component and an angle component, or an angle with respect to the reference pixel indicated by the each pixel.
5. The image decoding method of claim 1 , wherein the intra prediction feature data is obtained through neural network-based encoding.
6. The image decoding method of claim 1 , wherein the coding context information comprises at least one of a quantization parameter of the current block, a temporal layer of the current block, a split structure of the current block, a split structure of the first neighboring pixels, or a transformation type of the first neighboring pixels.
7. The image decoding method of claim 1 , wherein the coding context information is obtained through neural network-based encoding.
8. A neural network based image encoding method comprising:
obtaining, based on an original pixel of a current block, first neighboring pixels of the current block, and coding context information of the current block, intra prediction feature data of the current block and intra flow information indicating a reference pixel corresponding to each of the original pixel of the current block;
obtaining a predicted pixel of the original pixel based on the intra flow information of the current block; and
encoding the intra prediction feature data by reconstructing the current block based on the predicted pixel.
9. The image encoding method of claim 8 , further comprising:
applying the each of the original pixel of the current block, the first neighboring pixels of the current block, and the coding context information of the current block to the neural network;
obtaining, from the neural network, the intra prediction feature data of the current block, the intra flow information, and weight information about the reference pixel indicated by the intra flow information and a plurality of second neighboring pixels adjacent to the reference pixel; and
obtaining the predicted pixel based on a weighted average obtained based on the weight information, the reference pixel, and the plurality of second neighboring pixels.
10. The image encoding method of claim 8 , further comprises:
outputting the intra prediction feature data by inputting the original pixel, the reference pixel, and the coding context information into an encoding neural network, and
outputting the intra flow information by inputting the intra prediction feature data, the reference pixel, and the coding context information into a decoding neural network.
11. The image encoding method of claim 9 , wherein the decoding neural network further outputs the weight information.
12. The image encoding method of claim 8 , wherein the intra flow information is represented by one of a vector including an x component and a y component, a vector including a size component and an angle component, or an angle with respect to the reference pixel indicated by the each of the original pixel.
13. The image encoding method of claim 8 , wherein the coding context information comprises at least one of a quantization parameter of the current block, a temporal layer of the current block, a split structure of the current block, a split structure of the first neighboring pixels, or a transformation type of the first neighboring pixels.
14. A neural network based image decoding apparatus comprising:
a memory storing one or more instructions; and
at least one processor configured to operate according to the one or more instructions to:
obtain intra prediction feature data of a current block from a bitstream,
obtain intra flow information indicating a reference pixel corresponding to each pixel included in the current block, based on the intra prediction feature data, first neighboring pixels of the current block, and coding context information of the current block,
obtain a predicted pixel of the current block based on the intra flow information of the current block, and
reconstruct the current block based on the predicted pixel.
15. A neural network based image decoding method comprising:
obtaining, based on coding context information and first neighboring pixels of a current block, intra flow information indicating a reference pixel corresponding to each pixel included in the current block and weight information of the reference pixel and a plurality of second neighboring pixels adjacent to the reference pixel;
obtaining a predicted pixel of the current block based on the intra flow information and the weight information; and
reconstructing the current block based on the predicted pixel.