Granted Patent B2

US 12,354,312 · App. 17/983,843 · Granted Jul 8, 2025

Inventors: Quockhanh Dinh (Suwon-si, KR); Kwangpyo Choi (Suwon-si, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06T9/002G06T3/4046

Abstract

Claims (49)

1. An image decoding method using artificial intelligence (AI), the image decoding method comprising:

obtaining, from a bitstream, a current optical flow and correction data which are generated based on a current predicted image and a current image;

obtaining the current predicted image based on a previous reconstructed image and the current optical flow;

obtaining feature data of the current predicted image by applying the current predicted image to a neural network-based predicted image encoder; and

obtaining a current reconstructed image corresponding to the current image by applying the correction data and the feature data of the current predicted image to a neural network-based image decoder.

2. The image decoding method of claim 1 , wherein the feature data of the current predicted image is generated by the predicted image encoder based on a common feature included in the current image and the current predicted image, and

wherein the common feature does not include high-frequency errors included in the current predicted image.

3. The image decoding method of claim 1 , wherein the correction data is generated based on a feature which is included in the current image and which is not included in the current predicted image, by applying the current image and the current predicted image to a neural network-based image encoder.

4. The image decoding method of claim 1 , wherein the current optical flow is obtained by applying feature data of the current optical flow obtained from the bitstream to a neural network-based optical flow decoder.

5. The image decoding method of claim 1 , further comprising:

obtaining a first modified predicted image by applying a first weight map to the current predicted image; and

obtaining a second modified predicted image by applying a second weight map to the current predicted image,

wherein the obtaining of the feature data of the current predicted image comprises obtaining feature data of the first modified predicted image by applying the first modified predicted image to the predicted image encoder, and

wherein the obtaining of the current reconstructed image comprises:

applying the correction data and the feature data of the first modified predicted image to the image decoder; and

obtaining the current reconstructed image by combining an image obtained from the image decoder with the second modified predicted image.

6. The image decoding method of claim 5 , wherein the first weight map and the second weight map are obtained by applying feature data of the current optical flow obtained from the bitstream to a neural network-based optical flow decoder.

7. The image decoding method of claim 5 , wherein the first weight map and the second weight map are obtained by applying feature data of the current optical flow to a neural network-based weight decoder.

8. The image decoding method of claim 5 , wherein the first weight map and the second weight map are obtained by applying feature data of a weight map obtained from the bitstream to a neural network-based weight decoder.

9. The image decoding method of claim 1 , further comprising determining a decoding mode of the current image, based on information obtained from the bitstream,

wherein based on the decoding mode being a first mode, the current reconstructed image is obtained by applying the correction data and the feature data of the current predicted image to the image decoder, and

wherein based on the decoding mode being a second mode, the current reconstructed image is obtained by combining an image obtained by applying the correction data to the image decoder with the current predicted image.

10. The image decoding method of claim 9 , wherein the information obtained from the bitstream comprises at least one of a quantization parameter, a weight used to calculate a rate-distortion value, or information indicating the decoding mode.

11. A non-transitory computer-readable recording medium having recorded thereon instructions which, when executed by at least one processor, performs the method of claim 1 .

12. An image encoding method using artificial intelligence (AI), the image encoding method comprising:

obtaining a current optical flow using a current image and a previous reconstructed image;

generating a current predicted image based on the previous reconstructed image and the current optical flow;

obtaining correction data by applying the current image and the current predicted image to a neural network-based image encoder; and

generating a bitstream including feature data of the current optical flow and the correction data.

13. The image encoding method of claim 12 , further comprising determining an encoding mode corresponding to the current image,

wherein based on the encoding mode being a first mode, the correction data is obtained by applying the current image and the current predicted image to the image encoder, and

wherein based on the encoding mode being a second mode, the correction data is obtained by applying a residual image obtained based on the current image and the current predicted image to the image encoder.

14. The image encoding method of claim 13 , wherein a mode corresponding to a lower rate-distortion value from among a rate-distortion value of the first mode and a rate-distortion value of second mode is determined as the encoding mode.

15. The image encoding method of claim 13 , wherein the encoding mode is determined as the first mode or the second mode according to a result of comparing a bitrate of a bitstream obtained according to the first mode and a bitrate of a bitstream obtained according to the second mode.

16. The image encoding method of claim 13 , wherein the encoding mode is determined as the first mode or the second mode according to a result of comparing a pre-determined value with a quantization parameter or a weight used to calculate a rate-distortion value.

17. The image encoding method of claim 12 , wherein the obtaining of the current optical flow comprises:

obtaining feature data of the current optical flow by applying the current image and the previous reconstructed image to a neural network-based optical flow encoder; and

applying the feature data of the current optical flow to a neural network-based optical flow decoder.

18. The image encoding method of claim 12 , further comprising:

obtaining a first modified predicted image by applying a first weight map to the current predicted image; and

obtaining a second modified predicted image by applying a second weight map to the current predicted image, and

wherein the correction data is obtained based on a residual image obtained based on the second modified predicted image and the current image, and the first modified predicted image being applied to the image encoder.

19. A non-transitory computer-readable recording medium having recorded thereon a instructions which, when executed by at least one processor, performs the method of claim 12 .

20. An image decoding apparatus using artificial intelligence (AI), the image decoding apparatus comprising:

at least one processor configured to:

obtain, from a bitstream, a current optical flow and correction data which are generated based on a current predicted image and a current image,

obtain the current predicted image based on a previous reconstructed image and the current optical flow,

obtain feature data of the current predicted image by applying the current predicted image to a neural network-based predicted image encoder, and

obtain a current reconstructed image corresponding to the current image by applying the correction data and the feature data of the current predicted image to a neural network-based image decoder.

Assignments (1)

Priority Claims (3)

Continuity (2)

References Cited (38)