Granted Patent B2

US 12,581,093 · App. 18/426,793 · Granted Mar 17, 2026

Inventors: Je Won Kang (Seoul, KR); Jung Kyung Lee (Seoul, KR); Seung Wook Park (Yongin-si, KR); Jin Heo (Yongin-si, KR)

Assignees: HYUNDAI MOTOR COMPANY; KIA CORPORATION; ETHAN UNIVERSITY—INDUSTRY COLLABORATION FOUNDATION

H04N19/154H04N19/172H04N19/46

Abstract

Claims (51)

1 . A method for improving video quality of a reconstructed frame, performed by a video encoding apparatus, the method comprising:

encoding an original frame into a bitstream and generating a reconstructed frame of the original frame;

inputting the reconstructed frame into a deep learning-based improvement model to generate an output;

generating a first residual frame based on the output of the deep learning-based improvement model;

inputting the first residual frame into a linear model to generate a second residual frame, wherein the linear model includes parameters representing a linear relation between the first residual frame and the second residual frame; and

generating an improved reconstructed frame for the reconstructed frame by adding the second residual frame and the reconstructed frame.

2 . The method of claim 1 , wherein the first residual frame approximates an original residual frame, and

wherein the original residual frame is a difference between the original frame and the reconstructed frame.

3 . The method of claim 2 , wherein the deep learning-based improvement model is a deep learning model including multiple layers and is trained using a loss function based on a difference between the first residual frame and the original residual frame.

4 . The method of claim 1 , wherein the reconstructed frame is one of reconstructed signals stored in a decoded picture buffer (DPB), an output of a deblocking filter, an output of a sample adaptive offset (SAO) filter, or an output of an adaptive loop filter (ALF), and

wherein the reconstructed signals are a sum of predicted signals and inversely transformed signals.

5 . The method of claim 1 , further comprising:

signaling a flag indicating whether to apply the method of improving video quality.

6 . The method of claim 2 , further comprising:

estimating the parameters of the linear model by using a linear least square equation based on pixel values in the first residual frame and pixel values in the original residual frame.

7 . The method of claim 1 , further comprising:

deriving the parameters of the linear model; and

encoding the parameters of the linear model into the bitstream to transmit the parameters of the linear model to a video decoding apparatus.

8 . The method of claim 1 , further comprising:

using preset values as the parameters of the linear model, wherein the preset values are statistically frequently occurring values based on encoding and decoding information.

9 . The method of claim 1 , further comprising:

signaling an index indicating parameter values of the linear model to a video decoding apparatus, wherein the parameter values of the linear model and corresponding indices are set in advance.

10 . A method for improving video quality of a reconstructed frame, performed by a video decoding apparatus, the method comprising:

generating a reconstructed frame of an original frame from a bitstream;

inputting the reconstructed frame into a deep learning-based improvement model to generate an output;

generating a first residual frame based on the output of the deep learning-based improvement model;

inputting the first residual frame into a linear model to generate a second residual frame, wherein the linear model includes parameters representing a linear relation between the first residual frame and the second residual frame; and

generating an improved reconstructed frame for the reconstructed frame by adding the second residual frame and the reconstructed frame.

11 . The method of claim 10 , wherein the first residual frame approximates an original residual frame, and

wherein the original residual frame is a difference between the original frame and the reconstructed frame.

12 . The method of claim 11 , wherein the deep learning-based improvement model is a deep learning model including multiple layers and is trained using a loss function based on a difference between the first residual frame and the original residual frame.

13 . The method of claim 11 , further comprising:

estimating the parameters of the linear model by using a linear least square equation based on pixel values in the first residual frame and pixel values in the original residual frame.

14 . The method of claim 10 , wherein the reconstructed frame is one of reconstructed signals stored in a decoded picture buffer (DPB), an output of a deblocking filter, an output of a sample adaptive offset (SAO) filter, or an output of an adaptive loop filter (ALF), and

wherein the reconstructed signals are a sum of predicted signals and inversely transformed signals.

15 . The method of claim 10 , further comprising:

decoding a flag indicating whether to apply the method of improving video quality.

16 . The method of claim 10 , further comprising:

decoding the parameters of the linear model from the bitstream.

17 . The method of claim 10 , further comprising:

using preset values as the parameters of the linear model, wherein the preset values are statistically frequently occurring values based on encoding and decoding information.

18 . The method of claim 10 , further comprising:

decoding an index indicating parameter values of the linear model from a bitstream, wherein the parameter values of the linear model and corresponding indices are set in advance.

19 . A method for providing a video decoding apparatus with video data, the method comprising:

encoding the video data into a bitstream; and

transmitting the bitstream to the video decoding device, wherein the encoding of the video data comprises:

encoding an original frame into a bitstream and generating a reconstructed frame of the original frame;

inputting the reconstructed frame into a deep learning-based improvement model to generate an output;

generating a first residual frame based on the output of the deep learning-based improvement model;

inputting the first residual frame into a linear model to generate a second residual frame, wherein the linear model includes parameters representing a linear relation between the first residual frame and the second residual frame; and

generating an improved reconstructed frame for the reconstructed frame by adding the second residual frame and the reconstructed frame.

Assignments (1)

Priority Claims (2)

Continuity (2)

References Cited (29)