IP Library Granted Patent US 12,081,746
Granted Patent B2
US 12,081,746 · App. 17/569,108 · Granted Sep 3, 2024

Methods of simplification of temporal filtering

Inventors: Anish Mahendra Tamse (Seoul, KR); Xiang Li (Saratoga, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/117H04N19/124H04N19/132H04N19/172H04N19/31
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,081,746
App. No.
17/569,108
Granted
Sep 3, 2024
Kind
B2
Abstract

Systems and methods for implementing temporal filtering are provided. A method includes: applying a temporal filter to a current picture and encoding the current picture after the temporal filter is applied. The applying includes obtaining an exponential function; obtaining a weight of at least one neighboring picture by multiplying a scaling function with the exponential function; obtaining a filtered sample value of the current picture based on a sample value of the current picture and the weight of the at least one neighboring picture; and replacing the sample value of the current picture with the filtered sample value.

Claims (61)

1. A method performed by decoder, the method comprising:

receiving an encoded video bitstream comprising a current picture;

decoding the encoded current picture to generate a decoded picture;

applying a temporal filter to the decoded picture, the applying comprising:

obtaining an exponential function that includes a numerator with one or more factors corresponding to a motion estimation error and a denominator associated with a quantization parameter;

obtaining a weight of at least one neighboring picture by multiplying a scaling function with the exponential function;

obtaining a filtered sample value of the current picture based on a sample value of the current picture and the weight of the at least one neighboring picture; and

replacing the sample value of the current picture with the filtered sample value; and

reconstructing the decoded the current picture after the temporal filter is applied.

2. The method of claim 1 , wherein the obtaining the exponential function comprises computing an exponent of the exponential function as fixed-point values.

3. The method of claim 2 , wherein the computing the fixed-point values comprises representing the fixed-point values as 2 n bits, wherein n is an integer.

4. The method of claim 2 , wherein the computing the fixed-point values comprises scaling a floating-point value representation of a factor of the exponential function with a scaling factor.

5. The method of claim 1 , wherein the obtaining the exponential function comprises using at least one lookup table.

6. The method of claim 5 , wherein

the at least one lookup table comprises a first lookup table and a second lookup table, and

the first lookup table includes an integer part of the exponential function and the second lookup table includes a fractional part of the exponential function.

7. The method of claim 5 , wherein the at least one lookup table includes fixed-point values of the exponential function.

8. The method of claim 1 , further comprising:

converting factors of the scaling function to fixed-point representations,

wherein the obtaining the weight of the at least one neighboring picture comprises multiplying the fixed-point representations of the scaling function with the exponential function.

9. Claim 1 , further comprising:

performing a per-pixel loop for a plurality of iterations, each iteration of the plurality of iterations includes the obtaining the filtered sample value of the current picture, for a respective pixel from among pixels of the current picture and replacing the sample value of the current picture with the filtered sample value; and

converting, based on a factor from among the scaling function or the exponential function not varying for the pixels, the sample value to a fixed-point representation before performing the plurality of iterations of the per-pixel loop.

10. The method of claim 1 , wherein

the scaling function includes a numerator with at least one first factor and a denominator with at least one second factor,

the at least one first factor includes a noise level of the current picture, and

the at least one second factor includes a picture order count distance of the at least one neighboring picture from the current picture.

11. A system comprising:

at least one memory storing computer code; and

at least one processor configured to access the computer code and operate as instructed by the computer code, the computer code including:

receiving code configured to cause the at least one processor to receive an encoded video bitstream comprising a current picture;

decoding code configured to cause the at least one processor to decode the encoded current picture to generate a decoded picture;

temporal filter code that is configured to cause the at least one processor to apply a temporal filter to the decoded current picture, the temporal filter code comprising:

exponential function obtaining code that is configured to cause the at least one processor to obtain an exponential function that includes a numerator with one or more factors corresponding to a motion estimation error and a denominator associated with a quantization parameter;

weight obtaining code that is configured to cause the at least one processor to obtain a weight of at least one neighboring picture by multiplying a scaling function with the exponential function;

filtered sample value obtaining code that is configured to cause the at least one processor to obtain a filtered sample value of the current picture based on a sample value of the current picture and the weight of the at least one neighboring picture; and

sample value replacing code that is configured to cause the at least one processor to replace the sample value of the current picture with the filtered sample value; and

reconstructing code that is configured to cause the at least one processor to reconstruct the decoded current picture after the temporal filter is applied.

12. The system of claim 11 , wherein the exponential function obtaining code is configured to cause the at least one processor to compute an exponent of the exponential function as fixed-point values.

13. The system of claim 12 , wherein the exponential function obtaining code is configured to cause the at least one processor to represent the fixed-point values as 2 n bits, wherein n is an integer.

14. The system of claim 12 , wherein the exponential function obtaining code is configured to cause the at least one processor to scale a floating-point value representation of a factor of the exponential function with a scaling factor.

15. The system of claim 11 , wherein the exponential function obtaining code is configured to cause the at least one processor to obtain the exponential function by using at least one lookup table.

16. The system of claim 15 , wherein

the at least one lookup table comprises a first lookup table and a second lookup table, and

the first lookup table includes an integer part of the exponential function and the second lookup table includes a fractional part of the exponential function.

17. The system of claim 15 , wherein the at least one lookup table includes fixed-point values of the exponential function.

18. The system of claim 11 , wherein the computer code further includes:

converting code configured to cause the at least one processor to convert factors of the scaling function to fixed-point representations,

wherein the weight obtaining code is configured to cause the at least one processor to obtain the weight of the at least one neighboring picture by multiplying the fixed-point representations of the scaling function with the exponential function.

19. The system of claim 11 , wherein the computer code further includes:

loop code that comprises the filtered sample value obtaining code and that is configured to cause the at least one processor to perform a per-pixel loop for a plurality of iterations, each iteration of the plurality of iterations includes obtaining the filtered sample value of the current picture, for a respective pixel from among pixels of the current picture and replacing the sample value of the current picture with the filtered sample value; and

converting code that is configured to cause the at least one processor to convert, based on a factor from among the scaling function or the exponential function not varying for the pixels, the sample value to a fixed-point representation before the plurality of iterations of the per-pixel loop are performed.

20. A non-transitory computer-readable medium storing computer code that is configured to, when executed by at least one processor, cause the at least one processor to:

receive an encoded video bitstream comprising a current picture;

decode the encoded current picture to generate a decoded picture;

apply a temporal filter to the decoded picture, by:

obtaining an exponential function;

obtaining a weight of at least one neighboring picture by multiplying a scaling function with the exponential function that includes a numerator with one or more factors corresponding to a motion estimation error and a denominator associated with a quantization parameter;

obtaining a filtered sample value of the current picture based on a sample value of the current picture and the weight of the at least one neighboring picture; and

replacing the sample value of the current picture with the filtered sample value; and

reconstruct the decoded current picture after the temporal filter is applied.

Continuity (3)
Continuation 17240376 · Apr 26, 2021
Provisional Application 63034042 · Jun 3, 2020
Related Publication 20220132112A1 · Apr 28, 2022