IP Library Granted Patent US 11,438,633
Granted Patent B2
US 11,438,633 · App. 17/314,644 · Granted Sep 6, 2022

Temporal de-noising for video

Inventors: Jun Tian (Palo Alto, CA); Wen Gao (Palo Alto, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/85H04N19/136H04N19/182H04N19/186H04N19/597
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,438,633
App. No.
17/314,644
Granted
Sep 6, 2022
Kind
B2
Abstract

A method, computer program, and computer system is provided for video coding. Video data including one or more frames is received. A static background is estimated for each of the one or more frames based on a temporal average of the one or more frames. Pixels from among the one or more frames are identified as corresponding to the static background. Noise is removed in the static background based on the identified pixels.

Claims (40)

1. A method of video coding, executable by a processor, comprising:

receiving video data including one or more frames;

estimating a static background for each of the one or more frames based on a temporal average of the one or more frames;

identifying pixels from among the one or more frames as corresponding to the static background, wherein pixels from a current frame from among the one or more frames are identified based on a comparison of the estimated static background and pixel values of co-located pixels of future frames; and

removing noise in the static background based on the identified pixels.

2. The method of claim 1 , wherein pixels from among one or more frames are temporal averaged based on a color channel value, a color transform space, or a depth value.

3. The method of claim 1 , wherein estimating the static background comprises:

estimating a texture map and a depth map from the one or more frames based on temporally updating a background estimator; and

creating a synthesized background texture based on the estimated texture map and depth map.

4. The method of claim 1 , wherein the background is estimated based on a weighted average corresponding to recent frames having higher weight values.

5. The method of claim 1 , wherein the pixels are identified based on comparing co-located pixels of the current frame and the future frames from among the one or more frames with the estimated static background of one or more previous frames.

6. The method of claim 1 , wherein the noise is removed based on replace values of the identified pixels by an average value corresponding to the estimated static background.

7. The method of claim 1 , wherein the static background is estimated based on a sliding window.

8. A computer system for video coding, the computer system comprising:

one or more computer-readable non-transitory storage media configured to store computer program code; and

one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:

receiving code configured to cause the one or more computer processors to receive video data including one or more frames;

estimating code configured to cause the one or more computer processors to estimate a static background for each of the one or more frames based on a temporal average of the one or more frames;

identifying code configured to cause the one or more computer processors to identify pixels from among the one or more frames as corresponding to the static background, wherein pixels from a current frame from among the one or more frames are identified based on a comparison of the estimated static background and pixel values of co-located pixels of future frames; and

removing code configured to cause the one or more computer processors to remove noise in the static background based on the identified pixels.

9. The computer system of claim 8 , wherein pixels from among one or more frames are temporal averaged based on a color channel value, a color transform space, or a depth value.

10. The computer system of claim 8 , wherein the estimating code comprises:

estimating code configured to cause the one or more computer processors to estimate a texture map and a depth map from the one or more frames based on temporally updating a background estimator; and

creating code configured to cause the one or more computer processors to create a synthesized background texture based on the estimated texture map and depth map.

11. The computer system of claim 8 , wherein the background is estimated based on a weighted average corresponding to recent frames having higher weight values.

12. The computer system of claim 8 , wherein the pixels are identified based on comparing co-located pixels of the current frame and the future frames from among the one or more frames with the estimated static background of one or more previous frames.

13. The computer system of claim 8 , wherein the noise is removed based on replace values of the identified pixels by an average value corresponding to the estimated static background.

14. The computer system of claim 8 , wherein the static background is estimated based on a sliding window.

15. A non-transitory computer readable medium having stored thereon a computer program for video coding, the computer program configured to cause one or more computer processors to:

receive video data including one or more frames;

estimate a static background for each of the one or more frames based on a temporal average of the one or more frames;

identify pixels from among the one or more frames as corresponding to the static background, wherein pixels from a current frame from among the one or more frames are identified based on a comparison of the estimated static background and pixel values of co-located pixels of future frames; and

remove noise in the static background based on the identified pixels.

16. The computer readable medium of claim 15 , wherein pixels from among one or more frames are temporal averaged based on a color channel value, a color transform space, or a depth value.

17. The computer readable medium of claim 15 , wherein the computer program is further configured to cause one or more computer processors to:

estimate a texture map and a depth map from the one or more frames based on temporally updating a background estimator; and

create a synthesized background texture based on the estimated texture map and depth map.

18. The computer readable medium of claim 15 , wherein the background is estimated based on a weighted average corresponding to recent frames having higher weight values.

19. The computer readable medium of claim 15 , wherein the pixels are identified based on comparing co-located pixels of the current frame and the future frames from among the one or more frames with the estimated static background of one or more previous frames.

20. The computer readable medium of claim 15 , wherein the noise is removed based on replace values of the identified pixels by an average value corresponding to the estimated static background.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2021
From: TIAN, JUN; GAO, WEN; LIU, SHAN
To: TENCENT AMERICA LLC
Reel/Frame 056173/0052 →
Continuity (4)
Provisional Application 63036624 · Jun 9, 2020
Provisional Application 63036181 · Jun 8, 2020
Provisional Application 63035252 · Jun 5, 2020
Related Publication 20210385503A1 · Dec 9, 2021