IP Library Granted Patent US 11,677,938
Granted Patent B2
US 11,677,938 · App. 12/112,440 · Granted Jun 13, 2023

Method to reduce contour artifacts on recursive temporal filters

Inventor: Siu-Wai Wu (San Diego, CA)
Assignee: ARRIS Enterprises LLC
H04N19/117H04N19/80H04N19/86
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,677,938
App. No.
12/112,440
Granted
Jun 13, 2023
Kind
B2
Abstract

A method and encoding system for reducing artifacts in a digital video sequence of image frames. The method acquires a current frame of the digital video sequence, and retrieves a previous frame of the digital video sequence from a frame delay. The method applies a recursive temporal filter to the current frame and the previous frame to generate a filtered frame. The method then applies a mixer to the current frame and the filtered frame to generate an output frame. The method stores the output frame in the frame delay.

Claims (52)

1. A method for reducing artifacts in a digital video sequence of image frames, comprising:

in a leaky motion adaptive recursive temporal filter comprising an adaptive IIR filter kernel, an adaptive mixer, and a frame delay:

acquiring a current frame of the digital video sequence and comprising current frame pixel values;

retrieving a previous frame of the digital video sequence from a frame delay;

in the adaptive IIR filter kernel, applying a recursive temporal filter to the current frame pixel values and the previous frame to generate a filtered frame comprising filtered frame pixel values;

in the adaptive mixer, receiving a first input comprising the current frame pixel values, and receiving a second input comprising the filtered frame pixel values;

allowing noise to leak through the leaky motion adaptive recursive temporal filter by applying the adaptive mixer to the current frame and the filtered frame to generate an output frame, based at least on a non-linear leak function, wherein the applying comprises calculating output pixels in the adaptive mixer for the current frame by applying the current frame pixel values and the filtered frame pixel values to the non-linear leak function; and

storing the output frame in the frame delay.

2. The method of claim 1 , wherein the current frame includes input pixels, the previous frame includes previous output pixels, and the filtered frame includes filtered pixels, the applying of the recursive temporal filter further comprising:

computing a pixel difference between each input pixel and each previous output pixel at a corresponding location; and

computing a pixel value for each filtered pixel based on the pixel difference.

3. The method of claim 2 , wherein when the pixel difference is above an upper threshold, the pixel value is equal to a value of the input pixel, and when the pixel difference is below the upper threshold, the pixel value is a sum of the value of the input pixel and an adjustment based on the pixel difference.

4. The method of claim 3 , wherein when the pixel difference is below a lower threshold, the pixel value is equal to the value of the input pixel.

5. The method of claim 1 , wherein the current frame includes input pixels, the filtered frame includes filtered pixels, and the output frame includes output pixels, the applying of the mixer further comprising:

computing a pixel difference between each filtered pixel and each input pixel at a corresponding location; and

computing a pixel value for each output pixel based on the pixel difference.

6. The method of claim 5 , wherein when the pixel difference is above an upper threshold, the pixel value is equal to a value of the filtered pixel, and when the pixel difference is below the upper threshold, the pixel value is a sum of the value of the input pixel and an adjustment based on the pixel difference.

7. The method of claim 1 , wherein the mixer preserves the processing performed by the recursive temporal filter.

8. The method of claim 1 , wherein the recursive temporal filter is a motion adaptive recursive temporal filter, and the mixer is an adaptive mixer.

9. The method of claim 1 , wherein the recursive temporal filter is a motion compensated recursive temporal filter, and the mixer is a compensated mixer.

10. An encoding system for reducing artifacts in a digital video sequence of image frames, comprising a leaky motion adaptive recursive temporal filter, the leaky motion adaptive recursive temporal filter comprising:

a frame delay for storing a previous frame of the digital video sequence;

a recursive temporal filter comprising an adaptive IIR filter kernel for generating a filtered frame by filtering first pixel values of a current frame of the digital video sequence using previous pixel values of the previous frame; and

an adaptive mixer for allowing noise to leak through the leaky motion adaptive recursive temporal filter by generating an output frame from the current frame and the filtered frame, based at least on a non-linear leak function, by calculating output pixels in the adaptive mixer for the current frame,

wherein the adaptive mixer is configured to receive a first input comprising the first pixel values, and further configured to receive a second input comprising second pixel values based on the output from the adaptive IIR filter kernel, and

wherein the output pixels are calculated in the adaptive mixer for the current frame by applying the first input and the second input to the non-linear leak function.

11. The encoding system of claim 10 , wherein the current frame includes input pixels, the previous frame includes previous output pixels, and the filtered frame includes filtered pixels, the recursive temporal filter further configured to:

compute a pixel difference between each input pixel and each previous output pixel at a corresponding location; and

compute a pixel value for each filtered pixel based on the pixel difference.

12. The encoding system of claim 11 , wherein when the pixel difference is above an upper threshold, the pixel value is equal to a value of the input pixel, and when the pixel difference is below the upper threshold, the pixel value is a sum of the value of the input pixel and an adjustment based on the pixel difference.

13. The encoding system of claim 12 , wherein when the pixel difference is below a lower threshold, the pixel value is equal to the value of the input pixel.

14. The encoding system of claim 10 , wherein the current frame includes input pixels, the filtered frame includes filtered pixels, and the output frame includes output pixels, the mixer configured to:

compute a pixel difference between each filtered pixel and each input pixel at a corresponding location; and

compute a pixel value for each output pixel based on the pixel difference.

15. The encoding system of claim 14 , wherein when the pixel difference is above an upper threshold, the pixel value is equal to a value of the filtered pixel, and when the pixel difference is below the upper threshold, the pixel value is a sum of the value of the input pixel and an adjustment based on the pixel difference.

16. The encoding system of claim 10 , wherein the mixer preserves the processing performed by the recursive temporal filter.

17. The encoding system of claim 10 , wherein the recursive temporal filter is a motion adaptive recursive temporal filter, and the mixer is an adaptive mixer.

18. The encoding system of claim 10 , wherein the recursive temporal filter is a motion compensated recursive temporal filter, and the mixer is a compensated mixer.

19. A method for reducing artifacts in a digital video sequence of image frames, comprising:

in a leaky motion adaptive recursive temporal filter comprising an adaptive IIR filter kernel, an adaptive mixer, and a frame delay:

acquiring a current frame of the digital video sequence, the current frame including input pixels;

retrieving a previous frame of the digital video sequence from a frame delay, the previous frame including previous output pixels;

in the adaptive IIR filter kernel, applying a recursive temporal filter to the current frame and the previous frame to generate a filtered frame, the filtered frame including filtered pixels, the recursive temporal filter configured to:

compute a pixel difference between each input pixel and each previous output pixel at a corresponding location; and

compute a pixel value for each output pixel based on the pixel difference,

wherein when the pixel difference is above an upper threshold or below a lower threshold, the pixel value is equal to a value of the input pixel, and when the pixel difference is below the upper threshold and above the lower threshold, the pixel value is a sum of the value of the input pixel and an adjustment based on the pixel difference;

allowing noise to leak through the leaky motion adaptive recursive temporal filter by applying the adaptive mixer to the current frame and the filtered frame to generate an output frame, based at least on a non-linear leak function, by calculating output pixels in the adaptive mixer for the current frame,

wherein the adaptive mixer is configured to receive a first input comprising the input pixels from the current frame, and further configured to receive a second input comprising output pixels output from the adaptive IIR filter kernel, and

wherein the output pixels are calculated in the adaptive mixer for the current frame by applying the first input and the second input to the non-linear leak function; and

storing the output frame in the frame delay.

20. The method of claim 19 , wherein the recursive temporal filter is a motion adaptive recursive temporal filter.

21. The method of claim 19 , wherein the recursive temporal filter is a motion compensated recursive temporal filter, and the mixer is a compensated mixer.

Assignments (13)
SECURITY INTEREST Recorded Apr 8, 2026
From: ARRIS ENTERPRISES LLC; RUCKUS IP HOLDINGS LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 075476/0814 →
RELEASE OF SECURITY INTEREST AT REEL/FRAME 049905/0504 Recorded Dec 19, 2024
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: ARRIS ENTERPRISES LLC (F/K/A ARRIS ENTERPRISES, INC.); ARRIS TECHNOLOGY, INC.; ARRIS SOLUTIONS, INC.; COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; RUCKUS WIRELESS, LLC (F/K/A RUCKUS WIRELESS, INC.)
Reel/Frame 071477/0255 →
SECURITY INTEREST Recorded Dec 17, 2024
From: ARRIS ENTERPRISES LLC; COMMSCOPE TECHNOLOGIES LLC; COMMSCOPE INC., OF NORTH CAROLINA; OUTDOOR WIRELESS NETWORKS LLC; RUCKUS IP HOLDINGS LLC
To: APOLLO ADMINISTRATIVE AGENCY LLC
Reel/Frame 069889/0114 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2022
From: ARRIS TECHNOLOGY, INC.
To: ARRIS ENTERPRISES, INC.
Reel/Frame 060791/0583 →
PATENT SECURITY AGREEMENT Recorded Jul 3, 2019
From: ARRIS ENTERPRISES LLC
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 049820/0495 →
ABL SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049892/0396 →
TERM LOAN SECURITY AGREEMENT Recorded Jul 3, 2019
From: COMMSCOPE, INC. OF NORTH CAROLINA; COMMSCOPE TECHNOLOGIES LLC; ARRIS ENTERPRISES LLC; ARRIS TECHNOLOGY, INC.; RUCKUS WIRELESS, INC.; ARRIS SOLUTIONS, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 049905/0504 →
CHANGE OF NAME Recorded Jul 2, 2019
From: ARRIS ENTERPRISES, INC.
To: ARRIS ENTERPRISES LLC
Reel/Frame 049649/0062 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Apr 8, 2019
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS SOLUTIONS, INC.; ARRIS KOREA, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS, INC.; BIG BAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; 4HOME, INC.; ACADIA AIC, INC.; AEROCAST, INC.; BROADBUS TECHNOLOGIES, INC.; GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT AUTHORIZATION SERVICES, INC.; GENERAL INSTRUMENT INTERNATIONAL HOLDINGS, INC.; IMEDIA CORPORATION; JERROLD DC RADIO, INC.; LEAPSTONE SYSTEMS, INC.; MODULUS VIDEO, INC.; MOTOROLA WIRELINE NETWORKS, INC.; NETOPIA, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; QUANTUM BRIDGE COMMUNICATIONS, INC.; SETJAM, INC.; SUNUP DESIGN SYSTEMS, INC.; UCENTRIC SYSTEMS, INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC; CCE SOFTWARE LLC; THE GI REALTY TRUST 1996
Reel/Frame 048825/0294 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2015
From: ARRIS TECHNOLOGY, INC
To: ARRIS ENTERPRISES, INC.
Reel/Frame 037328/0341 →
MERGER AND CHANGE OF NAME Recorded Mar 10, 2015
From: GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT CORPORATION
To: ARRIS TECHNOLOGY, INC.
Reel/Frame 035176/0620 →
SECURITY AGREEMENT Recorded May 28, 2013
From: ARRIS GROUP, INC.; ARRIS ENTERPRISES, INC.; ARRIS SOLUTIONS, INC.; ARRIS KOREA, INC.; ARRIS HOLDINGS CORP. OF ILLINOIS; BIGBAND NETWORKS, INC.; TEXSCAN CORPORATION; POWER GUARD, INC.; 4HOME, INC.; ACADIA AIC, INC.; AEROCAST, INC.; BROADBUS TECHNOLOGIES, INC.; GENERAL INSTRUMENT CORPORATION; GENERAL INSTRUMENT AUTHORIZATION SERVICES, INC.; GENERAL INSTRUMENT INTERNATIONAL HOLDINGS, INC.; IMEDIA CORPORATION; JERROLD DC RADIO, INC.; LEAPSTONE SYSTEMS, INC.; MODULUS VIDEO, INC.; MOTOROLA WIRELINE NETWORKS, INC.; NETOPIA, INC.; NEXTLEVEL SYSTEMS (PUERTO RICO), INC.; QUANTUM BRIDGE COMMUNICATIONS, INC.; SETJAM, INC.; SUNUP DESIGN SYSTEMS, INC.; UCENTRIC SYSTEMS, INC.; GIC INTERNATIONAL HOLDCO LLC; GIC INTERNATIONAL CAPITAL LLC; CCE SOFTWARE LLC; THE GI REALTY TRUST 1996
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 030498/0023 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2008
From: WU, SIU-WAI
To: GENERAL INSTRUMENT CORPORATION
Reel/Frame 020982/0360 →
Continuity (1)
Related Publication 20090273716A1 · Nov 5, 2009